For the Unity side requirements mentioned above, the first thing to verify is to split the image so that the left and right eyes can display different content, which requires the help of Shader.
Why Shader
The new version of CardBoard SDK has access to UnityXR, while modern UnityXR uses SinglePassStereoRendering Rendering, also known as single channel stereo rendering.
Single channel stereo rendering can improve performance, but it also means that the camera layer is layered, and the way of looking at one patch with one eye is not feasible. So we need to write a Shader to make the same object display different maps when rendering the left and right eyes.
We use the following image to verify image clipping, which is a frame taken from VR video.
Modeling and UV
Let's not talk about Shader first, but about image display.
People who often watch VR videos can see at a glance that the picture above is a left and right eye image. The left eye looks at the left, and the right eye looks at the right. The texture width height ratio of each eye is 1:1.
Careful you may find that the image edge has arc black edges, and the table in the picture is curved, because the texture is a 180 degree hemispherical arc texture.
Of course, even the hemispherical surface, because the two eyes see different pictures, the human eye still has a three-dimensional sense, This is different from panoramic video 。
Open Blender to generate a sphere mesh:
The user views the content at the center of the sphere, so all faces are selected, Flip Normal , make all face normals face the center of the ball:
For the video above, we only need half a ball, so Delete the vertex of half a ball ：
After deleting the hemisphere, start to adjust the UVs. Before adjustment, your UVs should be like this (this blog deletes the former hemisphere, causing the left and right directions to be opposite, that is, the order of visible colors in the figure is opposite):
We need to make the image fill the sphere from left to right, so when dividing UVs, we need to make all faces closely arranged, and fill the texture from left to right. At the same time, note that the UV tiling sequence should be from left to right. If it is reversed, it needs to be corrected:
Finally, rotate the model 90 degrees around the x-axis, apply the transformation, and then rotate 90 degrees around the x-axis to export FBX to Unity.
Shader
For SinglePassStereoRendering, that is, single channel stereo rendering, there is only one camera available in the scene.
In order for the left and right eyes to see different images, we need to know Is the left eye or the right eye currently being rendered 。 Unity takes this situation into consideration and provides unity_StereoEyeIndex The parameter identifies the currently rendered eye.
Our image is cut horizontally, so we need to modify the x of the UV:
//Modified from Google VR/Demos/VideoDemo StereoShader, adding uv. x cutting, left/right/left judgment v2f vert (appdata_base v) { v2f o; o.pos = GvrUnityObjectToClipPos(v.vertex); o.uv = TRANSFORM_TEX (v.texcoord, _MainTex); o.uv.x *= 0.5f; If (unity_StereoEyeIndex==_IsLeftRightNormal) {//_IsLeftRightNormal: 0 Right left image 1 Left right image o.uv.x += 0.5f; } return o; }
Assign this Shader to the material of the hemispherical model, and set the above map for this material:
modify _IsLeftRightNormal The left and right eye images can be previewed in the editor (because unity_StereoEyeIndex in the editor is a fixed value of 0):
Put the camera at the center of the ball in the editor, and you can see the normal image:
Play video
The principle of playing video is very simple. The playing plug-in decodes, and then assigns the texture decoded by the plug-in to the material of the shader specified above, which completes the left/right/left type video display.
other
This article only describes the display of left/right/left 180 degree spherical VR video, and the upper/lower/lower/upper VR video is similar.
Next, we need to write Android Java code to solve the SMB to HTTP problem.
