How to Use Unity 2018.1 to Capture Stereoscopic 360 Images and Videos

1 Feb , 2018 Academy Mina Bradley

How to Use Unity 2018.1 to Capture Stereoscopic 360 Images and Videos

In a January 26th blog update, Unity announced the latest features released with Unity 2018.1 that enables stereo image and video capture natively in Unity.

The feature was based on Google’s Omni-Directional Stereo (ODS) technology, which supports rendering stereo cubemaps within Unity’s graphics pipeline on both Editor and on PC standalone player. These stereo cubemaps can then be converted to stereo equirectangular maps for 360 media players.

The blog briefly explained how to use this new feature and included APIs for the capture and convert function:

This tutorial aims at breaking down the steps further. Here we’re using the Example Project when installing Unity, which you can select as a component.

Now trying this feature out:

1. Start 3 RenderTexture Assets in your Project, named as follows:

Set the following parameters for cubemap_left and cubemap_right:

Unity-2018-2

Now configure equirect like this:

Unity-2018-3

2. Find the MainCamera in your set and attach the following script as can be found in the API:

using UnityEngine;
using UnityEngine.Rendering;

//attach this script to your camera object
public class CreateStereoCubemaps : MonoBehaviour
{
public RenderTexture cubemapLeft;
public RenderTexture cubemapRight;
public RenderTexture equirect;
public bool renderStereo = true;
public float stereoSeparation = 0.064f;

void LateUpdate()
{
Camera cam = GetComponent<Camera>();

if (cam == null)
{
cam = GetComponentInParent<Camera>();
}

if (cam == null)
{
Debug.Log("stereo 360 capture node has no camera or parent camera");
}

if (renderStereo)
{
cam.stereoSeparation = stereoSeparation;
cam.RenderToCubemap(cubemapLeft, 63, Camera.MonoOrStereoscopicEye.Left);
cam.RenderToCubemap(cubemapRight, 63, Camera.MonoOrStereoscopicEye.Right);
}
else
{
cam.RenderToCubemap(cubemapLeft, 63, Camera.MonoOrStereoscopicEye.Mono);
}

//optional: convert cubemaps to equirect

if (equirect == null)
{
return;
}

if (renderStereo)
{
cubemapLeft.ConvertToEquirect(equirect, Camera.MonoOrStereoscopicEye.Left);
cubemapRight.ConvertToEquirect(equirect, Camera.MonoOrStereoscopicEye.Right);
}
else
{
cubemapLeft.ConvertToEquirect(equirect, Camera.MonoOrStereoscopicEye.Mono);
}
}
}

3. Select MainCamera and use the RenderTexture Asset in Inspector to create stereo cubemaps

Unity-2018-4

4. Run the script, and now Equirect has already recorded MainCamera's stereo info

We created a new UI element, RawImage to monitor the output of Equirect, as seen in the upper left corner of this screen:

Remember to drag Equirect to the Texture attribute of RawImage.

All done, now we can play the whole scene!

Unity-2018-5

As for exporting media, Unity recommended the Unity Frame Recorder plug-in, which outputs RenderTexture into single frames that can be converted into 360-degree videos in commonly-used video editing software.

There’s also the Unity’s 360 Video Playback plug-in for playing panoramic videos in Unity.

The blog post also mentioned the 360 Stereo Capture option in XR Settings, which controls if your Unity projects support 360-degree shaders. This gives a lot of freedom to game developers’ imagination, as they can now enable players of their 3D/VR games to record in-game first-person 360 videos.

Unity-2018-6

This feature might not work perfectly with some developers’ custom shaders for now, and might result in glitchy capture in 360 degrees, and we are looking forward to improvements in that regard.

Let us know in the comments below if you have any questions about this feature, and subscribe to the VeeR VR Blog for more updates like this!

Follow us on Twitter, Facebook and Instagram @letsveer and tag us in your work!

Mina Bradley

Mina is a digital marketing strategist, content marketer, and editor-in-chief of the VeeR VR Blog. She specializes in feature shorts, tech news and product reviews.

For inquiries and requests, please contact mina@veer.tv.


1 Response

  1. Bobby Rajesh Malhotra says:

    thanx very much for this tutorial, i have a maybe noobish question: is there a way to convert a 6 sided cubemap to hdr?

Leave a Reply

Your email address will not be published. Required fields are marked *