Clyde DeSouza, VR Filmmaker and Tech Evangelist
Clyde DeSouza
We are honored to interview Clyde DeSouza today, one of our top creators. Clyde is a VR Filmmaker, Speaker, Tech Evangelist and writer. His non-fiction book, Think in 3D is well known with 3D filmmakers in Hollywood. His Scifi novel, MAYA, which his first VR film Dirrogate was based on, was published by Penguin | Randomhouse in 2015. Currently, he is working on his second cinematic VR film.
VeeR: Do you consider yourself more of an author or a VR video artist?
Clyde: I love a Q n A where the first question itself makes you think! I hadn’t asked myself this one before, but now, come to think of it, I’d say I’m more of a storyteller with a skill that I’m still honing, to bring stories in my head to life. While I believe there’s no replacing a book – because it allows for a close collaboration between author and reader to visualize a story’s world. I’m thankful now with the medium of Virtual Reality, it allows me to paint the story world and immerse the reader or viewer, right into it.
VeeR: It’s really amazing that you came up with the inspirations both for your novel and the film, we wonder which one came first?
Clyde: The inspiration to tell the story came first. I wanted to write it in hard science fiction form, which by itself I knew I’d be limiting my audience reach. While writing the pages, I couldn’t help visualizing them as scenes so vividly, I knew I had to try to bring them to life. Co-incidentally at that time, in a garage on the other side of the world, a teenager was building what would become the first popular VR headset – the oculus rift.
Maya by Clyde DeSouza
VeeR: Can you share with us some unforgettable stories on the production of Dirrogate, such as formation of the concept, site selecting and the production cycle of different parts?
Clyde: The formation of the story came when I used to visit food courts in Dubai and Singapore while writing some of the scenes in the book. While eyes-dropping on conversations of today’s mobile generation and their affinity to communicating via mobile phone screens, their world seemed to be full of snapchatting, facetime and instagramming the world around them – Digitizing as it were, the real world. These “Digital Breadcrumbs” that they were leaving everywhere is what formed the inspiration for the film.
During location scouting for the VR film, I had one place that I have briefly visited before, but I was overwhelmed when I finally visited it for the shoot – Study Street in Mumbai. This is a location described in the novel, and now seeing the scene described in the pages literally there – the young kids under streetlights, studying; it was surreal in a way!
There is one shot of the Mumbai Airport that I managed to get, guerilla filmmaking style. I was to board a flight to Dubai and had arrived early in the morning at the airport. The waiting area was deserted and I quickly placed my home-built stereoscopic 3D camera; two Ricoh thetas (first generation that did not shoot video) and took a 3D shot. I didn’t think I’d end up using it in the film, but after some curing of the scene and adding “morning fog” in after-effects to the window area, it ended up being in the final VR film.
VeeR: Please share some thoughts and experiences on playwriting/composing scripts, especially on VR videos?
Clyde: To me when composing scripts that will end up as VR stories, I’d say the devil and pleasure are in the details described in the scenes (and thus captured). For instance, I’d like scripts to dwell more on human interaction. This will naturally allow for close-ups of actors in VR. Watching the Human form and facial expressions in VR is a powerful and compelling experience.
Also I’d like to see scripts that bring the audience back to a familiar scene in the story again and again – this helps establish the VR world in the subconscious of the viewer, giving them in essence, a Déjà vu feeling which helps trick the viewer’s mind into thinking they “know” the places in the story.
VeeR: What VFX software did you use for editing VR videos?
Clyde: The VR film was done early 2015 – there were no VR capable software editing tools back then, much less stereoscopic (3D) VR tools! So everything was done by hand in AfterEffects. Software such as iClone was also used for VFX. It took a great deal of effort: Rendering out a single frame in Aftereffects, checking it in the Oculus Rift headset then re-adjusting, re-exporting and checking. If that wasn’t bad enough, this was being done in 3D 360!
VeeR: What VFX software and filming equipment would you recommend?
Mettle | 360/VR + 3D Plug-ins for Adobe Software
Clyde: I ended up advising and beta-testing for Mettle (makers of the excellent Skybox Studio plugins for aftereffects), and then after the release of Dirrogate VR, companies like Kolor released 3D viewing plugins for Adobe Premiere. Today there are a lot of solutions to ease the life of a VR filmmaker. Nuke and it’s dedicated plugin, CaraVR, being the royalty of VFX software for VR.
Today Adobe premiere seems to be the leader in video editing VR footage.
VeeR: How do you help the audience to adapt their roles in VR video, and how did you guide/direct them?
The guidelines of directing for live theater plays certainly apply. Directional sound is a powerful tool that a VR filmmaker can use. Then there are the usual tools that the master’s in older films used – Lighting. I detest today’s practice of normal filmmakers who overuse rack focus to draw attention. Fortunately you can’t really use rack-focus in a VR film or you run the risk of giving the audience a headache.
VeeR: How do you maximize immersion for the audience?
Clyde: Dirrogate VR did not aim at maximum immersion, since it was based on a formula I’d developed of brining the traditional Graphic Novel into VR. People stepped into the panels of the VR graphic novel and the scene then enveloped them in stereoscopic 360. But, getting back to your question – today I’d say to maximize immersion for the audience, avoid rapid montage like cuts and please, use stereoscopic 360 so the scale of the VR world and it’s inhabitants is correct.
VeeR: Any suggestions to minimize motion sickness?
Clyde: By not accelerating or decelerating the camera! Panning the camera can cause severe motion sickness, and Hollywood is guilty of being ignorant in this. An example is the short VR film done for Suicide Squad. The camera pans at some points and it goes against the will of the viewer.
VeeR: The biggest challenge you encountered when making VR videos and what was your solution?
Clyde: For Dirrogate VR, the biggest challenge was having to work with an “already stitched” equirectangular image coming of the camera which was a first generation Ricoh Theta. For stereoscopic 360, I needed to cut the image from the left and right Ricoh thetas (they were placed side by side) and swap them over for correct stereo 3D on the rear 180 degrees.
If I had access to the raw circular images, then that nightmare would not have been there. I solved it by manually using tools in AfterEffects (masks, blends, warps).
VeeR: We are coming to the end of the interview. Share one tip with our new creators?
Please shoot in stereoscopic 360 if you’re looking at making true “VR films”. Shooting with todays typical Gopro rigs just gives you what I call QTVR (Quick time Vr) which is what was done in the 1990’s only your now viewing it in a VR headset instead of on a PC monitor. The VR world is all wrong in 2D 360.
People look 20 feet tall, Car wheels look like giant ferris wheels – If you want to immerse your audience in a VR world and the VR world is not being created in a Game Engine, the very least you need to do is shoot in stereoscopic 360 video.
Camera manufacturers are as guilty of neglecting this area. There’s so many kickstarter and Indigogo “VR cameras” coming out there, but few are looking at creating synced, stereoscopic 360 cameras. It’s a lot harder to create stereoscopic VR films. You need to spend time in postproduction, but then again it does not mean you take the quick and dirty way out by shooting 2D 360 and calling it VR.
If you’re shooting a 360 video of a flight through the Grand Canyon or over the Great Wall of China, then by all means shoot in 2D 360 but for a VR film with people and a narrative storyline, you need stereoscopic 360.