Virtual Reality Against All Odds

In your opinion, when did the industry become interested in virtual reality ?

Céline Tricart : I believe it was the crowdfunding campaign for the launch of the Oculus Rift in 2012. What was significant about that was that the general public showed their massive interest in the technology (the campaign largely overcame its original target), and it wasn’t just imposed on them by one of the major brands in the industry. People just wanted a good VR headset, mostly to play games with. But now the appeal has broadened far beyond the original niche-market of hardcore gamers.
The industry’s giants, such as Google and Facebook, are massively investing in VR, especially for applications linked to social media networks (such as Facebook live 360 events). Simultaneously, the first documentaries and exhibits are appearing in film festivals, and even Spielberg is shooting his first VR film as we speak…

How did you get involved in VR ?

C.T. : Personally, I began by tinkering around by myself, filming a few 360° shots with a single digital camera, and then by experimenting with how to “stitch” them together. Then, I started using the Ricoh Theta. The Theta is a tiny little 360° camera that allows you to quickly iterate and educate yourself in the language of virtual reality. My first official VR project was filmed in 2015 using the prototype of the Nokia Ozo, and is called Marriage Equality. I co-directed it with Steve Schklair, it’s a hybrid documentary about the day when the Supreme Court of the United States ruled in favor of same-sex marriage in all 50 states. Even though the film seems to capture this important moment in the country’s history in a documentary manner, we actually staged everything with actors.

What did you learn from making that first movie ?

C.T. : We learnt a lot from making the film. One of the main issues was that when you decide to make a narrative film in virtual reality, you have to decide how you’ll guide the audience through the story. That doesn’t mean that you have to control things 100%, but rather you have to exercise some influence. For example, in “Marriage Equality”, we decided to use the character of a journalist who helps the audience follow the narration within the 360° sphere. At first, we placed the actors at least 2 meters away from the camera, as instructed to do so by the Nokia engineers because of “stitch” problems that parallax causes when elements come very close to the camera.
In fact, because these cameras capture images with fish-eye lenses so that the sphere can be assembled afterwards, the relationship to perspective is very different than when you’re filming with a traditional camera. You get a feeling of great emptiness when you follow the 2-meter-minimum rule, and you can’t really project yourself into the situation. Everything looks very far away, threatening the sense of presence we care so much about in VR. After reviewing a few takes, we decided to move everyone closer to the camera, around 1,3 meter away, and we were very pleased with the result. What’s most important for me is telling a story, and I think we shouldn’t hesitate to break the rules when needed, but only when we fully understand the consequences.

Even if you can’t correctly splice the images together ?

C.T. : It’s a trade-off. It’s also good to remember the lessons learned from 3D, where a lot of films were shot without following the minimum technical specifications. The general public knows the difference between good and bad 3D, maybe not on a technical level, but they can feel it. If their heads hurt, it’s bad 3D. When they can’t see the difference with or without the 3D glasses, it’s also bad 3D. They get it, and that’s why we shouldn’t make the same mistakes in VR.
Amongst the issues that we’ve already identified on current or past productions is first of all the difficulty of moving the camera in VR. A clue conflict arises in the brain between the visual and the vestibular (inner ear) systems when the entire VR sphere moves but the viewer’s inner ears don’t detect acceleration (as the viewer is indeed sitting with a headset on his/her head, and not moving).
This causes motion sickness. In order to move the camera in VR, slow travellings are preferred but more importantly, the motion must be as linear as possible, with no change of speed – it’s actually acceleration, not movement, that causes this terrible motion sickness. Also, the horizon must remain leveled, do not tilt or roll the camera (unless you definitely want to cause seasickness !). And of course, do not pan the camera left and right as the purpose of VR is to immerse the audience into a 360° sphere and let them decide where to look.
If these general rules are understood, then I think it is possible to move the camera in VR in a comfortable and meaningful manner.

Aren’t you frustrated as a director of photography that you can’t use the camera as you once used to ?

C.T. : It is true that the cinematographer’s responsibilities are different in VR. First of all, the shot list (if any) is completely different in VR. We don’t do establishing shots followed by medium shots and close-ups, with a couple of inserts here and there. Most of the time, one scene equals one location equals one shot, just like in theater. Also, we rarely choose our lenses as the all-in-one VR cameras come with their own, usually fish-eyes. You don’t get to pull focus (the lenses are set at the hyperfocal), and setting the stop is usually a combination of ISO and shutter speed adjustment.
The most important thing for VR DPs is to choose the right camera and the right camera support depending on the story and locations. I often take a few different systems with me if I have to shoot in normal and low light condition, if I need aerials, if I know the weather will be challenging, etc. Afterwards, I try to take advantage of the location as best I can as it is very important storytelling device in VR, much more than in traditional filmmaking. Working closely with the art director and the production designer is fundamental. In my opinion, the work of a VR cinematographer is more akin to visual co-directing.

Can you tell us a bit about Under the Canopy, the documentary you shot last year in the Amazon ?

C.T. : rainforest. It was directed by Patrick Meegan, who regularly works with the company Jaunt, the manufacturer of the Jaunt One camera and a distributor of VR content via its online platform Jauntvr.com. The movie is 11 minutes long and allows the viewer to discover the rainforest from the tips of the trees to the ground using various systems of aerial photography, such as cables and even drones.

What equipment did you use to shoot this film ?

C.T. : The majority of the film was shot using the aforementioned Jaunt One camera, pulled by cables attached to the trees. This camera is composed of 24 sensors (16 around the horizon, 4 on top and 4 on the bottom) which makes possible a final stitch up to 8K. An “optical flow” algorithm generates dimensionality as the same element in the 360° sphere is seen by multiple sensors. The maximum frame rate is 120 fps, which is great because it opens up a whole new world of possibilities for VR filmmakers : affordable slow motion in 360°.

What was your reference frame rate ?

C.T. : Common industry knowledge is that 60 frames per second (screened at 60) is the reference frame rate for VR. It provides a better experience in a VR headset, especially when the viewer turns his or her head quickly. Unfortunately most current VR cameras can only record up to 30 fps.

Aren’t you afraid that VR might just be a flash in the pan, like 3D was for cinema ?

C.T. : Ah…, well, first, let me correct you ! 3D television is definitely dead, but 3D cinema isn’t dead at all ! Each year, millions of dollars of revenue are generated by 3D blockbusters, and even though 3D films aren’t being produced in France any longer, they are still regularly made in Hollywood.
As for VR, I think that it will survive no matter what in the game industry. It all depends on when and how manufacturers will solve the traditional chicken-and-egg problem of selling headsets to the general public with no real high-quality content pool to support it. We are still in a nascent industry which relies mostly on early adopters.
Samsung made a good move in transforming their galaxy smartphones into “affordable” VR systems, and Google too in open-sourcing the design of the Google Cardboard VR viewer. Game consoles such as the Sony PS4 and its VR headset are also heading in that direction… Hopefully soon everyone will be able to watch VR content at home. We’re also witnessing the return of the traditional video game arcades, but in VR, where the general public can have access to high-end VR equipment like room-scale tracking, haptic body suits etc.

Unfortunately, the current state of the VR technology is not yet able to offer a true visually realistic experience. The standard 4K file format provides only one-fourth of the definition when watched on a headset. In order to improve that, we’d have to use an 8K or 16K format, which is currently prohibitively expensive and challenging.
No one is in charge of setting standards in this field, so we’re constantly confronted with visual differences between the various headsets, with different levels of black, different colorimetric spaces, different dynamic ranges.
In short, we’re still in pioneering stage, in both the technology and the storytelling aspect of VR. It’s indeed challenging, but this is why I find it so exciting !

(Interview conducted by François Reumont on behalf of the AFC, and traducted from French by Alexander Baron-Raiffe)

Watch the short film Under the Canopy

Watch the “Making Of”