Spectre Studios - Using mocap for previs

At Rokoko we strive to change the dynamics of how creators work with digital motion: creativity should know no boundaries, and motion should be nothing less than 100% intuitive. Last week we had the opportunity to connect with Rick Pearce, cofounder and creative director of Spectre VR, and a user of our Smartsuit Pro. During our latest chat he offered us some further insight into how he’s been using the Smartsuit in Spectre VR’s latest projects and pushing boundaries his own way.

Hey Rick, what is Spectre VR, and what kind of background do you come from?

Here at Spectre VR, we harness the power of extended reality in our work in the film and gaming industries. eXtended Reality (XR) encompasses all technology-mediated experiences that combine digital and biological realities. We enable content creation for virtual reality (VR), mixed reality (MR), augmented reality (AR), cinematic reality (CR), and more. With my background in film and visual effects (VFX), I spearhead all our VFX projects in film. VFX is the processes by which imagery is created or manipulated outside the context of a live action shot in film making. Visual effects involve in the integration of live-action footage and generated imagery to create environments which look realistic, but would be dangerous, expensive, impractical, or impossible to capture on film. We recently started our pursuit of implementing motion capture to add a new dimension to the VFX work that we do for video games. We also use the same motion capture process to extend our VFX services to previsualization work in films.

Which specific projects are you working on?

We recently spent some time in Montreal working on the latest X-Men film, Dark Phoenix, doing previsualization for the world leading stunt and previz company "Proxi VR". We worked with stunt performers for the movie in order to collect the motion capture data from certain scenes in real time. This allowed them to curate specific moments integral to the film. A great tool for high concept films. We are currently working with the talented Australian filmmaker brothers Kiah and Tristan Roache-Turner on their genre-mashing, sci-fi-horror, Nekromancer. We used our suit to help plan movement data for the VR component of the film we are building.

Spectre VR partnered with Jonathan Homsey a Melbourne-based artist to create a two-story installation based on dance choreography. Using motion capture to feature the music of Joyce Wrice (USA) to go on a journey to meet Mx.Red, a person beyond gender lines that shifts perception on physical intimacy. We captured performances over a few days and created assets for various parts of the project including VR, AR, and projection. We are working to bring culturally sensitive locations to life in VR with the help of the traditional land owners. We have been capturing these places rich in history using photogrammetry and then allowing people to experience them in full explorable detail that run along iMax documentary films with the first one being released in Feb 2018.

"The Suit is immediately able to handle the more complex movements made by trained performers, which ultimately provides us with far more freedom to create."

How are you using the Smartsuit Pro in your projects?

The Rokoko Smartsuit Pro is a crucial tool for our VFX and previsualization film and gaming projects. The motion capture animation allows a director, cinematographer, or VFX supervisor to experiment with different staging and art direction options—such as lighting, camera placement and movement, stage direction and editing—without having to incur the costs of actual production. Being able to slip out of an entire motion capture rig in a matter of minutes accelerates the process we go through to create computer and character animations. The Suit is immediately able to handle the more complex movements made by trained performers, which ultimately provides us with far more freedom to create.

What has your experience been with the Smartsuit Pro so far? Has it changed the way you work with character animation?

Fundamentally, it has changed everything. Mocap used to be very expensive, timely, and required the use of cumbersome 3D animation software. What we found with the Smartsuit Pro is that we’re able to rapidly prototype character animations and environment mockups for various uses. For a company like us speed and efficiency is critical to our work. We need quick, accurate data as soon as possible to know where we can spend the time in VFX as well as in VR, which are both processor intensive. It has changed our approach and allows us to constantly push the limits of Unreal Engine and utilize its ability to keep up with how fast we can now move with mocap. It’s also a lightweight product that permits us to work and share from virtually anywhere. We’re based in Sydney, but we’re able to connect with some of our workers in Brisbane seamlessly without missing a step. That’s something that those of us in this industry have never had the luxury of before.

What do you think about the ability to track fingers in the future, with SmartGloves, for example?

Well, the first thing that comes to mind with the introduction of SmartGloves is that we would be able to unlock the other main human element that people are looking for in real-time interactive performance.

What does the future hold for Spectre VR and what new possibilities does the Smartsuit Pro unlock?

Our main area of focus is VR and creating VR tie-ins for film. We scan sets and recreate them in Unreal Engine using photogrammetry to make them photorealistic, and then we add characters inside of the environment. We want to combine AI movements with our captures so that when you’re in these spaces every action you make triggers a reaction in the characters. We’re speaking with multiple theatres and looking into introducing narrative storytelling across various channels to engage a wider audience. We are experimenting with real-time VR cinema where an individual can experience a one of a kind performance created by performers rigged in the Smartsuit Pro. We believe there’s a bright future for real-time cinema, and that the Smartsuit Pro is just the vehicle to bring it into reality.

We are proud to be an integral component in Rick Pearce and Spectre VR’s work and look forward to continuing to aid his team as he pushes the limits of what XR can do.