What is Motion Capture, and How Does it Work in 2022?

Motion capture (mocap) technology is used to record movements and apply them to a 3D model. Physical mocap suits, specialty cameras, and advanced software are used to create photorealistic animations that can be used in film, sports, and even healthcare. 

What is motion capture?

Motion capture records movement and translates it into data that can be read by animation software and applied to a 3D rig or character. It's a common misconception that motion capture projects need a big budget and an entire production team. With evolving technology, you can even use your cellphone to do basic motion capture. For example, Instagram filters use a type of mocap to track your face in real-time and apply simple animation overlays. It’s a similar technology that is used in Rokoko’s facial mocap app for iOS, which allows 3D artists to capture facial motions as blendshapes to apply them on their custom characters in their 3D animation projects.

The rise of motion capture as a Hollywood must-have

Motion capture was first used in the film Sinbad: Beyond the Veil of Mists (2000). In the next few years, it was quickly popularized by the then-revolutionary animated character Gollum in The Lord of the Rings. The animated character interacted realistically with his live-action co-stars by relying on the mocap technique developed by Weta studios. Keep reading until the end of the article to see a video of what that looked like in practice.

From then on, mocap has become an almost mandatory feature in major films requiring VFX. In the late 2010’s motion capture evolved from tracking humans to tracking animals. It’s now possible to record movement from popular domestic animals such as dogs and horses. 

What applications are suitable for motion capture?

Motion capture can be used for many types of projects, not just VFX. These include: 

• Game animations use mocap to quickly build up a vast library of motions for each game character. 
• Previs (also known as previsualization) happens during pre-production. It’s when the creators bring the static storyboard to life. In complex scenes, directors will often use mocap to block out the motions of the scene and more accurately prepare for shooting and VFX. 
• Humanoid fantasy characters need to move realistically to avoid the uncanny valley effect. And that’s what mocap helps animators achieve. 
• The health and sports industry is a big user of motion capture technology. It’s been used to do everything from optimizing an athlete’s tennis swing to injury diagnosis and rehabilitation.
• In the military, mocap is used to create advanced simulations and improve training programs. 

What to consider when starting a motion capture project?

First, you need to be aware of the four main types of motion capture, what they mean, and how to choose one that best fits your project needs.

Optical-Passive: Retroreflective suit markers & infrared cameras

Retroreflective markers are placed on actors via a tight-fitting suit and tracked via infrared cameras. Historically, this was the most common way of doing motion capture. Large studios commonly use this type as it's the most accurate, yielding the impressive photorealistic tracking required for feature films. However, it can be resource-hungry and isn’t suitable to run on entry-level systems. 

Optical-Active: LED suit markers & cameras

Light-emitting LED markets are placed on actors the same way as optical-passive tracking, and special cameras record their movement. This isn’t used often anymore as the actors also need to carry some kind of charger or battery case, and the LED light can potentially spill into other filmed elements.

Video (Markerless): A sophisticated camera stage is used

Actors do not use suit markers of any kind. Instead, the acting area is covered by a grid on the floor and a network of cameras that shoot the scene from every possible angle. The recorded footage is analyzed by software and translated into motion data that animation software can read. However, the end result takes more time and includes more errors than other methods, meaning that a lot of data cleanup is needed in post-production. This type of motion capture is useful for large-scale productions that have post-production budgets. It captures the scene from every angle, reducing the need for retakes.

Inertial (Cameraless): Motion sensor suit

Unlike the other types, Inertial requires no cameras to capture the motion. Instead, inertial sensors (IMUs) are placed within a bodysuit and worn by the actor. The motion data is transmitted wirelessly to a nearby device. The gyroscopic motion sensors record the angle, position, and momentum of your body and accurately transcribe it into animated movement. This is the most cost-effective option and is popular with indie studios and game developers, like indie game developer Brian Parnell that completed all the character animations for his game “Praey for the Gods” with the Smartsuit Pro, get the full story here.

The benefits of using motion capture

There are three core benefits to using motion capture in your production

1. VFX costs and animation timelines are significantly reduced ​​​​​

Typically, 3D animators place keyframes for every major movement. Then, they adjust every frame with micro-movements. Sometimes repeating this process hundreds of times for each limb. Considering there are at least 24 frames per second, many productions quickly go over budget and miss deadlines due to animation. Using motion capture, the bulk of animation work is completed with the live-action actor's movements. 

2. Facial animation becomes way easier

Accurate facial animation is known to be one of the most challenging tasks - especially if you're aiming for a photorealistic outcome (just check out the length of this 2021 paper!). With a simple mocap setup, you can capture basic facial animations. To capture the more realistic animations seen in Hollywood films, you'll need a more sophisticated setup that often includes a direct 3D scan of the actor’s face to map movements correctly.

3. Previs for animations is cheap enough for small productions

Previs is essentially the previsualization of any movie, game scene, music video, or short film. It's often done through hand-drawn storyboards that are timed to voice-over or music. In productions that require a high level of planning (e.g. an animated dancer in a music video), you can use mocap to guarantee that your choreography is on time and in the frame. 3D Artist and Rokoko user Don Allen III is behind the motion capture recordings of Lil Nas X’s Panini music video:

The 4 basic steps when getting started with motion capture

Exactly what you need to know if you want to use motion capture for your production 

1. Decide on a type of motion capture

As you learned earlier in this article, there are four main types of motion capture. However, two types currently dominate the market; Optical-Passive and Inertial. We recommend that you only consider Optical Passive for larger projects with bigger budgets. It requires a bodysuit, software, and cameras capable of infrared capture. For all other purposes, the Inertial type is best suited. Intertial mocap can be captured in any location with any props as long as you have Wi-Fi access. 

2. Decide on a system & software

Most motion capture systems provide their own propriety software built to perform optimally with their suit and/or cameras.

3. Make sure your mocap data integrates with your animation software

Motion capture software isn't always what your animators will work in. Many studios operate exclusively on Autodesk Maya, Blender, Unreal Engine, Unity, Cinema 4D Houdini, and others. Make sure the mocap data captured can integrate with your systems. 

4. Capture motion and clean up the data 

While motion capture data can be exceptionally accurate, it's not immune to errors. And that margin of error increases with erratic movements and high-speed motion. So be aware of the extra animation time you might need in post-production if your movements are complex. Don’t forget to set aside a bit of time for you, and your animation team to clean up and refine the animation. 

Motion capture examples

Here's what motion capture looked like for Gollum when filming the original Lord of the Rings:

And later, for the prequel The Hobbit, Benedict Cumberbatch gives a captivating performance that makes Smaug the dragon truly terrifying:

Motion capture for the Incredible Hulk realistically translates the actor’s performance - no matter whether he's hulked up or human:

A motion capture example from the 2016 game "Paragon.” Notice how this motion is capture in real-time, but the movements are slightly jerky? A good motion capture actor will be comfortable creating sharp movements that are easier for animators to clean up.

For example, the God of War game animators trained with mocap actors rigorously before capturing any motion data. The resulting fighting moves were crisp and dramatic, winning over gamers with great gameplay.

The best motion capture systems are always evolving

Now that you know what motion capture is and how it works, you’re ready to start your next project. Take a look at Rokoko’s motion capture tools and free animation software. They’re built to be robust, multi-purpose, and easy to use.