Interview Questions for Motion Capture and Digital Puppetry

Optical and inertial motion capture systems are two fundamentally different approaches to capturing human or object movement. Optical systems use cameras to track reflective markers placed on the subject. These cameras triangulate the markers’ positions in 3D space, creating a detailed record of movement. Think of it like a high-tech version of surveying—multiple viewpoints pinpointing a single point. Inertial systems, on the other hand, use sensors (accelerometers and gyroscopes) attached to the subject to directly measure acceleration and rotation. These measurements are then integrated to calculate position and orientation. This is more like having a tiny GPS on each body segment, providing data regardless of visual line of sight.

The key difference lies in their setup and limitations. Optical systems require a carefully calibrated environment with ample camera coverage to ensure consistent tracking. They excel at accuracy and detail but can be hampered by occlusions (markers being hidden) and limited capture volume. Inertial systems are more portable and less sensitive to occlusion, but their accuracy can drift over time due to cumulative errors in integration. They are often used in conjunction with optical systems to compensate for each other’s weaknesses.

Cleaning and retargeting motion capture data is a crucial post-processing step that transforms raw data into usable animation. Cleaning involves removing noise and errors from the captured data. This can include smoothing out jittery movements, fixing gaps in the data caused by marker occlusion, and correcting for any inconsistencies in the actor’s performance. Think of it as editing a shaky video clip, smoothing out the bumps and making the motion look more natural.

Retargeting is the process of transferring motion data captured from one character rig (the skeleton representing the performer) onto another with different proportions or anatomy. This might involve transferring motion captured from a human actor onto a quadruped or even a fantastical creature. It’s like taking a dance routine and adapting it for a dancer with different body proportions. Sophisticated retargeting algorithms account for skeletal differences, scaling, and rotation to ensure the motion is realistically transferred. This often involves solving inverse kinematics (IK) and tweaking the resulting animation.

Tools like MotionBuilder and Maya provide powerful features for both cleaning and retargeting, often employing techniques like filtering, interpolation, and skeletal mapping.

Facial motion capture presents unique challenges due to the intricate and subtle movements of the face. The high density of muscles and the wide range of expressions make it difficult to capture accurately and realistically. Common challenges include:

• Marker tracking issues: Markers can be easily obscured by facial hair, makeup, or even rapid expressions.
• Data noise and artifacts: Subtle muscle movements can be easily lost in noise or misinterpreted by the system.
• Blendshape generation and weighting: Creating convincing blendshapes (morph targets for facial expressions) that cover the entire range of emotions is time-consuming and complex.

Addressing these challenges requires a multi-pronged approach. High-resolution cameras and advanced tracking algorithms are essential for minimizing noise and improving accuracy. Techniques like markerless facial capture (using image analysis to track features instead of markers) are becoming increasingly popular. Careful attention to blendshape creation and weighting is also essential to achieve realistic and nuanced expressions. Often, significant manual cleanup and adjustment are required to perfect the final animation.

Animating complex characters with motion capture data requires a combination of skill, creative problem-solving, and technical know-how. The process often involves several steps:

• Selecting appropriate motion capture data: Choose data that aligns as closely as possible to the desired animation. This often requires a combination of different motion capture clips and extensive editing.
• Retargeting and adaptation: Transferring motion data from the capture source to the complex character rig requires careful retargeting and potentially significant manual adjustment to account for anatomical differences.
• Blending and layering: Combining different motion capture clips and incorporating keyframe animation to seamlessly transition between movements and fill in gaps.
• IK and FK adjustments: Use IK to refine limb positioning and FK for detailed adjustments to specific body parts.
• Simulation and secondary animation: Use simulation techniques like cloth and hair simulation, and add subtle secondary animations to enhance realism and add life to the character.

Animating a dragon, for example, might require combining human motion capture for walking with quadrupedal animal motion for certain stances and actions, then further enhancing the animation with wing flapping simulations and facial expressions that are keyframed or partially driven by motion capture data for mouth movements.

I am proficient in several industry-standard software packages for motion capture processing and animation, including:

• Autodesk MotionBuilder: An industry-leading application for motion capture editing, retargeting, and animation.
• Autodesk Maya: A powerful 3D animation and modeling software extensively used for character rigging, animation, and simulation.
• Adobe After Effects: Useful for compositing and adding finishing touches to the final animation.
• Blender: A free and open-source alternative with growing capabilities in motion capture processing and animation.

My experience spans different pipelines, from working with raw motion capture data to creating polished final animations.

Forward kinematics (FK) and inverse kinematics (IK) are two fundamental approaches to character animation. FK is a method where the animation is controlled by directly manipulating the joints of the character’s skeleton. You move the joints in a sequence, and the resulting pose is determined by the chain of transformations. Think of it as manually controlling each joint like a puppet. It is intuitive for simple poses but can become cumbersome for complex animations and can lead to unnatural posing.

IK, on the other hand, is a method where the animation is controlled by specifying the position and orientation of the end effector (usually the hand or foot) and letting the system solve for the joint angles required to achieve that pose. Think of it as setting the goal, and the system figures out how to achieve it. This is particularly useful for complex animations where maintaining natural poses is crucial. However, it requires careful setup and can sometimes produce unexpected results if not constrained appropriately.

Often, a combination of both FK and IK is used in a production pipeline. FK is used for initial pose setup and more direct control of specific parts, while IK is used to refine poses and to ensure that the character’s limbs and other parts move realistically.

My experience encompasses various motion capture marker sets, each tailored to specific needs and applications. The choice of marker set depends on factors like the level of detail required, the type of motion being captured, and the available technology.

For full-body motion capture, we often use marker sets with markers placed on major joints and anatomical landmarks, allowing for precise tracking of movement. For facial capture, more extensive marker sets are used, with markers distributed densely across the face to capture subtle facial expressions. Some systems utilize passive markers (reflective markers) while others leverage active markers (markers with integrated LEDs for easier tracking). Active markers enhance precision and reduce issues with light conditions, but increase cost and set up time.

Specialized marker sets might be used for capturing very specific actions or movements, such as hand gestures or foot placement in running. The data from these marker sets is processed and cleaned using specialized software and techniques. Understanding these different marker sets and their applications is fundamental for achieving high-quality, accurate motion capture data.

Achieving accurate and realistic movement in digital puppetry hinges on a multifaceted approach combining high-quality motion capture (mocap) data, meticulous rigging, and sophisticated animation techniques. Think of it like puppeteering, but instead of strings, we use digital controls.

Firstly, the quality of the mocap data is paramount. Precise marker placement on the actor and a clean capture environment are critical. We use advanced systems with multiple cameras to ensure full body coverage and minimize occlusion (markers being hidden from view). Secondly, the digital puppet’s rig – the skeletal structure and its controls – needs to be meticulously crafted to accurately reflect the captured motion. Poor rigging will lead to unnatural deformations and unrealistic movements, no matter how good the mocap data. For example, subtle differences in limb lengths and joint rotations between the mocap actor and the digital puppet need to be addressed. Finally, we often use secondary animation techniques to refine the mocap data. This might include adjusting the timing or adding subtle details to the character’s performance, for example, to add more natural-looking weight to steps or to fix slight glitches.

In practice, this means a rigorous iterative process of capturing data, cleaning it, adjusting the rig, and adding polish. It’s similar to sculpting – refining and refining until the result looks perfect.

Dealing with noisy or artifact-ridden mocap data is a routine part of the pipeline. Think of it as cleaning up a messy sketch before finalizing a painting. We employ various strategies to mitigate these issues. Firstly, we use sophisticated filtering techniques to smooth out the data and remove unwanted jitter or spikes. This could involve applying low-pass filters or more complex algorithms designed specifically for mocap data. Secondly, we manually clean the data. This involves frame-by-frame inspection to identify and correct obvious errors or outliers. Advanced software allows us to easily manipulate and adjust individual joints or markers.

Sometimes, we might need to interpolate or extrapolate missing data. Interpolation means estimating values between known data points, while extrapolation is predicting values beyond the known range. For instance, if a marker was lost for a brief period, we can use interpolation to reconstruct its trajectory based on surrounding frames. We also use advanced algorithms to detect and compensate for issues like marker swaps, where the software incorrectly assigns markers during the capture process.

Inconsistencies in motion capture data can stem from various sources, such as poor marker tracking, actor inconsistencies, or even equipment malfunction. I’ve encountered situations where the captured motion would suddenly have unnatural breaks or jumps.

One project involved a scene with a character performing a complex acrobatic sequence. The mocap data showed some significant gaps and inconsistent foot placement, especially during mid-air phases. To solve this, I combined manual cleanup with intelligent interpolation. I carefully reviewed each frame, correcting obvious errors. Where data was entirely missing, I used the surrounding frames and knowledge of the correct body mechanics to infer plausible positions and reconstruct the missing movement. For the foot placement issues, I leveraged reference videos and consulted with a movement specialist to ensure realistic and safe movement before recreating the missing data.

Another common issue is drift, where the character’s position slowly deviates over time. This usually stems from cumulative error in the capture process. I’ve addressed this by using tools that automatically detect and correct for drift, sometimes employing root motion retargeting to maintain the character’s ground position.

My workflow for integrating mocap data into a game engine typically involves several key steps. First, I process and clean the mocap data using dedicated software to remove noise and artifacts, as discussed previously. The next step is to retarget the data to the game engine’s skeleton, which often differs in its structure and bone hierarchy from the mocap rig. This process maps the mocap data onto the game character’s skeleton. If there are any structural differences, this will necessitate manual adjustments to ensure the animation works naturally on the in-game model.

Next, I import the processed and retargeted animation data into the game engine. We often use FBX or BVH formats for this transfer. Within the game engine, we might need further adjustment using animation tools. This could include adjusting animation curves to fine-tune timing, blending different animations, or adding root motion to control the character’s overall movement in the game environment. Finally, I test the animation within the game context and make any further refinements as necessary to ensure seamless integration and realistic behavior.

For example, in a recent project, we used Unreal Engine. The mocap data, after processing, was exported as FBX files. These files were imported into Unreal and further refined using the engine’s animation tools. The process involved blending various animations – walking, running, jumping – together to ensure smooth transitions between actions. The final step was integrating the animations into the game’s level design, making sure the animations looked natural within the game’s environment.

Effective collaboration in motion capture production is crucial. I always start with clear communication and shared goals. Before a shoot, I meet with the director, actors, and technical crew to thoroughly understand the vision for the project. We discuss specific requirements, such as the range of motions, camera placement, and any special considerations for the actors’ performance. I ensure that everyone understands their roles and responsibilities.

During the capture process, I work closely with the actors to provide clear and concise direction. It’s essential to maintain a positive and supportive atmosphere, offering feedback and guidance while respecting their artistic contributions. Effective communication with the technical team ensures that all equipment is functioning correctly and that the capture is proceeding smoothly. Any technical issues are immediately addressed to minimize delays and wasted time. After the shoot, I stay closely in touch with other team members, such as animators and developers, during post-production. This ensures that any feedback or requests are promptly addressed and that we’re all aligned on the final result.

Think of it like a well-orchestrated symphony. Each musician needs to be in sync, playing their part perfectly to create a harmonious performance.

I have extensive experience with real-time motion capture systems, primarily using optical systems based on multiple cameras and marker tracking. Real-time systems offer several advantages, primarily the ability to immediately visualize and refine the captured performance. This instant feedback is invaluable, allowing for immediate corrections and refinements during the capture process. It speeds up the production process significantly, reducing the time spent on post-production cleaning and adjustments.

However, real-time systems can be more demanding in terms of setup and calibration. Ensuring precise camera placement and marker tracking is essential. Any issues with occlusion (markers being hidden) can significantly impact data quality. The data generated often still requires some post-processing, although significantly less than traditional offline systems. I’ve also worked with some newer systems incorporating inertial sensors, offering a blend of real-time capture and potentially greater robustness to occlusion. The choice of system depends on project requirements, budget, and the specific artistic goals.

There’s a range of motion capture suits available, each with its own advantages and disadvantages. Optical suits, using retroreflective markers, are commonly used. They offer high accuracy and detailed capture of fine movements. However, they can be expensive, susceptible to occlusion, and somewhat restrictive for actors to wear.

Inertial suits, using sensors embedded within the suit, are less susceptible to occlusion but can be affected by sensor drift and might not capture subtle movements as accurately. They are often more affordable and less restrictive. Magnetic suits use magnetic sensors, which require a specific capture volume, and can be prone to interference from metallic objects.

The best choice depends on the application. For example, optical suits are often preferred for high-fidelity animation, while inertial suits might be better suited for applications where occlusion is a major concern, like fast-paced action scenes or capturing performers in complex environments. I’ve used all three types in various projects, choosing the most appropriate technology to suit the unique requirements of each production. Each choice involves trade-offs between accuracy, cost, and convenience.

Balancing performance capture data with animation enhancements is crucial for achieving natural-looking results. Raw mocap data often contains imperfections, noise, and limitations. The key is to use the performance as a foundation, enhancing it with manual animation to address these shortcomings, but preserving the core performance and intention of the actor.

For example, imagine capturing a run. The mocap data might accurately represent the leg movements, but the arm swing might be stiff or unnatural. We use this data as a base, then we would go in and subtly adjust the arm animation, perhaps adding more fluidity, more anticipation and follow-through, or even subtle changes in the shoulder rotation to create a more nuanced performance that better conveys the actor’s intent. The aim is to subtly polish the gem, not replace it entirely.

This process often involves iterative refinements. We might preview the animation, identify areas needing improvement, and use animation tools to keyframe tweaks. Sometimes, we may even utilize procedural animation techniques to generate subtle details, like secondary movements of clothing or hair, which are difficult to capture cleanly in mocap.

A motion capture technician plays a vital role in ensuring the success of the entire motion capture process. Their responsibilities extend far beyond simply pressing record. They are responsible for the technical setup, calibration, and maintenance of the mocap system. This includes ensuring accurate marker placement on the actor, monitoring the data quality in real-time, and troubleshooting any technical issues that may arise. Pre-production involves camera placement and configuration, while during the capture process, they must maintain focus to spot markers dropping off or any interference. Post-production involves reviewing the data, cleaning up noise, and preparing the data for the animators.

They’re like the orchestra’s conductor for the technical aspects of the performance capture. They ensure all the instruments are in tune and playing in harmony to capture the best possible performance.

Biomechanics is the study of human movement and the mechanical principles governing it. Understanding biomechanics is fundamental to motion capture because it helps us interpret and enhance the captured data. Mocap provides us with the ‘what’, but biomechanics gives us the ‘why’. By understanding how the human body moves and how muscles, bones, and joints interact, we can identify unrealistic movements within the captured data and correct them.

For example, if an actor is performing a punch, understanding biomechanics allows us to identify and correct unnatural joint rotations or implausible muscle contractions. This would lead to a more realistic and convincing punch, one that respects the natural limitations of the human anatomy. An animator who understands biomechanics can improve the motion capture data, making it more convincing and believable by applying this understanding during post-processing.

Limitations in motion capture data are inevitable. The challenges can range from occlusions (markers being hidden from the cameras) to simply lacking the specific data we need. To address this, we employ a variety of techniques.

• Reference Footage: If the mocap data is incomplete for a specific action, we’ll often use reference footage (videos of actors performing the action) to inform the animation. This will give us visual cues to fill in the gaps in the data.
• Procedural Animation: For subtle details like clothing or hair simulation, procedural animation techniques can often be leveraged to generate realistic movement based on the underlying mocap data.
• Manual Keyframing: In cases where the data is simply unusable, it might be necessary to manually keyframe the missing animation. This requires more time and artistry, but allows for total creative control to maintain continuity and believability.
• Data Blending: If we have similar motion capture clips, we can attempt to blend them to create the desired action by carefully selecting and combining parts that fill in the data holes.

The approach depends on the severity of the limitation and the resources available. The goal is always to maintain realism and consistency with the rest of the animation.

Rigging is the process of creating a virtual skeleton and control system for a 3D character. Different rigging techniques have a significant impact on how well motion capture data can be applied. A well-designed rig ensures that the captured data translates smoothly and accurately onto the character model, while a poorly designed rig can lead to distortions, artifacts, and unnatural movements.

For example, a rig with a simple bone structure might not be able to handle complex movements accurately, whereas a rig with a sophisticated system of constraints and controls can reproduce subtle nuances and details. We might use a character rig with specialized controls for facial animation, or a more physically based rig that simulates the actual biomechanics of the body. The choice of rigging technique directly influences the quality of the final animation. The ideal rigging would be adaptable to accommodate future enhancements and changes needed for the project.

Creating convincing digital puppets with expressive performances begins with a strong foundation in character design and rigging. The puppet’s design should be expressive, capable of conveying a wide range of emotions through subtle changes in its features and posture. A well-built puppet rig will allow for intuitive control of the character’s movements, facial expressions, and other nuances. Then we layer on sophisticated animation techniques, such as procedural animation, to generate convincing details like the movement of clothing or hair.

Beyond technical aspects, understanding acting principles is essential. I study the performance capture data to discern the core emotion and intent. We will then leverage animation tools to enhance the performance, adding details like subtle blinks or micro-expressions to create a realistic and empathetic character. It’s like adding details to a painting — small touches create a masterpiece.

My expertise includes a broad range of performance capture editing software and tools, including industry-standard packages such as Autodesk MotionBuilder, Maya, and Blender. I’m proficient in using these tools to clean, edit, and refine motion capture data. This includes tasks such as marker cleanup, retargeting, and blending various capture sets. I also have experience with specialized tools for facial animation and virtual production pipelines, allowing me to integrate motion capture seamlessly into larger productions.

Beyond the core software, I have a deep understanding of the underlying principles of motion capture processing. This understanding is what sets an experienced professional apart and allows for efficient work flows. I also keep abreast of the newest developments in the field, guaranteeing that I am working with the most effective and efficient workflows.

Creating believable blends between motion capture (mocap) clips requires careful consideration of the transition between different movements. Think of it like editing film – you don’t want jarring cuts. Instead, we need smooth, seamless transitions. This is achieved through several techniques.

• Retargeting: If the clips are from different actors or have different skeletal structures, retargeting maps the motion data onto a common skeleton. This ensures the same movement is applied consistently.

• Blending weights: Software allows us to blend between clips using weighting systems. We can smoothly transition from one clip to another over a specified number of frames, adjusting the weighting to control the influence of each clip throughout the blend. For example, we might have a 50/50 blend at the midpoint and smoothly transition to 0/100 at the end, eliminating any abrupt changes.

• Manual Adjustment: In complex situations, we might need to manually adjust keyframes within the transition area to fine-tune the blend. This is particularly important where subtle changes in body posture or facial expressions might break the realism.

• Motion warping: Advanced techniques like motion warping can stretch and compress parts of the motion data to smooth out inconsistencies. This is helpful when aligning movements with different timings.

For instance, blending a walking clip with a running clip requires careful weighting to create a smooth acceleration. We’d start with the walking motion, gradually increase the weighting of the running clip, and finally, only the running clip remains.

Working with mocap data presents several challenges. Noise, artifacts, and inconsistencies in the data are common issues.

• Noise: Mocap suits can introduce noise into the data, causing jittery or unrealistic movements. We tackle this with filtering techniques, smoothing the data to remove unwanted oscillations. This involves using various filters based on the severity and type of noise.

• Artifacts: Occasionally, the mocap system might misinterpret data points, creating unnatural poses or movements (artifacts). Careful manual cleaning is often necessary. This involves reviewing the data frame by frame and correcting these anomalies.

• Inconsistencies: In long capture sessions, actors may change their performance slightly over time, leading to inconsistencies. This is addressed by adjusting the animation and possibly using multiple takes blended together for a consistent, believable performance. This might also involve segmenting the data and re-timing to adjust pacing.

• Marker loss: Mocap markers can be occluded by clothing or body parts. Interpolation or prediction algorithms can fill in gaps in this missing data, but it frequently involves manual intervention to ensure the filled portions are believable. Advanced techniques like machine learning can provide effective solutions in these complex situations.

Imagine a character whose arm suddenly jerks; this might be an artifact. We’d need to manually correct this frame to restore natural movement.

Keyframe animation and motion capture are both methods of creating animation, but they differ significantly in their approach. Keyframe animation is a manual process where animators set key poses at specific points in time. The computer then interpolates the movements between those keyframes. Mocap, conversely, uses sensors to capture the movement of an actor. The data are translated into a digital representation of movement.

• Keyframe Animation: Offers precise control over every detail of the animation, allowing for stylized and expressive movements. However, it’s time-consuming and requires high artistic skills.

• Motion Capture: Provides realistic and natural movements, reducing the time needed for creating complex animations. However, it requires specialized equipment and post-processing to correct any errors or inconsistencies in the captured data. It also limits artistic control and stylized expression unless significant post-processing and editing are applied.

Think of keyframe animation as sculpting, carefully crafting each pose. Mocap is like recording a performance and refining it afterwards. Often, the best approach is a combination of both techniques.

Incorporating realistic physics into digital puppet animation presents many challenges because it requires a sophisticated understanding of how fabrics, joints and rigid bodies interact. It’s far more complex than simply animating a rigid character model.

• Cloth Simulation: Simulating the realistic drape and movement of clothing requires complex algorithms, and even high-end simulations might produce unnatural results if not meticulously tweaked. The interaction of clothing with the underlying body often requires iterative adjustments.

• Joint Limits and Constraints: Digital puppets have joints, and these joints have limitations in their range of motion. Enforcing these constraints is vital to maintaining the integrity of the puppet. Otherwise, it can exhibit unnatural bending and breaking.

• Collision Detection: This is crucial for realistic interactions between the puppet’s body parts, clothing, and the environment. Accurate collision detection prevents the puppet from clipping through itself or its environment.

• Computational Cost: Realistic physics simulations are computationally intensive. Finding a balance between visual fidelity and performance is a constant challenge, particularly in real-time applications.

Imagine animating a puppet wearing a flowing cape; the cape’s physics must react realistically to the puppet’s movements and the environment’s forces, creating believable interactions.

Animating a character performing a complex acrobatic movement using motion capture would involve careful planning and execution, paying close attention to safety and data quality.

• Experienced Performer: The performer must have acrobatic skills and be able to consistently perform the movement. This ensures good quality mocap data.

• Multiple Takes: Capturing multiple takes of the acrobatic movement is essential to obtain the best possible data, allowing to select the cleanest and most consistent performance.

• High-Quality Mocap Setup: A high-quality motion capture system with enough cameras to avoid marker occlusion is crucial for detailed and accurate data. Using cameras with high framerates will help ensure accurate capture of dynamic motions.

• Data Cleaning and Editing: Once captured, the data needs thorough cleaning and editing. Noise removal and correction of any artifacts or inconsistencies are critical.

• Integration and Refinement: Finally, the cleaned mocap data needs to be integrated into the character model, possibly refined to improve realism. This may involve manual adjustment, in-betweening, and potentially combining with keyframe animation to refine finer details.

For a backflip, for example, we might need multiple cameras to track the performer’s entire body motion, ensuring we capture every detail of the rotation and landing. Post-processing then corrects for any imperfections and ensures a smooth animation.

Reference videos are indispensable in creating realistic animation, providing a visual benchmark for movement, timing, and overall performance. They act as a guide, ensuring the final animation aligns with reality. They are particularly crucial when capturing subtle nuances which are often difficult to convey through verbal instructions alone.

• Movement Analysis: Studying reference videos helps break down complex movements into smaller, manageable parts. We can analyze the timing, acceleration, and deceleration of each part. This is invaluable for replicating complex human actions.

• Performance Reference: Reference videos also provide insights into the character’s emotion and intent. These factors directly influence the performance. How the actor moves, their subtle expressions, and the overall energy all contribute towards realism.

• Technical Guidance: When working with mocap, reference videos can highlight issues with the capture process itself. They help check for inconsistencies or missing details, prompting adjustments during the data capture process or post-processing phase.

For example, animating a character pouring a cup of tea, it might be helpful to study several reference videos of people doing this, noting how their hands and wrists move to achieve accurate replication.

Creating believable lip sync using motion capture data involves more than just capturing mouth movements. It requires meticulous attention to detail and often a combination of mocap and manual adjustments.

• Specialized Mocap Systems: High-resolution facial mocap systems are crucial for capturing subtle lip and facial movements that are vital to achieving natural lip sync. Facial motion capture systems often utilize cameras, and sometimes markers directly on the face.

• Data Cleaning: Raw mocap data frequently requires extensive cleaning to remove noise and artifacts. The goal is to ensure the lip movements are smooth and consistent. This includes dealing with noise, and smoothing the data to improve the realism.

• Phoneme Alignment: Matching the captured lip movements to the audio track requires careful alignment of phonemes (units of sound). This step is crucial to creating smooth lip sync. Software facilitates the process, but often requires adjustments for optimal precision.

• Manual Adjustment: Even with advanced mocap techniques, manual adjustments are often necessary to fine-tune the lip sync, particularly for subtle movements or expressions. This is critical when seeking a very high degree of realism.

Imagine a character saying ‘fish’. The subtle changes in the mouth shape need accurate capture and alignment with the audio. Manually adjusting might be needed for smooth transitioning between sounds.