Control AI Actors with this Awesome Workflow (LivePortrait Tutorial)

Introduction

In today's world of AI and digital filmmaking, controlling an actor's performance using AI has become a groundbreaking opportunity for creators. With the help of AI video tools like Runway, Luma, and LivePortrait, you can project a recorded performance onto AI-generated videos, including both live-action and animated styles, with remarkable ease. In this tutorial, we'll walk you through the steps to use LivePortrait to bring your AI actors to life. Let's get started!

Step 1: Prepare Your Reference Image

The first step involves creating a reference image of the character you want to animate using an AI image generator such as Midjourney. Simply input prompts like “a woman facing a camera with a simple white background.” It's crucial to make sure that the subject's face is unobstructed, so avoid glasses or hair blocking the features.

Step 2: Animate the Image

Next, you need to animate the reference image. We’ll use Runway's Gen 3 for this example. Drag and drop your image into Runway, and input a prompt that describes the character's action, like “a woman talking to the camera.” It's beneficial to suggest a specific emotion for a more natural animation. Set the duration for 10 seconds and generate a few variations.

Step 3: Record Your Performance Video

Now, it’s time to record your video footage. It's best if you use a neutral background, such as standing in front of a window for optimal lighting, to facilitate AI tracking. Record your performance, focusing on conveying the desired emotion.

Step 4: Speed Up the Footage

To ensure compatibility with AI tools, you must speed up your video footage by 25%. Use Premiere Pro to adjust the speed: right-click on your video, navigate to "Speed/Duration," and input 125%. Export the video using the H.264 format.

Step 5: Face Swapping with LivePortrait

Using LivePortrait, an advanced face-swapping tool, is the next step. Instead of setting it up on your local machine, opt for cloud services like Run Comfy. Navigate to the “Live Portrait Vid to Vid” model and upload both your AI-generated video and your sped-up performance video.

Configure the frame count you want to render, ensuring that you have a few seconds of footage, and then click on the "Q prompt" button to initiate the rendering process. This may take anywhere from 30 seconds to one minute.

Step 6: Upscale the Video

Once you have your footage rendered, it's important to enhance the resolution using an AI upscaling tool. Applications like Topaz Video AI are industry standards for this purpose. Load your video, select the appropriate settings, and export it.

Conclusion

With these steps, you now have the ability to control an actor's performance using AI and blend it with different characters—whether they’re animated or live-action. This method opens up endless creative possibilities.

Make sure to select characters without obstructive features for optimal results.

For more resources, download assets used in this tutorial from Curious Refuge and check out their AI filmmaking courses.

Keywords

• AI video tools
• LivePortrait
• Animation
• Character
• Performance
• Upscale

FAQ

Q1: What tools do I need to control an actor's performance using AI?
A1: You’ll need an AI image generator (like Midjourney), video editing software (like Premiere Pro), and AI video tools (like Runway, LivePortrait, and Topaz Video AI).

Q2: Can I use this method for animated characters?
A2: Yes! This method works well for both live-action and animated characters.

Q3: Why do I need to speed up my reference footage?
A3: Speeding up the footage by 25% is necessary for compatibility with the AI tools used for face projection.

Q4: What formats should I export my videos in?
A4: For the AI tools, export your initial videos in H.264 format. For the final projects, consider using formats like ProRes for editing.

Q5: How can I improve the output quality of my final video?
A5: Use an AI upscaling tool like Topaz Video AI to enhance the resolution and clarity of your output video.