LivePortrait - Animate faces and create avatars with ComfyUI #comfyui #stablediffusion

Science & Technology

Introduction

In today's world, we are constantly moving towards a more just and better future. One of the ways we can achieve this is by using cutting-edge technology to create lifelike avatars effortlessly. With tools like LivePortrait and ComfyUI, this task becomes simple and incredibly efficient.

Introduction to LivePortrait

LivePortrait is a new face animation method that provides amazing results. Unlike diffusion methods, LivePortrait does not require a very powerful machine or renting expensive cloud GPUs. This tutorial focuses on using the LivePortrait node implementation by Kiji, which is easy to install and use. There is also a version from Shadow_wcz that allows more functionalities such as video-to-video animations or animating multiple persons. However, this requires manual model downloads and mixlab node installations. Luckily, Kiji’s development version can be used to bypass this requirement.

Installation Guide

  1. Install LivePortrait Node: Navigate to the ComfyUI manager and search for "LivePortrait" in custom nodes. Install the first custom node from Kiji.
  2. Download InsideFace Models: If you don't already have it, download the models from the last link in the InsideFace repository.
  3. Download Assets: Download example images and videos from the repository and place them in your input folder in ComfyUI.
  4. Run Workflow: Drag and drop the workflow example over the ComfyUI canvas. Choose the image and the reference video. The first run will take longer due to model downloads.

Creating Animations

To create your own live portraits, follow these steps:

  1. Select an Image and Video: Use the example workflow to select the image you want to animate and the video as the reference for facial movements.
  2. Customization: Adjust the parameters to fine-tune the animations for the best results.
  3. Multiple Faces and Video to Video: Install the development branch for more features. Stop your ComfyUI server, navigate to the custom nodes directory, delete the existing installation, and manually install the development branch. Restart your ComfyUI and make necessary node adjustments.

Advanced Features

Video-to-Video

  1. Add Load Video Node: Use a video to output a square format (e.g., 720x720).
  2. Select Reference Video: Use a reference video, set the frame load cap, and ensure the number of frames match.
  3. Adjust Parameters: Change the "relative" parameter to achieve more accurate facial movement replication.

Animate Multiple Faces

  1. Initial Setup: Run the standard workflow and identify the indexes of each face.
  2. Duplicate Nodes: Repeat the LivePortrait construction for each face and connect them in series.
  3. Adjust Frames: Ensure the number of frames in each video matches by making adjustments in the load video nodes.

Conclusion

LivePortrait with ComfyUI offers an exciting way to create animated avatars and lifelike faces. By following these steps, you can efficiently produce high-quality animations and customize them to meet your needs.

Thank you for reading!

Keywords

  • LivePortrait
  • Face Animation
  • ComfyUI
  • InsideFace
  • Kiji
  • Custom Nodes
  • Video-to-Video
  • Multiple Faces

FAQ

Q: What is LivePortrait? A: LivePortrait is a face animation tool that produces impressive results without requiring high-end hardware or expensive cloud services.

Q: How do I install LivePortrait on ComfyUI? A: Install the LivePortrait node from ComfyUI manager, download the necessary InsideFace models, and follow the instructions for integrating it into ComfyUI.

Q: Can I use my own images and videos? A: Yes, you can use your own images and videos. Place them in the input folder in ComfyUI and select them in the workflow.

Q: How can I animate multiple faces in a single image? A: By using the development branch of LivePortrait, repeat the LivePortrait node setup for each face, and connect them in series.

Q: What if my reference and target videos have different frame counts? A: Make sure to adjust the load video nodes so that both videos have the same number of frames, or use other tricks like repeating the shorter video to match the frame count.