Animate 3D Characters With AI - BIG Indepth AI Animation Workflow

Film & Animation

Animate 3D Characters With AI - BIG Indepth AI Animation Workflow

Hey everyone, it’s been a while! In this new video about AI animation, I wanted to showcase some of the latest tools and techniques in the 3D AI space. These tools can be used to create, texture, retexture, and animate models all within a single package. We'll also touch on importing these assets into Blender and Adobe After Effects to build fuller scenes. Finally, I'll show how to use another tool to add facial animation and lip-syncing and yet another tool to enhance the overall image to create something with even more fidelity. There’s a lot to cover, so let’s jump right in!

Before diving into the tools, here’s a sneak peek of a short film I created using this process. You’ll be able to see what can be built if you combine all these tools over some time.

Showcasing Meshi

I’ll be using a tool called Meshi, and you can find the link to their website in the video description below. Currently, there's an offer to save 50% on a monthly subscription via the affiliate link provided in the description. Meshi offers an integrated solution for 3D model creation, texturing, retexturing, and animation. I'll run through how to use these features step-by-step.

Creating a 3D Model in Meshi

Once logged into Meshi, you arrive at the Discover page, where you can explore other users' creations and remix them. For our purposes, we head over to the “Text to 3D” section, ensuring we’re using the new beta version. Here’s how to create your first model:

  1. Prompt Box Features:

    • Extract prompt from an image: Drag an image, hit ‘Generate Prompt’.
    • Prompt Library: Offers pre-written prompts for quick creation.
    • Import Past Generations: Use previously made assets as templates.

  2. Generating a Gnome Model:

    • Prompt Input: "fat garden gnome black hat black suit white shirt black tie t-pose character full body".
    • Negative Prompts: Use the prompt library or write your own.
    • Choose the Model: Meshi 3 Turbo model for faster animation-enabling generating.
    • Style Selection: Realistic, PBR, or sculpture style. We chose 'realistic' and hit ‘Generate’.

After 20 seconds, our gnome model was created. You can refine it further to increase texture richness and polygon count by using the refine option.

Fixing Textures in Meshi

Certain model parts like the nose and arms may need texture fixing. Here's how to leverage Meshi's AI texture editing tools:

  1. Smart Healing:

    • Draw a box around the problematic area and press ‘Smart Healing’.
    • For minor imperfections, repeat as necessary and apply to the model.

  2. AI Texture Editing:

    • Draw a box around the desired area, input a texture prompt, and hit ‘Retexture’.
    • Choose from the generated textures and apply the one you prefer.

Animating the 3D Model

Meshi allows for straightforward animation of the created models. Here’s how:

  1. Alignment and Rigging:

    • Position the character correctly, ensuring it’s in a T-pose or A-pose.
    • Mark body points for accurate rigging.

  2. Animating:

    • Choose from prebuilt animations like walking or running.
    • Blend different animations to create seamless transitions.

Download the resulting animated characters, which are compatible with Blender for further editing.

Importing and Refining in Blender

Once in Blender:

  1. Importing Animations:

    • Import the FBX files and organize them in the Nonlinear Animation editor.
    • Adjust textures and align them to avoid light emission issues in other software.

  2. Blending Animations:

    • Push down actions and set repeats for looping.
    • Combine walking and running animations adding blend values to ensure smooth transitions.

Enhancing Animations in Adobe After Effects

Adobe After Effects now supports GLB files, making the import process seamless:

  1. Importing and Editing:

    • Import your GLB file and configure animation options to align correctly.
    • Add HDR images, environment lights, and other effects for enhanced realism.

  2. Adding Facial Animation:

    • Use the "Live Portrait" tool to add facial animation and lip-syncing.
    • Record a video with proper lighting and frame rate for best results.

Refining with Upconversion Tools

Combat inconsistency in textures and quality using tools like:

  1. Creer AI Enhancer:

    • Import your final video and apply high-quality upconversion filters.
    • Adjust parameters for the best results in detail and fidelity.

  2. Audio Enhancements:

    • Using 11 Labs’ tools for converting and enhancing audio directly aligned with character animations.

Keyword

  • 3D AI animation
  • Meshi
  • Blender
  • Adobe After Effects
  • Text to 3D
  • AI texture editing
  • Nonlinear animation (NLA)
  • GLB files
  • HDR images
  • Environment lighting
  • Facial animation
  • Lip-syncing
  • AI video enhancer

FAQ

Q: What software is used for creating 3D models? A: The main software used is Meshi, wherein you can create, texture, and animate models.

Q: How is texture refinement handled in Meshi? A: Meshi offers Smart Healing and AI Texture Editing to fix and refine model textures.

Q: How are animations blended in Blender? A: Using Blender's Nonlinear Animation editor, you can overlap different animations and blend them for smooth transitions.

Q: Can Adobe After Effects handle 3D models directly? A: Yes, Adobe After Effects supports GLB files, allowing you to bring in 3D models with their animations.

Q: What tool is used for facial animation? A: "Live Portrait" is used to add facial animation and lip-syncing to 3D character models.

Q: How can the final video quality be enhanced? A: Tools like the Creer AI Video Enhancer can be used to upscale and refine the final video for better detail and fidelity.