Insane Image to Video - Gen 3 Runway

Insane Image to Video - Gen 3 Runway

Hey everyone, how's it going! Runway's image-to-video feature just went live yesterday, and I want to try it out with a few of the product images that I generated using DALL-E 3 (D3) and some of the images of people to see how good the image-to-video is right now for Runway.

I got an email about this, but I can't find it right now. But what I can do right now is go to their website and upload an image by selecting Gen 3 Alpha. This is a paid model, so I bought the subscription to show you how it looks.

Initially, you only had the option to add the prompt, but now you can upload images. I’m going to DALL-E 3 (D3) or Bing Image Creator and generating an image of a scented candle with the scent of chili. It seems contradictory, but I did it for fun. I took inspiration from a Lexica prompt that generated a cream with the essence of rice.

I went ahead and took the candle image and added a chili flavor to spice it up. I downloaded the image and then uploaded it to Gen 3 Alpha. You can add a prompt like “add movement to the flame, zoom out slowly, chili flakes moving,” but this is optional.

Runway charges 10 credits per second of video generation. I have 1,000 credits, so I can generate 100 seconds of video at this point, which cost around $ 15.

Once you select the important part of the image, you can crop it to fit the aspect ratio. Here’s the cropped image and some settings. You can add a seed number, which will help align the generation if you like a specific one.

You can also try another image. I went to Lexica and found a scenic view and a person’s portrait. You can use keyword prompts for animations. It’s helpful for product photography, animations, or real estate images.

Runway lets you generate multiple videos at once, which is great.

Example Generations

Upon generating the video, I noticed something interesting: the flame on the candle was moving, which was pretty impressive. It used out-painting techniques to add chili flakes, matching the prompt and zooming out smoothly.

Next, I tried animating a scenic forest image and a flower-blowing image. The forest had a monster running through it, which had nice face consistency and clean motion but with slightly awkward leg movements.

The flower animation, where I had flowers moving because of the wind, came out extremely natural. Even the leaves had subtle movements.

I also animated a Mage image, but it did not turn out as expected. The zoom-out and orb animation seemed incorrect, likely due to my prompt.

Finally, I tried a real estate image, generating a slow zoom-out showing the surrounding mountains. Although not perfect, it added extra scenic elements like a pool and somewhat maintained natural alignment.

Experiment with My Own Image

To experiment further, I tried my own image to create a timeline showing me aging from young to midlife and then finally becoming a cyborg. Unfortunately, the generation was not as good, failing to accurately show changes in my face and somewhat distorting the background.

Key Takeaway: Runway is great for non-realistic use cases such as imaginary creatures, fantastical landscapes, and other non-existent elements. However, for realistic scenarios, it can fall short.

Summary

In conclusion, Runway's Gen 3 is quite effective for animating static images, especially those of surreal or imaginary compositions. However, its performance dips when dealing with real-life elements. If you are into creative projects like animation, product marketing, or even real estate, it provides a considerable advantage in terms of animation quality and efficiency.

Keywords

• Runway
• Image to video
• Gen 3 Alpha
• DALL-E 3
• Animation
• Product photography
• Real estate animation
• Lexica
• Video generation
• AI models

FAQ

Q1: What is Runway's Gen 3 Alpha?
A1: Gen 3 Alpha is an advanced AI model from Runway that enables users to convert static images into animated videos.

Q2: Is Runway’s image-to-video feature free?
A2: No, it’s a paid model. Each second of video generation costs 10 credits, and a basic plan providing 1,000 credits costs around $ 15.

Q3: Can I upload multiple images for generation at once?
A3: Yes, Runway allows you to generate multiple videos simultaneously.

Q4: How effective is Runway for realistic scenarios?
A4: Runway is better suited for non-realistic, imagined scenarios rather than real-life ones, where it may struggle with maintaining natural consistency.

Q5: What kind of use cases can benefit from Runway’s image-to-video feature?
A5: It’s highly effective for animation projects, product marketing visuals, and creating engaging content for real estate listings.

Q6: What are some limitations of Gen 3 Alpha?
A6: The model may not perform well with existing real-life images, sometimes failing to maintain the natural look and feel in animations.