LTX Studio (Beta): AI-Powered Visual Storytelling, From Script to Screen in One App.

LTX Studio can generate consistent characters across
storyboard panels - even if one character is a dragon!

While text to image, and text to video (and image to video) AI tend to be getting a lot of the press, the real exciting aspect of generative AI implementation is how it can be used to speed up creator workflow.

Being able to realize your creative vision in a shorter length of time can lead to more ambitious projects. Particularly if you're a team of one, with a very limited budget, but you one day dream of creating your own epic animated feature film.

LTX Studio (beta), a new 'all-in-one' AI film making tool, is not going to let you realize that dream from a single text prompt but, by bringing a bunch of generative AI technologies together, the developers have created a one platform workflow that can help anyone rapidly visualize and deliver a story from initial idea to finished film in days rather than weeks (depending upon how ambitious the project is).

Even better, you can try their platform for free, with the main difference between free and paid accounts being the amount of computing seconds per month.

Using LTX Studio

This is the main scene that opens my Resident Dragon story.
TET, Grrr Dog, Froyd the cat, and Red in the basement.
The characters are all my Cartoon Animator Rigs.

The Script

You can opt to start with a completely blank project, or input an idea you have for a story and LTX's AI will help you turn it into a script. I went the third option of entering an existing script from episode two of my Resident Dragon Series.

The episodes are short and, since one character is a dragon, I wanted to see if LTX could handle a character that wasn't human, and not living in its natural environment.

Casting

LTX auto generates a cast of character images based upon your script. If you don't like these characters you can edit their text description and regenerate them. 

For example, in a lot of styles Red the dragon was coming up as a blue dragon so I changed the description to make red more prominent as a color.

Each character is also assigned an AI generated voice that you can change if you feel it doesn't quite suit.

You can also add additional characters and select the video style for your project from the cast page too. While there are many different styles there aren't too many for 2D animation. I went with Comic Book Style and used one of my Resident Dragon gag cartoons (shown above) to further influence the art style.

LTX Studio's Cast Page. Initially the AI only generated the first four characters. I added
Froyd and Grrr Dog. While you can't use your own art to influence character generation
it is possible to do face swaps by uploading your own images. I wasn't game to try it
because I was relatively happy with these generations.

The Breakdown

I was quite impressed by the next step, the Breakdown. Here the AI generates a synopsis of your script as well as a summary description of each scene. You can edit these descriptions (and even add new scenes) but I was more than happy with what the AI produced. 

This is the completely unedited AI synopsis of my script which very accurately sums
up the key events of the episode. Each of my three scenes had a similar scene summary.

Once you're happy with the script breakdown, hit the 'start' button and the AI will generate a complete storyboard. While it's not always one hundred percent consistent with keeping the character likenesses the same across storyboard panels it is pretty close. Each frame is editable and can be regenerated if anything just isn't right.

It is possible to upload your own images if you wish, or use the Canvas edit feature, to draw your scenes for the AI to use as a guide for its generations. In both cases you're completely bypassing the style and characters established on the Casting Page - making that step almost redundant.

I could not get the AI to put my characters in this scene so I tried the Canvas Editor which resulted
in a better composition but not in a style matching my characters or the rest of the storyboard.

Storyboard

While the AI creates a storyboard, it does so based on the scene breakdown, and does not include any of the dialogue. It's up to you to go through each panel, adding the dialogue (and extra panels for it) as you go.

There's also options to add sounds and music as you go as well, but I didn't get to this before my computing time ran out.

I had a lot of trouble generating my storyboard panels to be anything close to what I imagined. It was particularly difficult if I wanted two or more characters in a scene. 

Many times my scene image generations featured a single character that was an amalgamation of the characters mentioned in the scene description. Quite a few times I got Grrr Dog wearing a suit and tie. It happened enough that I dropped Grrr Dog and Froyd the cat from scene they were in even though they're kind of an important to the conversation.

There's a very cool feature that lets you move the camera in a scene (up to a point) if you want to change the composition but I couldn't get my initial scenes to look right to even want to explore angles.

Easier But Frustrating and Disappointing

The storyboard stage is the engine of LTX Studio. I imagine if your project includes standard human characters in a photographic style you'll have more success than I did, with a very specific, animated project.

I don't know why the AI completely ignored all the dialogue in my script to build the initial storyboard? Dialogue is pretty important. Especially when each panel has a place for dialogue, and the ability to choose who is speaking it (as well as previewing their voice).

I had to go through and manually add more scenes with all my dialogue. Also problematic is that only one character can talk per panel. If you have two people in frame (or in my case, one person and a dragon), generating the video for that panel didn't always result in the correct actor speaking the line.

Which I found very frustrating because I wasted a number of generations hoping to get the right actor to speak the lines with no luck. Ultimately I ran out of 'Computing Seconds' for the month and was unable to fully generate my whole storyboard to video. Eight out of 23 panels remained as still shots.

You would think the AI would use the cast images to correctly identify the right character speaking in a scene. It also has the name of the character in the scene to reference as well.

While character consistency across scenes isn't perfect, especially in the comic style, the real downfall is scene consistency. My script only has three locations but in every panel the characters are clearly not in the same room. It may be the same type of room but not the exact same room across panels.

Three consecutive panels from my storyboard, all taking place in the same break room
with the two characters standing next to each other... but it's clearly not the same room
in any shot! Background consistency is a real problem.

It's was also next to impossible to reproduce the same, character position, shot and camera angle across panels - something that is essential in conversational style scenes.

There's many more issues I could raise, including the final generated videos of each scene which you have little to no control over even if you describe in the scene prompt what action is happening.

Overall

I really wanted LTX Studio to at least be good and leave me feeling it has potential. I do think it would be great for putting together ideas on the fly, just to see where they lead, because, with  generative AI, the output ultimately ends up being a compromise of what you imagined and what the AI could understand and prioritize from the prompts.

My storyboard and video was unusable for anything. Even to pitch an idea using the automatically generated pitch deck you can generate (if you haven't run out of compute time I guess? Since I wasn't able to do this). 

I'm sure with a little more time I could use the included generative AI tools like remove and infill to kind of kit-bash my scenes but it still wouldn't be close to my vision.

As it stands, once you use up your computing time for the month, you're locked out from actually doing any work. You can view your projects but you can't do anything that requires the AI.

I feel AI is not really there yet for a tool like this. At least not if you have a very specific vision. On the plus side, this is as bad as it's ever going to be. As AI generation improves so will this I'm sure.

What Really Didn't Like...

Commercial use of LTX Studio is only available on the US$175.00 per month business plan. While the developers say they're targeting professionals, it shuts out smaller creators who could really benefit from the accelerated workflow.

It's a deal breaker for me. I'm not paying for anything that doesn't give me commercial rights usage at the lowest tier. I use AI to augment my ideas. At that price point it feels like my ideas are being sold back to me after I've given them to this tool for free. 

By all means charge a premium for the extra computing time and premium tools but don't hold creative's ideas hostage in the process.

o---o--- ---o--- o---

Did you find this article useful?
Subscribe to my newsletter and get the
latest articles delivered to your inbox.