Skip to main content

Revisiting Replica Studios - The Most Natural A.I. Text To Speech Based on Real Human Voices

Replica AI Voices Hero Website Image.
Replica A.I. Voices are great for
in game character voices.

I first trialed Replica Studios A.I. Voice Actor Text To Speech (TTS) voices shortly after they launched about a year ago and was suitably impressed by how natural they sounded. Some voices better than others but most sounding very close to human with the right inflections at the appropriate moments. Since then they have been working on improvements and adding more voices to their voice cast library.

I have used Replica Studios on projects in the past, most notably on my 2021 Reallusion, Lip Sync Contest entry featuring my character, Bat Storm. 

Mia and the Tourist and their R2-D2 replica droid.
Mia and the Tourist.

This time I've decided to use Replica to cast the other two voices in my upcoming Mia and the Tourist animated short that I've been blogging the process of creating through demonstrating the apps I'm using to make it.

So far I've covered scriptwriting, storyboarding, background creation, and character rigging.

When I say the other two voices I'm referring to Mia and the Car Rental Manager (see my post on scriptwriting if you want more details of this animated short). The Tourist is loosely based on myself so I typically use my normal speaking voice for this character.

Generating Replica Voices

Replica itself is a desktop app that connects to their server. Once installed and logged in you can begin creating voices right away. If you're using  iClone, Unreal Engine or Unity there are integrations available that let you use Replica directly from those apps.

Initially you start with 30 minutes of free credit and you can buy more from the app as needed. You can also earn credit by referring friends to the app with your unique link (as I've done with my link at the beginning of this article).

There are two ways to proceed in the app, the Sandbox, where you can generate individuals lines of dialogue and download as needed, or Projects, where you can enter an entire script worth of dialogue and download all at once. Unlike the Sandbox you can save multiple projects here too.

The Sandbox

Replica Studio's Sandbox interface. Change attributes like Speech Rate, Pitch, and the Style.
Replica Studio's Sandbox interface.
Change attributes like Speech Rate, Pitch,
and the Style.

The Sandbox is about as simple as it gets. Select a voice, type your dialogue, click Play to preview. Select any part of the text to make further adjustments like adding silence, volume, or changing the speech rate, pitch, or style (when options are available).

Saving a take will count towards your time credit so it's best to save only those takes you intend to download.

The sandbox will let you save takes with different character voices and dialogue so it is possible to generate all the voices you might need for a short project here.

When you have all your takes just download each individually with their download button.

Projects

Replica Projects Section helps keep your script organised and easy to enter.
Replica Projects Section helps keep your
script organised and easy to enter.

The Project section is a new addition to the app since I first reviewed it. At the individual line level it works the same as the sandbox but above that it allows you to pick your character voices and assign them to characters in your script. You can then enter your script broken up into its scenes.

The interface for entering dialogue is more of a list so you can work quickly, see an overview of key settings and make changes to text and character voices without having to get down to individual line level (you only need to go that deep if you want to play around with volume, speech rate, and pitch on individual words and phrases).

Just aware that if you preview the same line of dialogue multiple times it does start using up your credit. So keep an eye on this if your credit is running low.

Other useful features include the ability to move lines of dialogue around, and you can preview an entire scene with the play button at the bottom of the scene window.

Once you have all your dialogue entered you can download each line individually (which is more useful if you want to change a line of dialogue later) or download an entire scene at a time as a zip file in WAV, MP3, OGG, and FLAC formats. Each audio file is named after the line number in the scene.

Replica's Voice Library

New voices are being added to Replica all the time. You can browse the library and check out each voice with the pre-recorded samples just by going to the voice library section of the app. However it's actually easier to browse from the sandbox where, when you go to select a voice, you can filter the available voices with search options like Gender, Age, and Accent etc.

Like any Text To Speech system there will always be some voices better than others. Personally I think Replica still has some way to go before their voices will pass as human in every situation but many of them are good enough to use in situations that don't involve too many emotional extremes.

Adding the Voices to my Storyboard Animatic

As mentioned earlier I used Replica to voice two of my characters. The third is being voiced by me.

I recorded my lines using Audacity. I won't be covering that as part of this series but if you do want to get some insight into how I batch export my dialogue from Audacity into individual audio files for each line of dialogue check out my article, A Better Way to Export Clips From a Single Audio Dialogue Track in Audacity

For the purpose of letting you hear my generated Replica voices I imported my audio into Storyboarder and exported an animatic of my rough sketches (below). I wouldn't usually do this. Normally I'd make an animatic using my characters and sets in Cartoon Animator but now, having seen this, I'm glad I did.

As you can see (and hear) my Replica voices aren't as successful as I'd like. Mia's voice is kind of okay but has some issues with volume and pitch. The Rental Car Manager is all over the place. You can easily tell his voice is text to speech. I may have to try something else for him.

My own voice is fine, I guess, given the Tourist only has three very short lines.

Overall I think Replica's voices are coming along. While there is a good selection of voices not all of them work well as character voices. I really struggled to find a voice for Mia that I liked. The one you hear in the animatic is as close as I could get to what I imagined but it still sounds too youthful.

The next step for my animation is to start putting it altogether in Cartoon Animator.

Comments

Popular posts from this blog

Inochi2D - Free Open Source 2D VTuber Avatar Rigging and Puppeteering Software (Part 1)

Inochi2D Creator - Free Open Source VTuber Software. If you've been looking for a way to live perform as a 2D cartoon avatar on camera, whether it be for a live stream or for pre-recorded content like educational videos, then VTuber software is a low cost (or even no cost) option worth looking into. In my previous post, How to Become a VTuber - 2D and 3D Software for Creating and Controlling Your Avatar , I took a brief look at the relatively new but completely free and open source Inochi2D  which I thought showed great potential for my own needs of creating a live performance character rig for my own TET Avatar that I use for all my promotional materials. While it is possible to live perform my character using Cartoon Animator itself, Reallusion's MotionLive2D capture system isn't great - with lip sync in particular. More importantly though, I can't exactly teach people how to use Cartoon Animator if I'm using Cartoon Animator to control my Avatar. What is Inochi2D...

Krita AI Diffusion - Generative Image AI For Krita is Seriously Useful, Powerful and Free (If You Can Install it Locally)

Generative AI sequence of a woman in a business suit. From sketch to refined image using Krita AI Diffusion - by TET G enerative image AI, where you describe an image with a text prompt to an Artificial Intelligence model and it produces a new image based on your prompt, is gaining a strong hold as a tool for many artists. Krita AI Diffusion brings generative AI image tools right into your favourite free and opensource, graphics editor, Krita. Not only that, if you have a computer with decent specs (and at least 10GB of hard drive space), Krita AI Diffusion is completely free. What If I Don't Have a Powerful Computer? If you're in my situation, with a computer that was around before anyone in the mainstream had even heard of generative AI, you can still access Krita AI Diffusion for free, using a cloud based AI server, Interstice  and 300 tokens, to get you started. Once your initial tokens run out, purchase 5000 more for 10€ (approx US$11.00). Tokens never expire. I would...

Wonder Unit Storyboarder - Free Storyboarding Software for People Who Can (or Can't) Draw

Wonder Unit Storyboarder.  As an independent and solo animator I'm always tempted to try and skip storyboarding my animated shorts because they're usually only single scene sketch comedy type jokes. As a result I have many unfinished projects that kind of petered out due to having no clear finishing line. Storyboarding your productions, no matter how small, gives you a step by step guide of every shot that needs to be completed (no planning shots as you animate). It also allows you to create an animatic that gives you a rough preview of the finished production. In short, you shouldn't skip storyboards as they, generally, increase the chance of the project being completed. Disclaimer - I'm Not a Fan of Storyboarder Upfront, Wonder Unit's Storyboarder  is not my preferred storyboarding software. However it's completely free, has a number of very compelling featu...

The Ultimate Independent Animator's App and Resource List - Animation and Video Life

Image created with Cartoon Animator 4. Being an independent animator is not like a studio animation job. There's so much more to do that is indirectly related to the actual task of animating. Over the years I've sought out many apps, tools, and services that can help me achieve that one single task, expressing myself through animation. Below is my Ultimate Independent Animator's Resource List for 2024 (last updated Oct 2024). It started out as a list of free or low cost apps that could help you in every stage of producing either 2D or 3D animation, and then just kind of grew from there. You may not have been looking for a Time Management App as much as you needed something to get you started in 3D animation but when those commissioned projects start coming in you'll have a head start on maximizing your time. All the apps and services on this list had to meet two main criteria: They had to be useful and relevant to an Indy Animator/artist. The base app/se...

Glif (Alpha) - Make Your Own Tiny AI Powered Niche Image Generator Apps

I  first heard about Glif through a YouTube video that mentioned you could get access to Flux Pro (the latest 'game changer AI' generative image model) through the site for free. While I had a vague notion from the video of what Glif was, I wasn't expecting it to be so easy to get started with, and so good with my very first results. Glif is an easy to use, low-code platform for creating tiny AI-powered generators called Glifs. While that may not sound inspiring, what Glifs allow you to do is create a tiny app that niches down to a specific type of AI generation that the user modifies with their own inputs. The best way to really understand is with an example.  My First Glif I've recently been using VivaGo's AI platform  (free and unlimited at the time of writing) to consistently generate full body characters in a front facing T-Pose that I can rig as front facing characters in Cartoon Animator. Unfortunately it can be a bit hit or miss maintaining the T-Pose part...

Learn Reallusion's Cartoon Animator 5 Fast by Creating Fun, 2D, Animated Christmas Holiday Messages - Free Course by The Lazy Animator

H ave you ever wanted to send your friends and family fun, 2D animated, holiday messages, or just create 2D animated characters that you can make move and talk without learning difficult to master animation skills or spending hours drawing frame by frame? Reallusion's Cartoon Animator 5 is a professional quality, 2D animation studio software that is also very easy to produce impressive results with, even if you've never animated before, or can't draw. Don't Have Cartoon Animator 5 Download a free, 30 day trial . Easy Beginner Course I've created a completely free beginners course that teaches you my fast, easy way to rig and animate your first 2D custom character in Cartoon Animator 5 without complicated templates or any drawing skills at all. My online video course includes everything you need, including eight AI generated character sprite graphics, and four AI generated Human voice samples, so you can start learning right away. Eight AI generated character image...

LTX Studio (Beta): AI-Powered Visual Storytelling, From Script to Screen in One App.

LTX Studio can generate consistent characters across storyboard panels - even if one character is a dragon! W hile text to image, and text to video (and image to video) AI tend to be getting a lot of the press, the real exciting aspect of generative AI implementation is how it can be used to speed up creator workflow. Being able to realize your creative vision in a shorter length of time can lead to more ambitious projects. Particularly if you're a team of one, with a very limited budget, but you one day dream of creating your own epic animated feature film. LTX Studio (beta), a new 'all-in-one' AI film making tool, is not going to let you realize that dream from a single text prompt but, by bringing a bunch of generative AI technologies together, the developers have created a one platform workflow that can help anyone rapidly visualize and deliver a story from initial idea to finished film in days rather than weeks (depending upon how ambitious the project is). Even bette...