Immersive AI Voice Acted "Make Mission a Jedi" mod

[EpicStoffer 31]

Hello everyone,

First a little about the mod that I'm working on:

This is a very raw WIP progress update clip, but I wanted to be able to show people more clearly what I was working on, so I figurd I'd shar the clip anyway

The mod stage is currently:

Created the entire dialoge tree, and setup some conditionals to make Seelah behave immersively to the player in the relatively short conversation they have.

Setup scripts for making Mission a Jedi, and giving corresponding class coloured lightsabers (3), along with the "Vao Family Robe" (Custom Jedi Robe unique to Mission, slightly better stats wise to Jedi Mater robes).

Setup scripts that through dialogue, allow you to push Mission either to the Dark or Iight side. This means that if during the dialogue with Seelah, that you choose the DS conversational options for training Mission, that she'll have an altered aligntment to DS when she becomes a Jedi. This option is also present for the light side, however given as she's already LS from basegame, all I did with the LS path from a technichal point of view, was to match the the placement on the alignment scale (which were chosen for balance and costumizability reasons). The alignment choice gives you some additional control, over which kind of Jedi you'd like Mission to be, both from a technichal PoV (affects force power usage), but also from an RP PoV.

Begun work on using custom AI VA's for Mission and Seelah, and in progress of converting text to AI VA, along with fine tuneing the AI VA work. Lips files created, and will be continued to be created as I make more dialogue voiced.

Additionally I've begun work on Missions conversation tree Globally (i.e when you talk to her anywhere), where I've setup conditional scripts so that Mission will have unique alignment restricted responses to some of your questions. This is also a place where you get to ask Mission about everything in relation to becoming a Jedi, suddenly finding out she has a Sister, etc.

 

A brief TL DR overview of the dialogue with Seelah, is that she introduces Mission to "The Vao Legacy" that Mission discovers she's a part of, along with "The Vao Philosophy" that allows for Seelah to teach Mission either the DS or LS side of becoming a Jedi. She informs Mission of her Moter (who was also a Jedi), and the conversation also has a foreshadowing commentary thrown into the mix, as I have plans to continue work on the mod with a minor quest on Manaan once I've finished everything I've set out to do.

Mission also has a truly unique "Family Robe" that she gets from Seelah (and which belonged to Misions's mother), that's made from a bug interaction between a RH Robe and an interaction with JC's masters robe (I've gotten permission to use both in my mod, so as to be able to provide the texture installed without user having to do anything).

The Mod is near completion, with a TSL installer already setup, but I end up improving or adding new things to the mod, so ETA is currently unknown (particularly as I have an AI VA Quota limit that I'm dependent on). A "don't hang me up on it" estimate should be that it's done around 1 month from now at the latest (might very well be earlier).

Progress currently needed still:

Work on converting more of the text into AI VA, and fine tune it to sound good.

Continue work on Mission Global Dialogue file.


If people enjoy my mod, I have a continuation mod planned where I've begun the early steps on making it (very early steps), that involved meeting Seelah on Manaan who's got a lead on the person she believes killed Mission's Mother, that'll involve a minor "undercover with the Sith" aspect, kulmintating in a LS "Have Mercy" or DS "Get revenge" conclusion, that'll serve as the full conclusion to the K1 content I have planned in relation to Mission.



Sometimes it can be hard to retain motivation over longer periods of time (I've already spend countless hours on this project), so if anyone has any thoughts to share on what I've written, or just want to let me know that they're excited to see what I've come up with, it would all be very appreciated. ^^

Ps.

Will provide a WIP clip of what I've worked on so far, whenever I've managed to exhaust my new AI VA Quota for this month.
 

 

[Guest Qui-Gon Glenn]

This looks interesting. I assume that you were aware of this mod -

and maybe made use of some of the work there? It could have saved you a fair amount of work for some things, while others would obviously be discarded or replaced with something made for your more specific purposes.

If you were aware (probably) then happily ignore the above. If not, check it out... the items are OP as was the RedHawke usual, but many pieces of what you are doing are there, albeit implemented in the way of the early days when we knew so much less. It was an impressive mod at the time!

I do want to hear a sample of your new VO. All this new AI stuff frankly creeps me out, but in this case I can see how we can use it in a "harmless" manner.

At a certain point, this will become a legal issue, Max Headroom will report it.

[EpicStoffer 31]

Yeah, RedHawke's Mod is one of my primary inspirations for making the one that I'm making. 

Basically I loved the concept of what RH had done, and a lot of the work,  but I wasn't so much a fan of the actual execution of it. RedHawke's version was immersion breaking personified, with everything from the way things where written, to the fact that you couldn't even exit conversation, and basically only have the option to press "1" from start to finish.
    Which to me, made it feel little different from just adding the Jedi class manually outside of the game. However the concept for what RH had created was great, and so was the groundwork and amazing textures that RH had created for Mission's robes (despite me technically not using them like they were supposed to, and instead forcing a bug interaction to create a truly unique robe for Mission with). The stats on my version of the robe, is fully balanced, and only has a minor advantage over a Jedi Master Robe.

    So given that I'd always wished there was an immersive option ingame for turning Mission into a Jedi, and that I'd stumbled over RH's version of this, I figured I'd try my hand at my first Kotor mod, and see if I was able to create something else entirely, that'd behaved non-immersive breaking, while adding a bit of quality content (hopefully).

Thanks to the efforts of both RedHawke and Chainz, I've already used a few things from RH's mod (namely the robes RH made, that I force to create a bug interaction to create the unique robe that's going to be a part of my mod). I also used a few minor misc files that saved me some time.

However the Vast majority of what I've been working on with my mod, is obviously all my own creations (bar the ocassional assistance I've recieved from various helpful individuals of course). ^^


In regards to you wanting to hear a voice clip, I wouldn't mind showing you two quick lines real fast. I'll attach it to this message.

The "Final" clip is Mission speaking, and the "I'm so sorry" clip is her cousin Seelah speaking (based on Vette from SWTOR).
What do you think?

Final I Wont Forget Finished Spliced Wav.wav v2 Im so sorry mission.mp3

Small progress update:

I've exhausted my monthly quota (again), and managed to finally convert 100% of all text to audio, from the Dantoine section of my mod (Mission's Global Dialogue still needs to be voiced).

However I still need to do a lot of splice work, encode everything, create .lip files etc, so I'm still a bit off from completing the Dantoine section. However this should be the last that I need to do, to have fully completed all aspects of the Dantoine section of my mod ^^

Below is included an additional voice line from Mission for comparison.

MISSION Listne I know were busy and all.mp3

[DarthParametric 3,782]

40 minutes ago, EpicStoffer said:

What do you think?

Bravo. I can say with all honesty that this is the first AI generated VO for a game mod that I have heard that doesn't sound like complete and utter garbage (even that Geralt mod that had a custom model created by an AI site was pretty ropey). What website/service are you using to generate it?

I'm also curious as to the level of alteration you can apply, since you said the second voice was derived from Vette's lines, yet she is also voiced by Cat Taber and is using her regular voice (albeit a decade on from K1).

[EpicStoffer 31]

Thanks for saying so Parametric.

I use the "Elevenlabs" website to create these voices with, and I (sadly) agree entirely with you, in relation to all the other AI voices that I've personally heard as well. Small note though, that new accounts can't make these voice clones anymore, and now it's only a paid service to be able to produce these with new accounts.
    However with that being said, they've added a new "Design your own voice", that has roughly the same quality level as the clips I provided, best I could tell from a "glance" at them.
These you can create as a free account, so I'd definitely give it a look if you're interested in potentially making your own voices as well.

In terms of alterations to the voice, there's a decent amount of "re-generations" involved in getting just the right emotional inflection on words, and generally alter the voice to sound roughly how you'd want it to sound (even with using voice clones like I do). I'm probably feeling compelled to splice several generations of the same text manually together, for roughly around 30% of it, to get it all sounding just right.
Innately, Elevenlabs only supports sliders for "Stability", and "Similarity/Clarity" to directly manipulate the overall voice with for each generation, however with that being said, there's several ways you can attempt to make the AI add a specific emotional inflection to the way it says things. Say if you wanted it to SHOUT something, then you could write it as "She shouted: "Insert Text to be shouted", or "She said sadly: "Insert text to be said with sad inflection". This doesn't always work, but has a high degree of working.
Generally the AI will try its best to add the emotional inflection to words that it thinks it should, but it's very hit and miss, and either takes some re-generations to randomly get, or prompts like the one described just above.
 

In relation to using Cat Taber to voice both people, I personally don't think that you'd ever think it's the same person speaking, from just listening to it (I've yet to hear anyone point this out to me, unless I've specifically told them which template I based the two voices on).

In relation to her work as Vette, I'll say that yes I agree that there's a lot of similarities, but after having listened to near all of her voice files many times over, I personally believe that for the most part of her performance there is actually a genuinly (subtle) difference, despite obviously having used her Mission VA work as a foundation for creating it.

Given that the AI only uses around 8 minutes of audio, I just picked the voice files where I felt it was most pronounced, to try and give it that "same but different feel" to the point of sounding like two similiar but completely different people from one another.

I also think that if someone wanted to make a K2 voiced Mission mod, that if you went and took all the files from Vette where she sounds the most like Mission for the template, that you'd end up with the perfect voice for "An older Mission" fitting for a K2 setting (lowkey a dream of mine in the back of my head, if I ever manage to fully realize my goals for this project).

[DarthParametric 3,782]

21 minutes ago, EpicStoffer said:

if you're interested in potentially making your own voices as well

For mods my interest would mostly be in clones of existing VAs, as you are doing with Mission. For example, there are a number of Bastila's lines in K1 that I would like to correct for K1CP that are impossible to create without an AI (like the "phenome" line). I also had an idea for a Mission mod of my own, tempting her down the dark path throughout the Griff questline to provide an alternative option to killing her during the Lehon showdown. Seems I've missed the boat on this avenue though, at least without paying for the privilege.

[Guest Qui-Gon Glenn]

That was amazing honestly... hello future, tomorrow is today.

My quibble? I had always assumed it was Mission VAY-o, not VOW. This may be because I do not know the lore correctly, or because the AI reads dipthongs and pronounces them as such rather than being "intelligent" enough to understand this subtlety of language?

[DarthParametric 3,782]

5 hours ago, Qui-Gon Glenn said:

I had always assumed it was Mission VAY-o, not VOW.

NM03AEMISS14024_.mp3

NM11AAGADO02050_.mp3

[Guest Qui-Gon Glenn]

Aha... yeah.

So, this is the state of the art. It is impressive, most impressive, but not perfect yet. Still useful, and I imagine a little creative splicing could eliminate problems like that... just maybe limit how much she pronounces her last name in the mod so you don't have to do a lot of busywork. Splicing, is, splicing... if you know you know.

Someone needs to do this for Johnny Silverhand, just so I can make him say "Tsk, tsk" correctly rather than actually reading the letters lmao.

Or, ya know, CDPR could have caught that and asked him to re-record the line. This is the kind of meaningless crap that literally keeps me up at night (3:41am post)

[EpicStoffer 31]

Despite Parametrics proof of how it's supposed to sound, I'll admit I have a hard time adjusting my headcanon pronounciation after so many hours into the project

I've literally gone out of my way, to get this exact pronounciation

I'll give some thought to correcting it though, I'm guessing most others would rather the same pronouncation as ingame (obviously), but seeing as I've hit my quota for this month of AI VA generation, it most likely won't be corrected in the initial release.

Qui-Gon if you're able to splice a word in, without it being detectable, mid-sentence, then you're definitely a better splicer than I am I've tried doing that countless times in various ways, but ultimately I stick to only splicing at various pauses (think full stops, and longer comma breaks).

Right now I'm focused on getting through the tedium of splicing, converting, encoding, and creating .lip files. Once that's completed, I'll most likely release this mod (without the global dialogue changes). Then next month when I'm updating the mod and creating the voice for Mission Global Dialogue, if have quota left over, I'll see about doing some re-generations for the dialogue parts where the Vao pronounciation doesn't match how it's pronounced ingame. ^^

Thanks for all the feedback and comments so far, engaging in talks about the mod, has definitely helped me find some motivation to continue the work on the mod more often

Here's another small preview audio clip, and I apologize in advance to the people who's going to get irked by my current HC pronounciation

Seelah01.mp3

[Guest Qui-Gon Glenn]

I could do it maybe... My modding speed and delivery on promises is legendary here. I know a guy though. That said, he is in another country working and does not have time for my bs right now, and generally gets paid really well for sound work because that's what he does for a living. I can tell you where your lighting should come from, which technically should mean that I am a master of lightmaps. I am not. RL doesn't translate to modding always 😕

I love it that that is the pronunciation that you were going for! It makes me laugh, because I don't want that, but I want you to have it. And now you do. That said, I think most of us know Mission as Mission VAYo rather than VOW, and that can be a little guidance for how much you want the name to be spoken aloud, or that you may want to consider versioning of your mod for the patcher.

[EpicStoffer 31]

Quote

I love it that that is the pronunciation that you were going for! It makes me laugh, because I don't want that, but I want you to have it

I chuckled a fair bit over this

I'm not going to go as far as to make an entirely different version, I'll just stick with the current pronounciation until I've finished everything with the Dantoine section and Mission's global dialogue ^^
I'll get around to altering the pronounciation at some point, but it's a last priority at the moment. As I can't splice in an individual word without it being noticeable, it means I'm going to have to fully re-generate the segment spoken (or if I'm less lazy, whichever segment it's part of with a pause on either side of what's spoken so I can try to splice in a smaller part to put into the original audio file), both of which ar going to take up precious quota, and time I'd rather spend on actually "finishing up the project" first ^^

Don't worry, I won't hijack you to try and splice anything, I just wanted to highlight that it was outside of my abilities to do so, and give you a kudos if you were at a level were you're able to do so

Speaking of splicing, perhaps someone can offer some feedback on my latest splice attempt. I don't feel overly confident in it myself.

I've attached it to the post, and I Strongly advice that you listen to the "Unsure" file First.   Logic being that if you listen to them in reverse order, the difference is likely to stand out more, and affect whether the splice did wid well enough to "go under the radar" on a first listen.



 

UnsureSeelah04.wav Seelah04.mp3

[DarthParametric 3,782]

I don't think you can give a universal yay or nay. It ultimately comes down to individual user tolerance for that sort of thing. The pause due to the splice is definitely noticeable, but if the average slob can tolerate Kaevee they probably won't have too much to complain about with your work. And it's not like the original VO (both in KOTOR and many other games) isn't already full of jarring elements, like the Bastila examples I mentioned earlier. I'm pretty sure the number one complaint is going to be the name pronunciation.

[EpicStoffer 31]

That's a fair answer, though the splicework was only in *removing the pauses* (as much as possible) The "Unsure" file has two jarring pause segements significantly diminished, but I couldn't decide if it was as jarring as the pauses or not

[DarthParametric 3,782]

The pauses are a little awkward, but I think most people will probably give it a pass due to the overall quality. If it was janky robo-speech on top of that then it might be a different matter.

[ebmar 893]

Congrats with the attempt, @EpicStoffer -- impressed with what I've been hearing so far.

7 hours ago, EpicStoffer said:

Speaking of splicing, perhaps someone can offer some feedback on my latest splice attempt. I don't feel overly confident in it myself.

I'm guessing that AI had troubles mostly with pacing, so maybe you can try editing the tempo. Is there an editor for it? Perhaps you can adjust so they are not in a rush when speaking words that followed by "it", as noticed from your latest splices there. But overall the quality is good. Nice!

Was curious so I took an edit on how I'd like the whole sentence to be spoken, then maybe you can catch what I mean: unsureseelah04_test.mp3

[EpicStoffer 31]

Thanks for the well wishes towards the project, I'm excited for when I'm able to show a video clip of the dantoine section, whenever I finish it up sometime this month ^^

I tried listening to your file, but I think your file is broken, there's this strange robotic noise added to the clip now

There's sadly not an editor for it, so all of my edits has been me with Audacity so far. While the AI generation is great, it takes a lot of re-generations and splices, to get them all "just right".

It's still in the beta stage, so the user control is all over the place, and quite limited still.

[EpicStoffer 31]

First WIP clip of the mod so far
This is a very raw WIP progress update clip, but I wanted to be able to show people more clearly what I was working on, so I figurd I'd share the clip anyway

Progress update:
 I've gotten even further with the Dantoine section, and now mostly only need to "clean up" various things. All voice has been done, and only a single splice project, some minor written dialogue corrections. and that one voice reference correction, should be needed now (for the time being at least).

The choices you make in the Dialogue with Master Seelah, determines whether you push Mission's alignment further to the light side, or over to the dark side
At the conclusion of the dialogue, you'll have given Mission an alignment shift, and given her a Jedi class of your chosing
You'll also get a truly unique robe not seen Anywhere else (Thanks to RH and JC), along with 3 light sabers that corresponds in colour to your chosen class
After having gone through the dialogue with Master Seelah, Mission will unlock new dialogue choices when you speak to her (yet to be voiced)

Any and all feedback is welcome, though I'd prefer if it was constructive ^^

Thanks to Ebmar who provided assistance to me, when I suddenly got an issue with the dialogue speeding by in my mod (fixed now, and turned out to be an issue with the way I'd encoded the voice files the first time around).

[LDR 234]

That AI actually sounds like a person. 😀 It can still be a bit too monotonous at times... but overall it sounds great. Once AI can master emotional inflections then it's all over.

The future is now!

[EpicStoffer 31]

Fun fact LDR, emotional inflection control is one of the things they've been working on a lot recently, and going by their announcements about it so far, I already think I should be able to get a lot better emotional inflection control moving forward (though sadly that's also going to go hand in hand, with a lot more splicework for me).

Another downside of this enhanced emotional inflection, is that I expect that it's most likely going to take up more of my monthly quota, to get the optimal results from the new control we've gotten over how the AI VA says the TTS.

The new way it's now setup is to give you the emotiional inflection you want, by writing out your TTS sentences something like this:

"She shouted angrily: I don't want you to go there!". She continued in an apologetic voice: I'm sorry, I didn't mean to shout at you and be so angry"

Essentially we can now narrate the emotional inflection we want any given sentence to be said with, which I think is a huge step forward in gaining better results (I didn't have access to this feature, when I created all the audio in the WIP clip that I've shared).

Edit:

Only 5 days left before my quota resets, so hopefully I'll be uploading another WIP clip in the next 1-2 weeks of my progress at that point.

[EpicStoffer 31]

Progress Report:

I've fixed all the issues present in the WIP clip, meaning I corrected all the written audio to match perfectly with what was spoken, and fixed a couple of voice issues.

I worked a bit more on Mission's Global Dialogue, mostly voicing what I'd aready written, and added a bit more dialogue options.

I've conceptualized the major points that I want Mission to be able to talk about with the MC, while traveling along to the different planets.

New Idea Progress:


To give the mod some additional depth, and let the player have a more genuine influence on how Mission evolves as a character, I've also decided to try to incorporate alignment conditionals in her dialogue, so as to give her around 60-70% universal dialogue regardless of her alignment, and the remaining 30-40% be LS or DS exclusive dialogue.

I've also decided to add around at least 2 conversation prompts from Mission on each planet, sharing her newly found Jedi PoV on either the planet or before/after a major event.

Thanks to the help of Leulikin I've figured out how to setup and shape triggers, and my goal is for Mission to initiate conversation around at least 2 times on each planet, chiming in with her thoughts on the general aspects of the planet from her newly found Jedi PoV, and/or certain bigger events from the same PoV.

Thanks to the help of Fury  I've also managed to get my code working for the conditional required for the new idea (checking for being a jedi, their current alignment, and specifically which module they're currently in).  Thanks to SithSpecter, I also got a working module name for Dantoinne Courtyard, as "danm14aa" wasn't working with "GetModuleName" (Hoping that "GetModuleFileName will alllow me to use danm14aa for the module reference in the script).

I've also got a working prompt script on triggers that checks if Mission is a Jedi, and then prompt conversation between Mission and the MC.

Which effectively means that I've got all the technichal aspects mostly out of the way, and I'm now able to setup triggers to incorporate my new idea(s) with Mission, in relation to setting up at least 2 triggers on each planet, and letting Mission share her newly found Jedi PoV with you about the planet and major events, while also setting up the conditionals so that around 30-40% will be LS/DS exclusive dialogue.

Any inputs as to the new direction I'm taking the mod?


 

[EpicStoffer 31]

To give an impression of what I mean, here's a first draft example of one such trigger on Kashyyk:

Trigger point: Kashyyk - Somewhere around the Czerka office, probably towards the bridge area exit.

For DS (Czerka) Mission - I feel conflicted about the Czerka being on Kashyyk... On one hand, I know Big Z hates them, and because he's my friend I want to destroy them for him, but on the other hand... I can't really say that I blame them too much for using their strength to dominate those weaker than themselves, ya' know?

Reply option 1: I understand what you mean, I feel the same way.
Reply Mission 1: See? That's what I like about you. You understand that strength is the only real truth in the universe. [End Dialogue]

Reply option 2: Are you listening to yourself? This sounds like the Dark side talking
Reply Mission 2: You know what? I'm okay with that, if it gives me the strength to do what I think is right. [End Dialogue]

Reply option 3: I'm not really sure how I feel about it yet.
Reply Mission 3: Then I think you better make up your mind about it soon. [End Dialogue]

For LS Mission (Czerka) - I can't believe Czerka would enslave an entire race... This is wrong. I feel the force echoing with the cries of the enslaved Wookies..Don't you feel it <FullName>? The Czerka corporation are corrupting this planet, we have to help the Wookies somehow!

Reply option 1: I feel it too, we have to do something to help them soon.
Reopy Mission 1: See? That's what I like about you. You don't just do nothing when you see injustices commited around you. You stand up and fight for what's right, no matter what. [End Dialogue]

Reply option 2: This is the way of the universe, there's nothing we can do to change it.
Reply Mission 2: How can you say that? I thought Big Z was you friend, and that our mission was to SAVE the universe, not just stand by while it gets slowly destroyed! [End Dialogue]

Reply option 3: I'm not sure how I feel about it yet.
Reply Mission 3: Then I think you better make up your mind about it soon. [End Dialogue]


I might end up scratching some or all of what's written above, as I haven't cross references what dialogue is spoken by her in that module yet (and it's just a first draft), but I figured that this should give a better general impression of what I'm planning to add.

I figure some triggers will lead to slightly longer conversations, but basically this is the kind of way that I'd want to go about dividing up the unique alignment dialogue.

 

[EpicStoffer 31]

ElevenLabs has now officially ended support of voice cloning features, to the free accounts that still had access to it.

This means that unless this changes, I find another AI VA at the same level of quality (doubtful), or I end up paying for a subscription, that this mod project is going to be put on hold indefinitely.

I might release what I've got at some point as the WIP clip only with corrected dialogue, once I've made up my mind about whether to give up on the continuing the project because of the above reason.

[DarthParametric 3,782]

2 hours ago, EpicStoffer said:

another AI VA at the same level of quality (doubtful)

Nah, it's inevitable, it's just a case of being patient. It's likely not going to be doable via a website though, since most Western companies are going to be scared about legal ramifications. But eventually there will be better open source tools that will let you do it locally. The only question is how long it's going to take. Might be six months, might be six years.

[EpicStoffer 31]

Update:

Thanks to the help of Skpy, who's kindly offered to generate voice lines for me through Elevenlabs, I've returned to working on the project again

 

DarthParametric -> You're right of course, when I wrote the part you quoted, I was thinking of "withing a reasonable timeline, where I'd still have a burning passion for seeing my mod work completed".
I figured that based on what I've seen, and the general progress of it, that we were talking around 1year+ at the very least before a comparable free software was released (which I'll happily admit, was just a gut feeling estimate based on the above).