Lip Syncing?

[Sith Holocron 2,473]

How does one do lip syncing now with tools that work?   A current tutorial would be most appreciated.  Seeing that neither Ndix UR or I will be doing this aspect of the update project for the Korriban Expansion mod means someone else is going to have do that.

The main reason I'm asking about this is because I've received two PM's this week about this.  (Apparently the notice that I'm not on the Staff anymore hasn't taken root.)

[JCarter426 1,214]

Nowadays, one doesn't. If you don't already have the necessary tools, there's no way to get them because of bad practices on the toolmakers' part. Unless the folks in Oregon get their servers running again, nobody can install CSLU, which is the only thing that will generate phoneme data that LipSynchEditor can read to generate .lip files for KOTOR. We could, in theory, find another tool to generate the phonemes. I've looked through LipSynchEditor's source scripts and I'm pretty sure if I had phonemes from another source as a text file, I could write a program to convert it from that format to CSLU's so LipSynchEditor would read it. Or, I'm sure, one of the many better programmers on this site could come up with a similar solution. If we had another tool. I've looked, and other people have looked, and it seems CSLU's might be the only one around that works and doesn't cost money. Although right now it doesn't matter how much you money you have if you don't already CSLU installed. You can't install the program unless their servers are running, and you can't copy an installation from one computer to another because the program checks hardware IDs and will refuse to run. Even though it's a free program.

Currently, the only way to generate lips is to convince someone who already has CSLU to make them. Though, it would be possible to do most of the work even if you don't have CSLU - most of the work is writing text files - and just have someone who has CSLU batch convert them and send over the files.

[Sith Holocron 2,473]

15 hours ago, JCarter426 said:

I'm pretty sure if I had phonemes from another source as a text file, I could write a program to convert it from that format to CSLU's so LipSynchEditor would read it. Or, I'm sure, one of the many better programmers on this site could come up with a similar solution. If we had another tool.

I thought @ndix URwas working on a phoneme project of his own before the porting situation was changed to allow stuff to be used between KotOR1, KotOR2, and SWTOR.  (I think the theory was to add the existing sound files into a database so you could create new dialogue using everyone's voices.)  Perhaps his material might be of assistance?

[eNoodles 31]

He was actually using syllable data that he was able to export from the CMU Sphinx project as a base. From what I understand, he was going to make his own tool to convert the syllable data into mouth shapes/phonemes for KotOR's LIP files. He told me he simply hasn't had the time to work on it yet.

[JCarter426 1,214]

That's the sort of thing we'd need, yeah. Any sort of speech recognition that actually works will work. The PHN format that the CSLU toolkit outputs is not that complicated. It looks like this:

Spoiler

MillisecondsPerFrame: 1.0
END OF HEADER
0 10 .pau
10 80 .pau
80 90 dc
90 140 dZ
140 280 E
280 340 N
340 350 kc
350 360 kh
360 410 s
410 490 s
490 510 tc
510 530 th
530 630 @
630 670 n
670 680 dc
680 690 d
690 760 I
760 790 N
790 820 bc
820 850 b
850 1030 aI
1030 1190 .pau

 

Those are just start and end timecodes (in milliseconds) followed by the phoneme. The only tricky part of the format is the phonemes, as KOTOR only accepts a few specific phonemes and LipSynchEditor is set up to filter them based on CSLU's naming conventions. So long as whatever replacement tool is able to output its phonemes as a text file, it wouldn't be difficult to write a program to convert its text format to match CSLU's, or edit LipSynchEditor to filter based on the new format.

[DarthVarkor 384]

I have all the tools required for creating custom lip files. I’d be happy to share them and write up a tutorial on how to use it. I’ve been doing it for about a year now so I feel confident I could write a cohesive tutorial. Or maybe even a video tutorial? Whatever people think would be easiest.

[Sith Holocron 2,473]

9 minutes ago, DarthVarkor said:

I have all the tools required for creating custom lip files. I’d be happy to share them and write up a tutorial on how to use it. I’ve been doing it for about a year now so I feel confident I could write a cohesive tutorial. Or maybe even a video tutorial? Whatever people think would be easiest.

All of the above? Text version and a video version please?

[eNoodles 31]

42 minutes ago, DarthVarkor said:

I have all the tools required for creating custom lip files. I’d be happy to share them and write up a tutorial on how to use it. I’ve been doing it for about a year now so I feel confident I could write a cohesive tutorial. Or maybe even a video tutorial? Whatever people think would be easiest.

That sounds great! Did you find these tools or create them yourself?

[DarthVarkor 384]

@Sith Holocron Sure thing! Give me a few days, as re-casting Zekk in my films is the priority at the moment.

@eNoodles Off the top of my head (away from PC atm) all you need is the LipSyncEditor software, dlg_editor & the makephn.tcl programme (and access to notepad, obviously). I think I was sent them a while back from either @JCarter426 or DarthYcey. As I say i’m happy to share these programmes.

Either way, I’ll try to get on this as soon as I can. If anyone has any questions, ask them here and I’ll address them in the tutorial.

[bead-v 251]

1 hour ago, DarthVarkor said:

the makephn.tcl programme

I'm interested in this as well... so this is not the CSLU thing?

[JCarter426 1,214]

It is the CSLU thing.

[bead-v 251]

So ....

21 hours ago, JCarter426 said:

You can't install the program unless their servers are running, and you can't copy an installation from one computer to another because the program checks hardware IDs and will refuse to run.

... it's not happening.

[DarthVarkor 384]

10 hours ago, bead-v said:

So ....

... it's not happening.

I don't recall ever having to install anything. I have the files in a folder on my desktop and it runs fine. Try this. But, I could be wrong. This contains the LipSync editor and the makephn.tcl file, extract the RAR and let me know if anything works.

LipSynchEditor.zip

[bead-v 251]

7 minutes ago, DarthVarkor said:

I don't recall ever having to install anything. I have the files in a folder on my desktop and it runs fine. Try this. But, I could be wrong.

LipSynchEditor.zip

That's the LipSynchEditor, yeah. But where are you gonna get the .phn file from? Of course, you could enter the data manually, but that's not what the folks above are talking about.

[DarthVarkor 384]

Just now, bead-v said:

That's the LipSynchEditor, yeah. But where are you gonna get the .phn file from? Of course, you could enter the data manually, but that's not what the folks above are talking about.

The .phn file is in that RAR folder. Navigate to: Extras - Importing - makephn.tcl

[bead-v 251]

2 minutes ago, DarthVarkor said:

The .phn file is in that RAR folder. Navigate to: Extras - Importing - makephn.tcl

That's not a .phn file, that's a .tcl file. You can't import that into the LipSyncEditor.

[DarthVarkor 384]

2 minutes ago, bead-v said:

That's not a .phn file, that's a .tcl file. You can't import that into the LipSyncEditor.

Uh, yeah... I knew that! ignore that then  (Haven't had my morning coffee yet!)

As far as I know, you use that .tcl to create the phn files. After you have your line saved as a mono wav file and a text file (containing a transcript of the audio file) with it both named the same, you drop their name into a .txt file named "list.txt" (say: "line01.wav, line01.txt) and run the makephn.tcl file using CMD, that then creates the phn files you can then import into LipSyncEditor to create the .lip files. If I'm on the right path here let me know and I'll start on the tutorial. Apologies if I got this wrong and got anyone's hopes up in the process, just thought this would help.

[bead-v 251]

You do use the .tcl, but that uses the CSLU toolkit, which you have installed apparently.
 

 

[DarthVarkor 384]

2 minutes ago, bead-v said:

You do use the .tcl, but that uses the CSLU toolkit, which you have installed apparently.
 

 

Oh, right. I'm with you now. Just ran a search through my downloads and I do have the CSLU toolkit, which is odd because I don't ever remember installing it. Must have been a long time ago. If what the others say about it no longer being available/installer not working is true, then it probably won't work. But I'll attach the installer anyway if you want to chance your luck.

cslu206.exe

[ebmar 893]

35 minutes ago, DarthVarkor said:

But I'll attach the installer anyway if you want to chance your luck.

cslu206.exe

Thanks for sharing, DV! The installer works, but in the end it failed. 🙁

It still needs access to a "download" directory which I have no idea where it was.

And wow. It seems this program side-by-side with The Miles Sound System wins the "Most Wanted Software 2018" awards lol.

[DarthVarkor 384]

1 minute ago, ebmar said:

Thanks for sharing, DV! The installer works, but in the end it failed. 🙁

It still needs access to a "download" directory which I have no idea where it was.

And wow. It seems this program side-by-side with The Miles Sound System wins the "Most Wanted Software 2018" awards lol.

Oh, that sucks. Afraid I can't be of any more help there, was worth a try anyway.

Unless "download directory" just means where you originally downloaded the installer to? Probably not but worth a shot I guess. 

[ebmar 893]

Just now, DarthVarkor said:

Afraid I can't be of any more help there...

No worries!

Quote

Unless "download directory" just means where you originally downloaded the installer to? Probably not but worth a shot I guess.

Nope, not working. This what I get from the failed installation process:

Spoiler

I have no idea with what's happening so, I hope to hear the good news from others on this attempt.

[JCarter426 1,214]

2 hours ago, ebmar said:

The installer works, but in the end it failed. 🙁

It still needs access to a "download" directory which I have no idea where it was.

That's not the installer. I mean, it is, but all it does is ping the CSLU server to download the actual installer. And their server has been down for over a year. Their server did go down once before, and came back after a few months, but never for this long. Their website is still online, though, so who knows what's going on in Oregon.

We've also tried manually copying the installation and making registry edits, but the checksum blocks the program from running because it checks hardware IDs and knows the other computer is not the one it was installed on. So you have to legitimately install the program by pinging their server which hasn't been online in over a year and without their server, the EXE is useless.

Direct all complaint emails to these people.

2 hours ago, ebmar said:

It seems this program side-by-side with The Miles Sound System wins the "Most Wanted Software 2018" awards lol.

I do have the installer for an older version of Miles (the real installer) that used to be hosted on their website for free, but I don't know if I'm allowed to share it.

[ebmar 893]

28 minutes ago, JCarter426 said:

That's not the installer. I mean, it is, but all it does is ping the CSLU server to download the actual installer.

...

Direct all complaint emails to these people.

Thanks for the insight and for the reference!

Quote

I do have the installer for an older version of Miles (the real installer) that used to be hosted on their website for free, but I don't know if I'm allowed to share it.

Did it have terms and agreements attached or any appearance of it when you first acquired the installer? If there are no points about restricting distribution- I think legitimately it was allowed to be redistributed then. 🤔 And even if there are any please could you share it. 😂 [different case with the newer version I believe, which surely not allowed to be.]

[JCarter426 1,214]

Huh, it seems to be a moot point because the Internet Archive saved it: download Miles.