Leaderboard

reone toolkit version 0.3 has just received a new tool. LIP Composer is a complete replacement of CSLU Toolkit and LipSynchEditor, enabling modders to create LIP files from text and audio files. And it does a better job, too. From my experiments, LipSynchEditor and derivatives incorrectly translate phonemes to LIP shapes. Most noticeably, shape 0 is being interpreted as phoneme EE, while if you look at the animation, it is clearly supposed to be rest position. LIP Composer algorithm is the following: Analyze the audio file and find continuous spans of silence, controlled by parameters min silence duration and max silence amplitude Split text into word groups, ignoring punctuation. By default, whole text is considered a single group. User can create groups by wrapping multiple words in parentheses. Match word groups to non-silent sound spans, indicated by white lines on the waveform. Number of word groups must be equal to the number of sound spans. Within each word group, convert words into phonemes using open-source CMU Pronouncing Dictionary, and evenly spread phonemes across the corresponding sound span. For every word that is not present in the CMU dictionary, user can add it's phonemes in the Pronounciation tab. Finally, convert phonemes to LIP keyframes, also creating rest keyframes for each span of silence. From my testing, resulting LIP files are almost undistinguishable from original ones.