Lip Synch

New Lip Synch Breakdown Script

I’ve been working on lip synch today, using Lost Marble’s Papagayo.

It’s alright. I’m still using verion 1.1 since version 2 appears to have no Linux version, which is a shame.

Anyway, while it’s certainly a time-saver to let the computer do the breakdown into visemes, Papagayo is quite cumbersome for doing so with a scene with many voices. Each voice is broken down individually, their words spread out over the whole scene. It seems to take quite a lot of time to pull the phrases into roughly the right area before more fine grained work can begin.

So I decided to write a quick program to try and do this more efficiently. My program takes a script as input, breaks down the words into phonemes and the phonemes into visemes as Papagayo’s breakdown does (and indeed using their dictionaries etc.), but instead of spreading each voice’s lines over the whole scene, it puts them in order, placing the whole ensemble’s lines over the whole scene in order.

It generates a papagayo .pgo file and dumps that either to stdout or a file supplied on the command line.

Having done five minutes worth of lip-sync in the last couple of hours with it, I am please I wrote it, I think it’ll eventually save more time than it took to write.

You can download the script here:

(you’ll likely have to rename it from .php.txt to .php).

Warning: It’s big. Nine meg. Mostly those dictionaries.

The script should have a filename of the .wav file you intend to lip-sync to in the first line, the second line should say how many frames long the entire vocal parts take (usualy the same length as the Wav I’d think.

Any line starting with a name followed by a colon switches voice. All other lines are spoken.

I find it useful to add a spare voice for spacing, describing scenes during which there is no talking.

The code is licenced under the GPL, as indeed it must be since the dictionaries are taken from papagayo itself, which is also GPL.

This means you can copy it, edit it, redistribute changes, but must give others the rights to do the same with your result.

Sorry it’s in PHP, I started it in Perl but my Perl is rusty and I was just confusing myself with all the referencing and dereferencing that Perl uses, and it’s distinction between arrays and hashes, and the way arrays can only be one deep.

Leave a Reply