DoubleCheeseburger's xVASynth Based Mod Voicification Thread

[Executaball]

15 minutes ago, FredOof said:

I was referring to your doublecheeseburger babodialogue and SLHH packs in this thread, however I just noticed that these actually do not come with an esl like the other packs that you have uploaded on your own thread. Your words about esp-esl cleared up my query regarding forward compatibility, however I am still just curious as to how come some of the bsa packs (e.g. babodialogue and SLHH) do not come with esl while others (the ones on your own thread) do if an empty esl is required to load the bsa?

 

Some of the bsa packs have an identical file name to the original mod .esp. Such if you have the original mod it will suffice to load it. But this means it may conflict if in the future the mod updates with it's own .bsa file, or if you wish to pack the mod into a .bsa file. For POP for example I had to merge the original bsa file to create a new one that overwrites the original one. 

 

So since then I've been mostly using the esp-esl method.

Edited July 15, 2021 by executaball

[judge007]

9 minutes ago, FredOof said:

I was referring to your doublecheeseburger babodialogue and SLHH packs in this thread, however I just noticed that these actually do not come with an esl like the other packs that you have uploaded on your own thread. Your words about esp-esl cleared up my query regarding forward compatibility, however I am still just curious as to how come some of the bsa packs (e.g. babodialogue and SLHH) do not come with esl while others (the ones on your own thread) do if an empty esl is required to load the bsa?

If the original mod doesn't have a "sounds" folder, the voice just need adding to a .bsa archive named the same as the plugin.

If the original mod has a "sounds" folder then it's better to add a second, eslified & empty plugin with original plugin as master and name .bsa after it.

 

That way the .bsa archive is registered to the appropriate mod either way.

Empty, eslified plugins use no FormID #'s and occupy little space in memory, while not taking up an .esp slot.

[FredOof]

53 minutes ago, judge007 said:

If the original mod doesn't have a "sounds" folder, the voice just need adding to a .bsa archive named the same as the plugin.

If the original mod has a "sounds" folder then it's better to add a second, eslified & empty plugin with original plugin as master and name .bsa after it.

 

That way the .bsa archive is registered to the appropriate mod either way.

Empty, eslified plugins use no FormID #'s and occupy little space in memory, while not taking up an .esp slot.

Got it thanks! I have no problem with installing esls for the voice packs as I am aware of their lightweight nature, was just curious of why some uses esl while other don’t haha

[judge007]

15 minutes ago, FredOof said:

Got it thanks! I have no problem with installing esls for the voice packs as I am aware of their lightweight nature, was just curious of why some uses esl while other don’t haha

I'm currently on my 15th mod and it's easier to just create the .bsa with an eslified esp, than keep thirty explorer windows open to check if folder "sound" exist.

 

[DoubleCheeseburger]

On 7/14/2021 at 6:56 PM, Mister X said:

 

Hey,

I wanted to try this out, but I stumbled upon something:

as for now, your xEdit script creates a file which results in a wav file of the same line for every voice that's possible. Is there a way to get which voices actually use that line in the mod? That way, xVA Synth would have way less to do, right?

Right. I'm hoping to finish with the new version of the xEdit script this weekend. It will do better job filtering extra dialogues out.

 

Although I noticed that some plugins do quite a lot of stuff via scripts. For example, BaboDialogue fills quest aliases via Papyrus scripts for majority of dialogues. So, in such cases, there is pretty much nothing we can do but to generate lines for all possible voices.

[judge007]

@DoubleCheeseburger is there a way to generate lip files other than Creation Kit?

 

Or selectively through Creation Kit?

Possibly a script that gathers the required lip files one or more .csv files.

 

I'm trying to do Sexist Guards mod and have finally gotten the .wavs out.

but keep choking in Creation Kit on Lip file generation.

 

[Mister X]

I tried creating lip files for Lauras Bondage Shop recently and had a funny observation: I started up the CK, which then loaded the needed plugins and started creating.

 

After a few seconds the window became unresponding, so no crash. First, I thought that CK wasn't even able to crash anymore, but in the normal File Explorer I could watch the number of files in the voice folder still rising up ^^ So I just let CK sit in its unresponding state and it really was able to create all lip files in one go and even closed itself properly at the end. I was quite buffled myself ?

 

Is that normal or should the window normally still react? As I said, it was my first try.

 

Oh and thank you @DoubleCheeseburger for that already awesome xEdit script, that automates the main bunch of work ?

[judge007]

12 minutes ago, Mister X said:

I tried creating lip files for Lauras Bondage Shop recently and had a funny observation: I started up the CK, which then loaded the needed plugins and started creating.

 

After a few seconds the window became unresponding, so no crash. First, I thought that CK wasn't even able to crash anymore, but in the normal File Explorer I could watch the number of files in the voice folder still rising up ^^ So I just let CK sit in its unresponding state and it really was able to create all lip files in one go and even closed itself properly at the end. I was quite buffled myself ?

 

Is that normal or should the window normally still react? As I said, it was my first try.

 

Oh and thank you @DoubleCheeseburger for that already awesome xEdit script, that automates the main bunch of work ?

Yup, does that all the time here too.

When it does crash, it'll pop up an error message and save a crash dump.

[Mister X]

Just now, judge007 said:

Yup, does that all the time here too.

When it does crash, it'll pop up an error message and save a crash dump.

 

Ok, that I got with SE CK all the time ^^ So then I tried it with LE CK which worked like a charm and ran the files through CAO at the end for SE usage ...

[judge007]

.lip & .fuz are the same LE & SE

 

No need to do that with Cathedral

[Mister X]

11 hours ago, judge007 said:

.lip & .fuz are the same LE & SE

 

No need to do that with Cathedral

 

Oh well, at least it then creates a working SE .bsa ^^

 

BTW, can I use Cathedral to create an LE .bsa, too? I don't want to install yet another helper tool and the Bethesda Archive.exe is kind of intricate to use

NVM, got aware of the usage profiles, need to use my glasses more often ?

Edited July 16, 2021 by Mister X

[judge007]

11 minutes ago, Mister X said:

 

Oh well, at least it then creates a working SE .bsa ^^

 

BTW, can I use Cathedral to create an LE .bsa, too? I don't want to install yet another helper tool and the Bethesda Archive.exe is kind of intricate to use

NVM, got aware of the usage profiles, need to use my glasses more often ?

I guess I need glasses too.

 

I thought you meant you were processing the files to be LE compatible.

 

Duh.

[DoubleCheeseburger]

19 hours ago, judge007 said:

@DoubleCheeseburger is there a way to generate lip files other than Creation Kit?

 

Or selectively through Creation Kit?

Possibly a script that gathers the required lip files one or more .csv files.

 

I'm trying to do Sexist Guards mod and have finally gotten the .wavs out.

but keep choking in Creation Kit on Lip file generation.

 

I don't know about any method outside CK that could generate .lip files with decent lip sync. Having tool like this would definetely make life much much easier.

 

If you don't really care about lip sync you could:

1. Use dummy .lip files as described in my guide. This is probably the fastest and the laziest way to make NPCs lips move.

2. Use Silent Voice Generator: https://www.nexusmods.com/skyrim/mods/82953. This tool is actually able calculate the duration of the .wav files and copy somewhat accurate dummy .lip next to it. Lip sync will still be inaccurate but at least the duration of the lip animation will be more or less matching the audio. The tool itself is pretty buggy at least for me though, and there is not much documentation provided on it either. Moreover, if I recall it correctly, it requires dialogue lines exported via CK.

3. @executaball actually created another tool that is able to calculate the duration of the .wav files and populate directories with .lip files. But I haven't seen it being shared anywhere publicly.

[judge007]

What I've been doing is:

 

Run CK to get Lip files

Then I start a command prompt and copy your dummy lip file across.

 

Then I merge CK's output with the rest.

But I'll talk to @executaball about his utility

 

Thanks!

[Executaball]

1 hour ago, judge007 said:

What I've been doing is:

 

Run CK to get Lip files

Then I start a command prompt and copy your dummy lip file across.

 

Then I merge CK's output with the rest.

But I'll talk to @executaball about his utility

 

Thanks!

 

 

Hope it helps, let me know if it works for you ^^

[DoubleCheeseburger]

Uploaded the updated version of the dialogue export script to the guide post. The file is named xVASynth Dialogue Export v2 Experimental.zip.

It should do a better job filtering out voice models that are not required for specific dialogue line. Different mods will benefit differently from it however. For example, the number of exported dialogue lines for SexLab Solutions is now 2.5 times less than before. On another hand, SexLab Romance did not benefit from these changes at all. And this makes sense, because Solutions is a quest driven mod with the finite number of speakers, while Romance is a sandbox speak-to-anyone mod.

I was able to use this version of the script to re-export dialogues from plugins featured in the original post without any problem, but changes I made are quite significant so please report any encountered issue.

 

The next big step would be to filter dialogue lines by faction members, but this can't be done easily or exported lines would be load order specific or be omitted because faction members can be changed at runtime. I have some ideas on how to minimize something like this from happening and still benefit from factions filtering, but this will definitely take time to implement. So maybe later 

[Beef Cheeks]

I'm reading on the entire "CK crashing" issue, but for me CK crashes and no files are put in Overwrite. The log doesn't show that it's generating any lip files at all. I've tried launching CK multiple times. The log shows it loads the ESP and then shortly crashes with no changes to the folders.

 

Not entirely sure what to do here. My voice folders are untouched.

 

Edited July 23, 2021 by XenoDrake

[Beef Cheeks]

On 7/16/2021 at 5:32 PM, Executaball said:

 

 

Hope it helps, let me know if it works for you ^^

Does this generate dummy lip files or actual lip files that sync?

 

Regardless, there is one bug: It expects the folder to be named *.esp. But shows the warning even if the folder is named *.esm. Might want to be sure it also allows for *.esl. All three are valid plugin files.

[Executaball]

1 hour ago, XenoDrake said:

Does this generate dummy lip files or actual lip files that sync?

 

Regardless, there is one bug: It expects the folder to be named *.esp. But shows the warning even if the folder is named *.esm. Might want to be sure it also allows for *.esl. All three are valid plugin files.

 

Dummy lip files but trimmed to be within one integer second of the actual .wav file. So at least the length will match. It's the same repeating phoneme files from Fus Roh D-oh. 

 

And yeah I know about that bug, it's just an arbitrary check so just skip the warning and it should be fine. I've been holding off bug fixes since I am trying to see if I can directly call FaceFX to generate real .lip files without Creation Kit. Though after some digging around in the binaries, I might have inadvertently found a way to fix the CK .lip generation crashing issue while still using CK. I'll do some more experimenting but if it works I'll push it with the next update of SynthUtil. The actual FaceFX lip generation is actually quite complicated it seems, it reads both the .wav file and the voice line, calculates the phonemes then streams it into its closed-source pre-processed neural network. 

[Executaball]

1 hour ago, XenoDrake said:

I'm reading on the entire "CK crashing" issue, but for me CK crashes and no files are put in Overwrite. The log doesn't show that it's generating any lip files at all. I've tried launching CK multiple times. The log shows it loads the ESP and then shortly crashes with no changes to the folders.

 

Not entirely sure what to do here. My voice folders are untouched.

 

 

Does your CK work normally while not generating lip files? 

 

Did you install

1. SSE CreationKit Fixes

2. FaceFXWrapper (in above link)

3. FonixData

 

 

[Executaball]

2 hours ago, Executaball said:

Though after some digging around in the binaries, I might have inadvertently found a way to fix the CK .lip generation crashing issue while still using CK. I'll do some more experimenting but if it works I'll push it with the next update of SynthUtil. The actual FaceFX lip generation is actually quite complicated it seems, it reads both the .wav file and the voice line, calculates the phonemes then streams it into its closed-source pre-processed neural network. 

 

Graphically smoothing out artifact spikes in the xVASynth exported audio and resampling the .wav to 16kHz before handing off to CK seems to resolve most of the problems. Since usually resampling the already quite heavily artifact-laden .wav from xVASynth with conventional methods usually makes it even worse.

 

Finally exporting in PCM 16 bit signed little-endian because that's the format FaceFX ends up converting to. 

 

Also there might be a bug in FaceFXWrapper's floating point conversion module causing memory overflow when an audio track of specific length is imported, but I'll need to test further.

 

So far generating 40,000+ .lip files for The Ancient Profession in CK and did not encounter CK errors. Some files that originally had heavy artifacts (the oldgrumpy voices, female Khajiit, etc.) will still process, won't crash CK, but take significantly longer (CK and FaceFXWrapper may hang on those files for 1-5 minutes before resuming full speed processing on other files)

 

 

 

 

[Beef Cheeks]

3 hours ago, Executaball said:

 

Does your CK work normally while not generating lip files? 

 

Did you install

1. SSE CreationKit Fixes

2. FaceFXWrapper (in above link)

3. FonixData

 

 

Yes.
Yes.
Yes.
Yes.

[Executaball]

5 minutes ago, XenoDrake said:

Yes.
Yes.
Yes.
Yes.

 

Okay just to confirm, you can load the mod you're trying to generate lips for in CK as normal without the generate lip argument? And it loads normally? Because it sounds like there may be a dependency / loading issue.

 

Usually if CK crashes due to a lip gen issue it will have at least attempted to create some temp files for processing which will be in your overwrite.

 

Also if you're trying to do DLC voices your mod needs to have the DLCs in it's master list for CK to automatically load them I believe.

[Seeker999]

I created a list of the nearly 60 mods (so far) which have voice packs. I will try to keep it updated as quickly as you guys are making the packs!

[Artful Dodger]

Installed doublecheeseburger's Fame, Romance, Spank That Ass and judge007's DrunkRedux, super awesome, but those loose files added ~40sec. if not more of loading time to my SSD, damn...

 

Anyway, this xVASynth stuff is truly amazing. Thanks !