Jump to content

Project Echo - Voice Packs - Deep Learning Voice Synthesis

Executaball

Recommended Posts

Antiope_Apollonia

Antiope_Apollonia

Posted
Posted (edited)
11 hours ago, Roggvir said:

The dialogues in SLTR can be easily split into two categories:

  1. what the owner says
  2. what random npcs say

The owner can only be a follower.

You get yourself a list of all possible followers here, and once you get that list, you can easily put together a list of voices - except, you don't even need a list of voices, a formlist with all followers is all you need.

You could do one better and split the followers into male and female, then modularise the voice pack.  Most users are only going to be interested in playing with one or the other—Masters or Mistresses—so you could cut down on the download size quite a bit by allowing people to pick and choose which elements to install.  And people who want to play with both genders would still have the option to install both.

 

My guess would be that a number of these mods would admit some modularity in the voice packs, but I don't have experience with most of them to say for sure.  SLTR, though, I can say from experience would be a good candidate.

Edited by Antiope_Apollonia
Link to comment
Executaball

Executaball

Posted
  • Author
Posted
1 hour ago, Antiope_Apollonia said:

You could do one better and split the followers into male and female, then modularise the voice pack.  Most users are only going to be interested in playing with one or the other—Masters or Mistresses—so you could cut down on the download size quite a bit by allowing people to pick and choose which elements to install.  And people who want to play with both genders would still have the option to install both.

 

My guess would be that a number of these mods would admit some modularity in the voice packs, but I don't have experience with most of them to say for sure.  SLTR, though, I can say from experience would be a good candidate.

 

Well the only thing is I would end up with multiple BSA and esps each with a small size. Which is not *that* bad with SE ESLs but it'll eat into LE .esp limits if you're not using MO1's managed archives (like vortex or MO2 users)

Link to comment
Antiope_Apollonia

Antiope_Apollonia

Posted
Posted
15 minutes ago, Executaball said:

Well the only thing is I would end up with multiple BSA and esps each with a small size. Which is not *that* bad with SE ESLs but it'll eat into LE .esp limits if you're not using MO1's managed archives (like vortex or MO2 users)

Well, from an SE standpoint, that's a clear win.  For LE users, I guess it's pretty necessary to be acquainted with manually merging things to still be getting by on that antiquated platform, so people could always merge their desired modules themselves.  Or you could offer pre-packaged downloads for each combination of features, I suppose.

Link to comment
Roggvir

Roggvir

Posted
Posted
2 hours ago, Antiope_Apollonia said:

You could do one better and split the followers into male and female, then modularise the voice pack.

Yeah, for SLTR this may make sense, but i would also understand why nobody would want to do this, because it gets annoying having to maintain more and more download packages.
In my personal voice pack for SLTR, i separate most of the owner-only lines and keep them in separate packs per single voice, and install only the voices i think i may need, so it definitely is possible, but i wouldn't expect that to become a thing due to the hassle of maintaining more files.

 

What would make me happy enough, would be if the voice package authors would strive to make packages as "clean" as possible, without including unused files. Anything beyond that would be just a nice bonus, but nothing i couldn't live without.
 

Btw. if you look at the SLTR voice package available in this topic's first post, it states it contains 116,133 voice lines, which is TWICE the amount that is actually needed (minus ~4 thousand or so) - now imagine most, if not all, voice packs have similar or even worse amounts of unnecessary bloat - no disrespect intended to the authors, it's better to have something than nothing, and i know well enough that it takes work to do this in any form, but... i just can't stop thinking about all that wasted space, it irks me ?

Link to comment
Roggvir

Roggvir

Posted
Posted (edited)
1 hour ago, Executaball said:

Well the only thing is I would end up with multiple BSA and esps each with a small size. Which is not *that* bad with SE ESLs but it'll eat into LE .esp limits if you're not using MO1's managed archives (like vortex or MO2 users)

Do you know (at least roughly) what is the BSA vs. loose files ratio?
Is it a big difference? is it worth packing the files into a BSA?
Especially if the voices are compressed into XWM and stored as FUZ with LIP files also included inside?
I never checked the difference.

btw. loose files have at least one benefit, compared to BSA - the loose files are accessed only when needed, while the BSA is kept open all the time while the game is running, which costs a file handle - i don't know if that really could be a problem, even with some really heavy load orders, but if it could be a problem, it would be another thing to add on the proverbial scales when weighting whether to pack into BSA or not.
And if loose files would come out better from this, then that would solve the problem of N packages per voice pack as proposed by Antiope_Apollonia.

Edited by Roggvir
Link to comment
Executaball

Executaball

Posted
  • Author
Posted (edited)
4 minutes ago, Roggvir said:

Do you know (at least roughly) what is the BSA vs. loose files ratio?
Is it a big difference? is it worth packing the files into a BSA?
Especially if the voices are compressed into XWM and stored as FUZ with LIP files also included inside?
I never checked the difference.

btw. loose files have at least one benefit, compared to BSA - the loose files are accessed only when needed, while the BSA is kept open all the time while the game is running, which costs a file handle - i don't know if that really could be a problem, even with some really heavy load orders, but if it could be a problem, it would be another thing to add on the proverbial scales when weighting whether to pack itno BSA or not.

 

There is no noticeable size difference, since XWM is already a highly compressed format. The main issue is dramatically extended load times (at least in SE with MO2 or any VFS-based mod manager)

 

I assume it's due to the fact you only need to copy 1 dynamic link instead of thousands.

 

With 2/3 large voice packs in loose format your load time could go up to 4/5 minutes just to get to the main menu.

Edited by Executaball
Link to comment
Roggvir

Roggvir

Posted
Posted
4 minutes ago, Executaball said:

 

There is no noticeable size difference, since XWM is already a highly compressed format. The main issue is dramatically extended load times (at least in SE with MO2 or any VFS-based mod manager)

 

I assume it's due to the fact you only need to copy 1 dynamic link instead of thousands.

 

With 2/3 large voice packs in loose format your load time could go up to 4/5 minutes just to get to the main menu.

Holy crap, that is insane ?  And weird, the VFS implementation must be really badly designed for it to be that incredibly slow.
This ends any discussion, no way around it.

Link to comment
Executaball

Executaball

Posted
  • Author
Posted (edited)
25 minutes ago, Roggvir said:

Yeah, for SLTR this may make sense, but i would also understand why nobody would want to do this, because it gets annoying having to maintain more and more download packages.
In my personal voice pack for SLTR, i separate most of the owner-only lines and keep them in separate packs per single voice, and install only the voices i think i may need, so it definitely is possible, but i wouldn't expect that to become a thing due to the hassle of maintaining more files.

 

What would make me happy enough, would be if the voice package authors would strive to make packages as "clean" as possible, without including unused files. Anything beyond that would be just a nice bonus, but nothing i couldn't live without.
 

Btw. if you look at the SLTR voice package available in this topic's first post, it states it contains 116,133 voice lines, which is TWICE the amount that is actually needed (minus ~4 thousand or so) - now imagine most, if not all, voice packs have similar or even worse amounts of unnecessary bloat - no disrespect intended to the authors, it's better to have something than nothing, and i know well enough that it takes work to do this in any form, but... i just can't stop thinking about all that wasted space, it irks me ?

 

I did go over trying the process you mentioned. It seems the only difference for SLTR is the non-follower voices which I included since Skyrim does not provide a method natively to infer that information automatically. I'll look into that as a possible optimization for SLTR, but it doesn't seem like that might really apply to other non-follower mods? Most other information is available within the esp itself and level / quest data. 

 

Have you tried going through mods like ToH / Babodialogue / Slaverun etc. to see how many lines you got through your process?

 

Though from my end, looking over a plotted distribution of ToH voices, there are a few voices having more uniformness than the other ones. Not sure if that's really significant repetition or globally applied dialogue.

 

Fundamentally as far as I can see, everything besides followers (which is external information) could be added to an automated xEdit script without needing CK or manual parsing.

Edited by Executaball
Link to comment
Roggvir

Roggvir

Posted
Posted (edited)
58 minutes ago, Executaball said:

It seems the only difference for SLTR is the non-follower voices which I included since Skyrim does not provide a method natively to infer that information automatically. I'll look into that as a possible optimization for SLTR, but it doesn't seem like that might really apply to other non-follower mods? Most other information is available within the esp itself and level / quest data.

Yes, SLTR is probably one of very few mods, if not the only one, where the voice files can be meaningfully split in such way.

 

I do it in a lazy way by splitting the exported files from CK into two "batches" which i then process separately.
One "batch" contains dialog exports from the quests that contain strictly owner-only dialogue lines, namely these quests:

Spoiler

vkjArmorRestriction, vkjBringGold, vkjBurden, vkjCleanSweep, vkjEnoughAction, vkjEnterPalace, vkjFashionSlave, vkjFasterLolaFaster, vkjFeedMe, vkjFindTreasure, vkjFriendlyFire, vkjGagged, vkjHeavyStuff, vkjHomeBeating, vkjIdleComments, vkjLickFeet, vkjLightsOff, vkjMQ, vkjOwnerComment, vkjParadeWhiterun, vkjReturnToDFC, vkjRoadTrip, vkjRoped, vkjSex, vkjShaveHead, vkjStrongHand, vkjTrick


Another "batch" contains dialog exports from quests that contain non-owner, or mixed dialogue (for owner and/or other npcs), which is the rest:

Spoiler

vkjBardPlay, vkjConfessions, vkjDanceComments, vkjFetchBooze, vkjNewBuddy, vkjPimpedBasic, vkjPlaymate, vkjPonyExpress, vkjShopping, vkjSpanking, vkjSpankNazeem, vkjTagged, vkjTempleOfLove, vkjTrophySlave

 

I put each of these "batches" into a separate folder, and then run the _split.py script inside to generate the CSV for xVASynth.
Then i let xVASynth process the _ALL_COMBINED "mixed" batch, and for the "owners-only" batch i process it voice by voice.

 

It is not perfect, as some of those quests i keep in the "mixed" batch contain few owner-only lines, and i am not manually moving them into the "owner" voice packages.
So, when i install the mandatory "mixed" base package, it already contains few owner-only lines for owner voices i am not going to use.
But majority of the owner-only lines does end up in the "owner" packages, so it isn't a big deal.

 

 

58 minutes ago, Executaball said:

Have you tried going through mods like ToH / Babodialogue / Slaverun etc. to see how many lines you got through your process?

Not yet. I plan to do so, but i am swamped with stuff i need to do, si it will take a significant time until i get to it.
But i expect significant enough difference in file counts, when one tries to limit the voice types for export by adding some filtering conditions.

 

 

58 minutes ago, Executaball said:

Fundamentally as far as I can see, everything besides followers (which is external information) could be added to an automated xEdit script without needing CK or manual parsing.

MAYBE. It depends on the type of mod and how it intends any of its lines to be used.
For example, in SLTR, you have some lines that are to be spoken only by blacksmiths, or another lines only for bards, and another bunch of lines only for merchants - none of those intended for the followers.
If you do not set the appropriate conditions on those quests or topic infos, you endup with ~60-79 times the amount of lines (there is roughly 80 voice types that would all otherwise apply).
Let's say there is 100 lines for blacksmiths, and there is 26 blacksmiths in vanilla+DLCs game, some using same voice type, making it only 14 different voice types.
- If you set the conditions right, those 100 lines will result in 1.400 voice files.

- If you do not set the conditions and produce those 100 lines for every NPC-aplicable voice, you end up with ~8.000 files.
That is only for the blacksmiths. Bards and merchants are roughly the same situation, so if you don't use conditions, you create ~20.000 excess files that will never be used.


So, it isn't just about followers vs. non-followers.
Specific NPCs, factions, races, sex, just about anything can be a factor - it depends on the mod, you cannot blindly assume (if you care about not creating excess files).

Edited by Roggvir
1
Link to comment
Pfiffy

Pfiffy

Posted
Posted

Since you guys know how to make voice packs....

 

I'm missing female dremoras in Skyrim. And even if I create some on my own, I don't have a fitting voice for them. Is it possible for you to crate something more female out of the male dremora voice or create something new?

 

 

And 

 

(this will be hell of a work) Is it possible to make the player voiced?

Link to comment
Antiope_Apollonia

Antiope_Apollonia

Posted
Posted
7 hours ago, Pfiffy said:

I'm missing female dremoras in Skyrim. And even if I create some on my own, I don't have a fitting voice for them. Is it possible for you to crate something more female out of the male dremora voice or create something new?

It is theoretically possible, but as far as I know, xVASynth isn't anywhere close to being able to do so.  That's a much bigger technical challenge than having an existing voice say new things, and it requires a very different solution.

  

7 hours ago, Pfiffy said:

(this will be hell of a work) Is it possible to make the player voiced?

Already done.

Link to comment
Executaball

Executaball

Posted
  • Author
Posted
7 hours ago, Pfiffy said:

Since you guys know how to make voice packs....

 

I'm missing female dremoras in Skyrim. And even if I create some on my own, I don't have a fitting voice for them. Is it possible for you to crate something more female out of the male dremora voice or create something new?

 

Yes it's possible. I can train new voices if there are audio samples with annotations. Also there are numerous voices from other games if you're interested in anything there. Or I could just modulate the pitch of male dremora voices but I'm not sure how good that'd be.

Link to comment
Pfiffy

Pfiffy

Posted
Posted
37 minutes ago, Executaball said:

 

Yes it's possible. I can train new voices if there are audio samples with annotations. Also there are numerous voices from other games if you're interested in anything there. Or I could just modulate the pitch of male dremora voices but I'm not sure how good that'd be.

I stumbled over a female Dremora voice in the 'Daedra of Cold Harbor' mod, but that seemed very reduced. And it is not a source I can work with. I also don't know how big the differences between a voice pack for a mod and a voice type for the game is. Let me see if I can manage to add some female dremoras to the game and then we might check if you can do a voicetype for them.

1 hour ago, Antiope_Apollonia said:

It is theoretically possible, but as far as I know, xVASynth isn't anywhere close to being able to do so.  That's a much bigger technical challenge than having an existing voice say new things, and it requires a very different solution.

  

Already done.

I wonder if it will work with AE since it is only compatible with SKSE 2.0.19 

Link to comment
chaimhewast

chaimhewast

Posted
Posted
1 hour ago, Pfiffy said:

I wonder if it will work with AE since it is only compatible with SKSE 2.0.19 

It doesn't look like it has any DLLs of its own, so as long as you have a working PapyrusUtil it should be fine. People in the comments section are saying it works in AE, as well.

Link to comment

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Go to topic listing

  • Recently Browsing   0 members

    • No registered users viewing this page.

View Community Rules

Help Support LoversLab

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. For more information, see our Privacy Policy & Terms of Use