Custom robot race FACE MORPHS (lipsync) not working

[Roggvir]

Working on a new "humanoid face" head for Servitrons.

I made a custom head based on vanilla human female head, including morphs.

It works great when put on a human NPC, but when put on a test NPC using my custom ServitronRace, the chargen face morphs do not work properly - only the TRI morphs work, but any face customization relying on scaling/moving of the face bones does not have any effect.

 

What am i doing wrong?

 

Here is what i did:

 

1. Made the ServitronRace
   I make copies of HumanRace, HumanRaceSubGraphData, and DLC01HumanRaceAdditiveSubGraphData,
   and rename them to ServitronRace, ServitronRaceSubGraphData, and ServitronRaceAdditiveSubGraphData.
   ServitronRaceAdditiveSubGraphData is set to add to the ServitronRaceSubGraphData, which is used as a template by ServitronRace (same as how the vanilla HumanRace is put together).
   I edited some values, switched keywords ActorTypeNPC and ActorTypeHuman for ActorTypeRobot and new keyword ActorTypeServitron, changed the voice type, etc..
   (btw. i need to copy all three race records, so i can alter the subGraph data to remove the ability of using a Power Armor, or eating and drinking)
    
2. Set ServitronRace to use custom skeleton
   All three race records mentioned in #1, are set to use custom skeleton Meshes\Servitron\skeleton.nif.
   Vanilla game also has skeleton_faceBones.nif and skeleton_female_faceBones.nif, so i also made copies of those files and placed them in the same folder Meshes\Servitron\.
   All three skeleton files are a copy of the appropriate vanilla human file, only with added connection points to attach robot legs/torso when building the robot in the Robot Workbench.
    
3. Created custom head
   I made copies of:
   Meshes\Actor\Character\CharacterAssets\BaseFemaleHead.nif, saved as Meshes\Servitron\HumanoidFemaleHead.nif,
   Meshes\Actor\Character\CharacterAssets\BaseFemaleHead.tri, saved as Meshes\Servitron\HumanoidFemaleHead.tri,
   Meshes\Actor\Character\CharacterAssets\BaseFemaleHead_faceBones.nif, saved as Meshes\Servitron\HumanoidFemaleHead_faceBones.nif,
   Meshes\Actor\Character\CharacterAssets\BaseFemaleHeadChargen.nif, saved as Meshes\Servitron\HumanoidFemaleHeadChargen.nif.
   I edited the base head shape, and all TRI morphs using Outfit Studio - head shape was sculpted by moving and scaling some vertices, and all TRI morphs were edited accordingly (using clear slider data from unmasked vertices on all morphs).
   These files appear to be fine - after all, the TRI morphs is the only thing that is working even when the head is used on the ServitronRace.
    
4. Made a new headpart using the files from #3
   I made a copy of the FemaleHeadHuman head part, saved it as ServitronHumanoidHead, and changed all its file paths to point to the appropriate files from step #3.

 

 

This is of course just a minimal test scenario, not a complete mod.

Now, if i give the ServitronHumanoidHead to any human NPC, i can edit all chargen face morphs in CK, and see that it all works.

But if i create a new NPC based on my custom ServitronRace, or change any human NPC into a ServitronRace NPC, the TRI-based morphs do work (type of eyes/nose/etc.), but any bone-based morphs do NOT have any effect (things like size/position of the nose/eyes, or size of the cheekbones).

 

Any ideas?

[Roggvir]

I forgot to mention:

 

When you look at the vanilla human female head, BaseFemaleHead.nif, it does not contain the face bones, and as such cannot even be weighted to those bones.

A head mesh that does contain the face bones and is weighted onto them, is BaseFemaleHead_faceBones.nif, but i can't find that file being referenced anywhere.

 

So, i am missing an important piece of the puzzle - how does the game know it should use BaseFemaleHead_faceBones.nif, and where to look for it?

It seems obvious that the suffix "_faceBones" would be something hardcoded, and the game knows to look for such file at the same path where the BaseFemaleHead.nif is, so i placed my version of this file accordingly - yet, it does not work, so maybe the path is different? if so, how? where should i put it?

 

It is also possible that this is not the cause and maybe the game can find the file.

After all, i do not see any errors or warnings related to this file in the logs.

[Roggvir]

What the hell is even the point of having TWO virtualy identical files - one with face bones, and one without - why not simply use only the one with the bones?
This crap is sooo confusing

[Roggvir]

I made a new test plugin, as simple as possible, only a copy of vanilla HumanRace.

Then i went on adding and changing stuff, restarting CK after every single change and then checking if the face morphs are still working.

 

I gave my test race its own unique body and head, both still 1:1 copies of vanilla records.

Then i started changing things - added custom skeleton NIF, and then i set the head part to use my custom NIF/TRI files.

All chargen morphs kept working, until i removed my custom HumanoidFemaleHead_faceBones.nif - as expected.

CK also complained in the log with

Quote

MODELS:                Could not find model Servitron_TEST\HumanoidFemaleHead_faceBones.nif

Still, this is the first time i see this error.

My original plugin where the face morphs weren't working never threw this error, so its too soon to say the problem there was caused by the game/CK not being able to find this file.

 

Anyway, at least now i KNOW that the HumanoidFemaleHead_faceBones.nif is really required for things to work properly.

Before i got this proof, i wasn't 100% sure, so it's nice to finally know at least something - i'd call that a progress.

 

(now it stands to reason that the skeleton_faceBones.nif and skeleton_female_faceBones.nif are probably required too, although i am yet to see how their absence affects anything - maybe it can be noticed only in certain situations i am not yet aware of).

[Roggvir]

I think i now also understand why the two head NIFs - one with face bones, the other without.

It starts making sense once you realize what the bones are used for, eg. morhping the face during character generation or when making an NPC face in CK - after you're done, the game stores, and keeps using the resulting generated NIF without the face bones.

It probably helps with performance if every NPC in game doesn't load those 60 additional facebones - so, its not stupid at all.

[Roggvir]

Looks like the problem is solved.

I dont know exactly how, i just recreated the ServitronRace and its template/additive subGraphs again from scratch.

Few things i did do differently this time:

1. Set the ServitronRace as "ungendered"
   Since the new Servitrons will have the AAF Gender Override module, we dont need the real sex attribute anymore.
   This alone surely isn't what fixed it, but because of this i made the other following changes, so maybe it was something i did there.
    
2. Replaced all Male data with Female data
   I used xEdit to drag & drop all kinds of data from female values/attributes to male (did this for everything that had female and male data variant).
   I had to do this due to changing the race to "ungendered" - since the robot will be technically spawned as a male, but we want it to have feminine face and morphs and whatnot - i had to copy those female data into the male fields.
    
3. Removed all female related data from ServitronRace
   Reason for this was to clean the plugin from any unneccessary data, hoping that IF there is anything the game needs, resaving in CK will put it back.
   So i used xEdit to remove any female-related fields, and then resaved the plugin in CK.

The result is now a robot that works as usual, but can now have a custom fully working humanoid-like face.
 

...this is of course just a proof of concept, not a final result.

 

[Reginald_001]

Awesome stuff.

 

(I say that because I was(AM) so stumped on giving the DARRYL eyebot from my Extended Lore mod some proper lip sync and emotion. Now I know where to start looking into making that work.)

[Roggvir]

1 hour ago, Reginald_001 said:

Awesome stuff.

 

(I say that because I was(AM) so stumped on giving the DARRYL eyebot from my Extended Lore mod some proper lip sync and emotion. Now I know where to start looking into making that work.)

The battle is not over yet.

 

The robot is still refusing to move the mouth while speaking.

I thought its because the dialogue lines for any of the robot voice types do not have any lipsync data, but its still not moving mouth after changing the voice to FemaleEvenToned.

It does blink, looks around with its eyes, and "mfg morphs" console commands works too, so i hope it is just some small thing i overlooked.

 

(it is a different problem though, so i labeled this topic as solved - but if i find how to fix it, i will post to keep the info in one thread)

[Roggvir]

33 minutes ago, Roggvir said:

It does blink, looks around with its eyes, and "mfg morphs" console commands works too, so i hope it is just some small thing i overlooked.

It also makes an angry face when it becomes hostile after i attack it.

[Roggvir]

Little bit of progress...

 

When i try to talk to the robot, but there is no dialog and the robot keeps responding to my attempts with "Huh?" or "Yes" - its mouth is not moving when it says these things.

Because my test robot is lacking some keywords, or something that would give him any meaningful dialogue/combat/attack lines, there wasn't really much (if any) opportunity to see it speak (still, even those "Huh?" should have a lipsync, right?).

 

Anyway, i set my test robot NPC as a replacement for Myrna (a unique vanilla NPC), by using xEdit to simply change my robot NPC's formId to that of Myrna (i did this so i could use xEdit's side comparison of the Robot NPC and this surely working vanilla NPC, to see if i can find something that maybe does the trick).

I didn't change anything on the Robot (keeping the ServitronRace and everything), letting it completely override any original data from Myrna (including voicetype, which i set to FemaleBoston on the robot), but even without any changes, just being Myrna, it now had some idle and combat/attack lines.

So, while idle the robot says things like "Hi, how are you?", and while fighting it (i used console commands to disable the robots movement, so i could watch its mouth without it jumping around), the robot occasionaly said some combat/attack line like "You think i am scared?", and its mouth is moving.

 

So, now i have a proof, that all face morphs, mimics, face archetypes, lipsyncing, all of it can and does work.

I am still suspicious of the "Huh?" and "Yes" not playing lipsync animations, and i will keep investigating this to find out why the mouth didn't moved.

The robot probably needs to have properly setup dialogue lines with lipsync data, which i was missing.

[Roggvir]

26 minutes ago, Roggvir said:

still suspicious of the "Huh?" and "Yes" not playing lipsync animations

Even with the robot still altered to replace Myrna, as described above, these simple sounds are not playing any lipsync animations.

I think i found those lines in CK, and they do have the "Has LIP file" checked, but there is no way telling if it actually contains any working lipsync data.

 

Anyway, other lines do work (like the "Hi, how are you?").

[Roggvir]

One big problem:

 

If you create your robot as NPC in CK, everything works.

But if you build the robot from scratch in the Robot Workbench, the faceGen head (or face) is missing.

 

I suspect it is because the race of the default robot you start from, a protectron, does not use a faceGen head.
I'd expect the head to appear as soon as the race of the robot changes to my ServitronRace, but it does not.

 

If i change the default robot building recipe to my Servitron NPC, then the face does show when the robot is first built in the workbench.

But as soon as i change any robot part, the faceGen head disappears and never shows again.

[Roggvir]

For the sake of dilligence, i tried using the face mesh as a Robot Workbench Module:

• used one of the head armors for this, by changing the NIF path to point to the faceGen mesh.
• added the required ConnectionPoint:Children data to the NIF, so the workbench knows where this thing goes.

RESULT: not the solution

Of course the face shows on the robot when i install the modified head armor module, but it doesn't morph/animate when the robot speaks (and i did make sure its using some human voice which surealy has lipsync data).

I wonder if maybe some F4SE commands could be used to somehow refresh the 3D of the robot, to make the true faceGen head show (something like the old Skyrim NiOverride function set which had means to force the 3D to refresh after applying morphs - not saying that it would help, just wondering).

 

I removed "[SOLVED]" from the topic title, because technically it still isn't completely 100% solved.

[Roggvir]

FINALLY some progress...

Found a workaround for the face disappearing after modifying the robot in Robot Workbench.

The following conditions must be fulfilled in order to make it work:

• Custom robot race with faceGen capability, using custom head
  The process of creating such race may be different depending on what you want the robot to look like, or what is it supposed to be, and how is it supposed to work.
  I needed it to work with human animations, so i made copies of HumanRace, HumanRaceSubGraphData, and DLC01HumanRaceAdditiveSubGraphData, set to use custom skeleton and custom headpart for the face, set as "Ungendered" (i do not require otherwise, so i decided to keep it consistent with vanilla robots).
  I used xEdit drag&drop to replace all male data with female data (because the robots are technically all male, but i wanted this one to have feminine appearance, so i needed to put the female data into the male fields), and then i removed/cleared all female data fields, because CK kept complaining about some duplicate morphs (it seems it doesn't like if the same morphs with same IDs are defined for both male and female).
  - The skeleton is a copy of the vanilla human skeleton with added Connection Parents for bot legs and torso.
  Additionally to the skeleton.nif, vanilla game also has skeleton_faceBones.nif and skeleton_female_faceBones.nif, and even though i do not understand their purpose, i made copy of those as well, also with the added Connection Parents, and put them into the same folder as my skeleton.nif).
  - The headpart is using custom BaseFemaleHead.nif, BaseFemaleHead.tri, BaseFemaleHeadChargen.tri, and custom BaseFemaleHead_faceBones.nif (the last one being kind of a hidden requirement, not listed/referenced in the plugin data, but the game requires it).
   
• Custom default robot to use as a base
  The vanilla base robot (DLC01CompWorkbenchBot) is a ProtectronRace which is not set to have a faceGen head, so if you try building the robot in the Robot Workbench from scratch (thus starting with this base protectron), or try to modify any existing robot which is based on this vanilla base robot, there is no way to make the faceGen head appear (even after you change the robot's race to your custom, faceGen-capable race, it won't matter, because the actor will still have the Protectron set as its BASE).
  So, create a new NPC, set its race to your custom robot race, and set everything else the same way as it is on the vanilla base robot record DLC01CompWorkbenchBot, especially the keywords (at least the DLC01ModdableBot) and factions.

Now instead of using the Robot Workbench to create a robot from scratch (which would spawn the vanilla base Protectron), use Player.PlaceAtMe ######## to spawn your custom robot NPC - the robot will spawn with a faceGen head (unless you did something wrong).

If you set the NPC record correctly (copying the relevant faction/keyword data from DLC01CompWorkbenchBot), you should be able to modify it in the Robot Workbench to your liking.

When you change any part on the robot, the faceGen head will disappear, but there is a fix: exit the Robot Workbench, open console to select the robot and use disable, then enable it again, and the faceGen head should be back.

 

Obviously, nobody wants to force players to mess with the console, so the robot can be spawned using some script, maybe from a custom workbench, whatever... and the Disable/Enable commands can be also issued from some script - it is only matter of finding the most convenient way.

[zechosen]

I hope this works well, id love to see DLX synth skin working on it

[Danest]

Looks interesting!  For me at least, it wouldn't be too terrible if a robot had a face that wasn't fully functional.  I can picture a robot with ahuman face being able to make speech without needing to move the mouth.  I mean, it'd be cool to see full face functionality, but it's all good.  

[RadioActive8]

Awesome work, is it possible to download the mod?