4 hours ago, Bad Dog said:

Right now, this system is blazingly fast, at least compared to other export/import. It will slow down as I add actual necessary features, but I expect it will have performance on the order of OS. 

Honestly, I'm fine with long-ass freeze times during import or export (as I'm already used to them) as long as the tool itself works, and I bet I'm not the only one. Besides, "OS performance" is still pretty damn good IMO anyway.

Yeah, OS performance counts as "blazingly fast" compared to other export/import.

Transforms are going to kill me, I just know it. I can export a simple shape, skinned, with bones and weights but it shows up as a dot in nifskope until I turn off skinning at which point I get my shape. So fine, I have to get the transforms right, I'm guessing (they're all 0 now except scale, which is 1).

But I cannot wrap my head around the transforms. Take the skyrim female hands for example. The shape is positioned at z=120, but Ousnius that's ignored. The skin is positioned at z=-120, which would seem to reverse the shape transform. This is the same setup as for heads. The verts are all between z=-60 and z=-40, so maybe the shape is lifted up to head position and the hands are relative to that. And thats fine, though I don't know why -120 lifts the shape, but maybe it's the inverse of what I expect.

What I can't get is the bone transforms. Every bone weight entry under NiSkinData has a different translation and rotation and I can't get how or why. The vertices already have positions. So does the bone transform warp that position to get a final position? But why is that useful? And why don't the hands get deformed if I turn off skinning in nifskope?

On import, if I take the vert positions as given, I get a recognizable mesh. And the bones are given positions in their NiNodes. And the shape is put in position by NiSkinData. So what are the skinned bone transforms doing?
 

Looking at how niftools imports blender meshes, I see that they have the +120 transform on the shape but I can find no record of the -120. Then vert positions are relative to the shape, and blender will show them in global coordinates or relative coordinates. The bones seem to be brought in exactly as they are listed in the bone list. None of which answers my questions about the transform that's on the bone reference in the skin. ?

49 minutes ago, Bad Dog said:

Transforms are going to kill me, I just know it.

You've probably noticed this already, but Outfit Studio has a specific "Coordinates" tab in the Shape Properties window (double-click on any loaded shape) that lists the skin transform at least for FO4, though it does so in a weird way (it seems to list the inverse of the transformation applied to the mesh from the "real"/base location in the nif to the actual location when skinned/ingame). So for example for standard FO4 nifs that transform in Z is "-120" which would mean you actually have to fetch the base location and move it +120 units in Z to get the skinned location.

Anyway, my point is, OS can read that data field, so it has to exist somewhere in the nif. Is there any way you can look at the way OS fetches it so you can read it as well? And also write to it when saving the skinned nifs yourself?

Quote

So does the bone transform warp that position to get a final position? But why is that useful?

I'm going to assume the reason here is the whole "half precision" thing. Half precision means vertex coords tend to get more scrambled the further away they are from the origin, but positions very close to the origin remain unaltered, so you want meshes to not get scrambled (e.g. for heads and head parts), you'll need your "base" vertex locations to actually be as close to the origin as possible and simply use the skin transform thingy to lift it back to its proper location ingame.

...the thing is, while that kinda makes sense for FO4, it does not for Skyrim since half-precision isn't a thing there, so I have no idea why Beth would have used skin transforms on Skyrim nifs in the first place.

For head parts, I'm guessing putting the head at 0,0,0 made it easy to do the head view in CK. Then the full-body view is skinned, so they show up where they should. Doesn't explain why the rest of the body parts are done the same way, except if you're loading them up in one view in your 3D modeling program it makes sense that all the settings would be the same. 

Looking at my claw hands, which were exported with niftools, seems that none of these transformations were maintained (either because I zeroed them in blender or niftools tossed them). But they work and they are simpler so I'll spend some time with them.

All right, I think I'm getting it--the model, anyway, and that's what was holding me up.

My issue was, since every vert is defined in 3-space, what's the need for all these transformations? But for a skinned mesh, the game doesn't care about the original vert location, it has to put the vert where the moving skeleton has moved it. So what matters is the vert's location relative to the bones that move it, both in position and orientation. And the game doesn't want to have to compute that at run time. So the there's a translation associated with the bone-in-shape that, if applied to each vertex affected by the bone, multiplied by the bone weight at that vertex, moves the vertex to the right position relative to the bone position. That transformation doesn't have to change as the bone moves because it's relative to the bone position and orientation.

So my simple tetragon had null transformations to its bones and I'm betting I put all the bones at 0,0,0 (I dragged the tails around but I bet I didn't move the heads), so all the verts were positioned at 0 when the thing was skinned. 

The bone NiNodes that every skinned nif has to include are stored with a translation and orientation, which has to match that of the base skeleton. Since bone-in-skin transformations are relative, if the actual skeleton has them in a different position the mesh will be warped or moved. This is how my digi feet work. But the vanilla nifs never seem to leverage this--they use a transformation at the shape or skin level.

There's a set of routines in OS that calculate and store the necessary transformations on store, and I'm just going to copy those and adapt from there. I'm still not sure about the need for a reference skeleton--sure, the bone position has to match between the nif and the skeleton, but if you loaded the bones from a nif in the first place, won't all the positions and orientations be correct? But I don't trust that I'll get it right if I try to be too creative at this point. 

5 minutes ago, Bad Dog said:

So what matters is the vert's location relative to the bones that move it, both in position and orientation. And the game doesn't want to have to compute that at run time. So the there's a translation associated with the bone-in-shape that, if applied to each vertex affected by the bone, multiplied by the bone weight at that vertex, moves the vertex to the right position relative to the bone position

Brain melting stuff. You're doing amazing work here

PARTY!

My first skinned nif file export:

Now, it's just from code not from blender. And it doesn't have any partitions. But weights and transforms seem to be correct. The code is lifting significant bits of OS, so now I need to encapsulate them properly, but the DLL wrapper on them, and build the python layers that will do this from code. So many steps yet. But this was an especially hard bit.

So I'm *writing* transforms from blender. Not sure I'm writing the right *ones* tho.

6 hours ago, Bad Dog said:

So I'm *writing* transforms from blender. Not sure I'm writing the right *ones* tho.

Gotta level with you, I'm not even sure what a transform means in this context. You've certainly done something to that mesh, though

6 hours ago, Bad Dog said:

So I'm *writing* transforms from blender. Not sure I'm writing the right *ones* tho.

 

Missing bones?

But some parts seem to be OK.

If you're awesome and you know it clap your hands  ? ?

 If you're awesome and you know it clap your hands  ? ?

 If you're awesome and you know it then your code will surely show it

If you're awesome and you know it clap your hands  ? ?

That's a weighted nif generated from a blender object fair and square, no cheating. Looks kinda... normal, huh? I swear I was pretty sure weights and transforms were gonna kill me. Y'all don't know how many hours I've spent beating my head against this. I know have tests that generate this model at four separate levels, because I was trying to isolate the problem, starting with calling nifly directly from c++ and moving my way up until I was generating from blender. (And it still wasn't working. Turned out to be I had to generate weights in the same order as bones. Which is pretty sure a bug down in the depths but I don't think it will be too hard to find.)

So tonight I celebrate and tomorrow maybe start thinking about partitions. They can't possibly be this hard surely? (There's actually all kinds of crazy cruft in there.) And nobody mention that this is just generating for one game, got to make sure the rest work too.

?You're just amazing ?

9 hours ago, Bad Dog said:

If you're awesome and you know it clap your hands  ? ?

 If you're awesome and you know it clap your hands  ? ?

 If you're awesome and you know it then your code will surely show it

If you're awesome and you know it clap your hands  ? ?

Lmao, that legitimately made me crack up. GG.

And yes, you're awesome. ?Can't wait 'till this is fully functional and released.

That's a raider outfit exported from blender and loaded into CK. As you can see, it's properly rigged up for the game. I haven't added the shader so as to be sure I was seeing my new mesh. Fallout's CK is sometimes a bitch about reloading files. That's a model I exported from blender directly to the FO4 directory and loaded into CK. No fricking around with multiple steps through multiple tools. I think I can now start playing with simple models like tails and gas masks.

I've been fiddling with transforms more, to make sure I cover the basic types used by FO4 and Skyrim. They both have a funky system for dropping the body parts to be down by the origin, but they do it differently. I'm sure I haven't covered every situation, but I am doing some of the most usual ones. I've been writing tests to explore the behavior and make sure I'm not doing stupid things. 

I should do partitions, but are they even a thing in FO4? I'm not seeing anything in the nifs that suggest they are. Maybe I'll ignore them for now.

More fun to go on to tris, so I can fix the lykaios. I want to rig the heads so the head morphs actually work in the game, which means figuring out _facebones files. I'm thinking make it possible to rig a head to both the regular bones and the facebones, then do one export. If the exporter sees regular bones it creates that file; if facebones the facebones file; if both, both.

I might do tails first because I think they don't need anything more than I have now. Then go on from there.

4 hours ago, Bad Dog said:

That's a raider outfit exported from blender and loaded into CK. As you can see, it's properly rigged up for the game. [...] That's a model I exported from blender directly to the FO4 directory and loaded into CK. No fricking around with multiple steps through multiple tools. I think I can now start playing with simple models like tails and gas masks.

?????????????

Quote

I should do partitions, but are they even a thing in FO4? I'm not seeing anything in the nifs that suggest they are. Maybe I'll ignore them for now.

As far as I know, they are a thing in the form of "Segments". IIRC they're supposed to be used for body dismemberment and possibly also hiding the outfit sleeves on the player (for the Pipboy, if you're one of those troglodytes that don't use PipPad or Tactical Tablet, lol), but CBBE and (I think) BT3 don't actually use them and thus don't support dismemberment due to some limit on the amount of vertices you can have in a segmented mesh or something like that I think.

Either way, the point is the only tangible way in which I've ever seen Segments aka FO4's Partitions being used is to create "hat-compatible" hair meshes where a portion of the mesh is hidden when wearing hats or caps. So I guess it wouldn't hurt to have it in the plugin as well if you're really invested in it, but it can also be done in Outfit Studio just fine so I wouldn't consider it a priority at all.

Quote

More fun to go on to tris, so I can fix the lykaios. I want to rig the heads so the head morphs actually work in the game, which means figuring out _facebones files. I'm thinking make it possible to rig a head to both the regular bones and the facebones, then do one export. If the exporter sees regular bones it creates that file; if facebones the facebones file; if both, both.

 

I might do tails first because I think they don't need anything more than I have now. Then go on from there.

This all sounds great and I'm really looking forward to it.

Do you have an example of hair or body that does that? I couldn't find one that had anything that looked like segments, including Piper's hair and the vanilla male body..

3 hours ago, Bad Dog said:

Do you have an example of hair or body that does that? I couldn't find one that had anything that looked like segments, including Piper's hair and the vanilla male body..

Any hair that only has a part of the mesh (as opposed to the whole TriShape) disappear when wearing the Military Cap or the Baseball Cap or the Ship Captain's Hat and such so there's no clipping with the hat itself 100% has segments.

Which should actually include Piper's hair indeed or else she wouldn't be able to wear her cap while still showing her hair underneath. So I dunno what to say here.

If you want a mesh where the segments were created 100% from scratch using Outfit Studio alone, check the mane with bangs hair from the Lupine race. I personally added those myself and know for a fact they work because hats hide the bang properly, so the data has to be in the nif somewhere.

I'll poke around with OS at some point and see if I can pin it down.

Meanwhile, vulpine tails are using cloth bones which aren't part of the normal skeleton and right now I'm only dealing with bones from the normal skeleton because that was the most straightforward way to deal with OS. I could add them, but I'll have to deal with unknown bones sooner or later and I'll probably take this opportunity to teach the code to do that.

LOL. I keep exporting test files for the wrong game because exporting to different games SO FUCKING EASY and all the games are FUCKING SUPPORTED.

First mesh exported through my system and loaded in the game:

Latest version of Blender and the only fiddling that was needed on the exported file was to copy the shader over.  Looking back over this log, that's 3 weeks to get to a usable FO4 model from a standing start. Pretty good I say, even if the OS folks did the heavy lifting. 

...and I'm reminded how much I hate the weight painting on the Lykaios necks. I'll look at that once I get the tails sorted.

Congratulations on your success !  You did a great job ?

12 hours ago, Bad Dog said:

Looking back over this log, that's 3 weeks to get to a usable FO4 model from a standing start. Pretty good I say, even if the OS folks did the heavy lifting.

To be honest it did take me by surprise how quickly you got the whole thing up and running, lol. Some really good work right here.

Can't wait to see how progress goes on FF now that the base tools are done(ish). But please do post the plugin itself publicly once you think it's ready, I'm sure there's a lot of people in the community that would love to get their hands on it and tinker around, maybe even help you figure out new features and solve bugs and such.

Nothing's ever freaking easy.

So... I was loading up the tails to exercise the code, and the export is fine (mostly, there's a thing about I can put body parts in the right place if you do them the way FO4 does and position them at the origin then lift them up to the ground by skinning, but I don't seem to be able to not do that, but it's fine because tails are body parts, right?) and they seem good in nifskope and when you look at them in CK, but they won't show up on a character. They turn out to be tricky because they have a bunch of cloth bones which aren't on the regular skeleton and I'm not sure how they're used. There's a BSClothExtraData which seems to have something to do with it (and which has 68K of data attached, what the fuck) but I don't know what. Maybe they just do a cloth physics thing and those bones aren't animated at all? 

Anyway I couldn't get the tails working. I played with a bunch of different things--loading the vulpine tail mesh instead, etc, etc--and wasn't able to get them working and I'm not sure if it's the nif or the record not identifying that body part or the materials file fucking up in some way. 

So I gave that up for a bad job and tried doing a better version of the helmet the DC guards wear and that would have been fine except Photoshop refused to open the diffuse, which makes it hard to figure what you're keeping and what you're cutting out. 

So I ignored that problem and thought maybe I could do a facebones thing and opened up one of those files and hoo-boy, that is not what I was expecting. The bones are all hooked up in parent/child relationships which, when skinned, lay the helmet on its side (why?) and there's a bunch of NiTransformController nodes which I have no idea what to do with. I still have the idea that I can do one general headgear which the facebones system warps, maybe for the different races or maybe just so the face shape sliders will work. But I don't know what to do with those and the armature was kinda funky so I gave that up for a bad job.

Then I exported my helmet--which doesn't look bad--and tried to copy in the shader and nifskope bit the dust. So I'm poking around to figure out why and freakin' A, I've exported for Skyrim. Because exporting for the wrong game is SO DAMN EASY because they're all FUCKING INTERCHANGEABLE. Maybe I'll tweak the code so if you have an armature it looks at the bones and defaults to the game that it seems to be for. 

So then re-exported the helmet--WHICH TAKES ABOUT TWO DAMN MILLISECONDS--and copied in the shader and the thing looks fine in game, tho I'll tweak it some because now I see how the texture lies on it there's some fiddling to do.

(Tho what is it with the characters turning their back on you when you go into free camera mode? I circle around to get a look at his face and he's like, fuck you dude, and turns his back on me again.)

Anyway.

This isn't nearly soup yet, but now that I've got ONE MODEL into the game I think I'll get all this pushed up to a github somewhere and maybe the community can have a look and see where I'm doing stupid stuff. There's a bunch about making it a well-behaved addon that I'm pretty sure I've got wrong.

13 minutes ago, Bad Dog said:

They turn out to be tricky because they have a bunch of cloth bones which aren't on the regular skeleton and I'm not sure how they're used. There's a BSClothExtraData which seems to have something to do with it (and which has 68K of data attached, what the fuck) but I don't know what. Maybe they just do a cloth physics thing and those bones aren't animated at all? 

AFAIK that "BSClothExtraData" thing is the physics data/info/config for the mesh, just like XMLs would be for Skyrim HDT pieces. Don't think you can create or edit those in any way other than with 3DS Max and the proper Havok plugin(s), but I could be wrong.

As for how the bones themselves are handled, they are not in the skeleton indeed and I assume they aren't "animated" either (other than inherinting the base position from whatever standard bone they are parented to, maybe), they are just physics'd up by Havok. So just like HDT stuff in Skyrim too.

Quote

Anyway I couldn't get the tails working. I played with a bunch of different things--loading the vulpine tail mesh instead, etc, etc--and wasn't able to get them working and I'm not sure if it's the nif or the record not identifying that body part or the materials file fucking up in some way. 

Feel free to send the file(s) my way so I can check them out if you want.

Quote

So I gave that up for a bad job and tried doing a better version of the helmet the DC guards wear and that would have been fine except Photoshop refused to open the diffuse, which makes it hard to figure what you're keeping and what you're cutting out. 

Make sure to grab the latest version of the DDS plugin (I think the good one is Intel's) so you can open BC5 and BC7 files (which the older one may not support). Otherwise just grab Paint.Net which should support all existing DDS compression types and can work to quickly check out textures and do minor edits.

Quote

So I ignored that problem and thought maybe I could do a facebones thing and opened up one of those files and hoo-boy, that is not what I was expecting. The bones are all hooked up in parent/child relationships which, when skinned, lay the helmet on its side (why?) and there's a bunch of NiTransformController nodes which I have no idea what to do with. I still have the idea that I can do one general headgear which the facebones system warps, maybe for the different races or maybe just so the face shape sliders will work. But I don't know what to do with those and the armature was kinda funky so I gave that up for a bad job.

Honestly I'd just opt for having base nifs of each helmet refit for each of the different races in particular (since head shape won't always be cross-compatible) and either not have any facebones mesh or have them just be the same as the "base" one but with Copy Bone Weights'd from the corresponding race's _FaceBones head mesh (if that makes any sense).

Quote

(Tho what is it with the characters turning their back on you when you go into free camera mode? I circle around to get a look at his face and he's like, fuck you dude, and turns his back on me again.)

Camera controls still work on the player even when in TFC mode, that's why. You can also check this out if you unholster your weapon and move the mouse up and down while in "weapon ready" stance in TFC: the PC will aim up and down matching the mouse movement.

The only way to avoid this is A) using TFC 1 (which fucks up something about how ENB calculates ambient light so it will make things look LSD'd to high heaven if you use ENB) B) unholstering your weapon and waiting for the "relaxed/weapon down" stance to kick in (in which case the PC will remain static) or C) using a pose (which will also keep the PC static).

Quote

This isn't nearly soup yet, but now that I've got ONE MODEL into the game I think I'll get all this pushed up to a github somewhere and maybe the community can have a look and see where I'm doing stupid stuff. There's a bunch about making it a well-behaved addon that I'm pretty sure I've got wrong.

Really looking forward to see where this goes, lol.

Well, maybe I figured out that the tails weren't working because I was missing slot 38 on the body record.

But now they're showing up warped (which the lykaios tails did *not*) and I think that might have to do with not handling the rotations on the cloth bones correctly.