Jump to content
  • entries
    12
  • comments
    638
  • views
    13340

The Effin Export/Import Project


Bad Dog

11599 views

Edit: For the record, this tool is implemented and has its own thread here: 

 

I can't really do what I want to do with FO4 if I can't do proper weight painting and I don't have full control over the nifs I create. So I've decided to have a look at making proper import/export tools between Blender and FO4. I'm using this blog to track thoughts and progress notes along the way.

 

There are two possible routes:

  1. Extend the niftools package. That is currently being actively worked on, tho there was a long hiatus for the past few years, which meant I was stuck with old versions of Blender and a multi-step conversion process. The recent versions support the latest Blender and maybe have better export--I haven't yet seen a version which exports properly but I haven't tested the latest. Upside of this method is it ties into an existing project and extends the tools that have been around forever. Downside is this stack is very complicated--translation generated from XML files, read/write through pyffi, and so forth. I'm told the XML for FO4 has been written but pyffi needs updating to use it. Presumably higher levels of the stack would need work too? Dunno. 
  2. Bodyslide/OS has export/import to all games and they stay up to date all the time. Their nif code is cleanly separated into a package called Nifly, which is used by BS/OS, Cathedral tools, and Nif Optimizer. So it seems like robust code with an active community behind it. Downside is I'd have to access it from a Blender python addon, and I have to figure out how to walk the Blender representation of a model so as to push it at Nifly.

 

I've looked a little bit at both options. My feeling right now is that the hard part of the niftools route is digging into XML, generated code, and pyffi. The hard part of the Nifly route is figuring out how to bolt it into Blender and parse the Blender data model. That seems more straightforward. It's all known stuff--Blender addons and such. So until I hit a roadblock, I'm going to explore that.

 

The biggest issue is whether I can call the C++ Nifly code from a Blender addon. I hunted through other addons and found an import/export addon that does exactly that. So in theory I just copy their method.

 

So here's the project:

  1. Figure out how to make a Blender export/import addon
  2. Figure out how to make a DLL
  3. Make a dummy DLL and call it from a dummy export/import addon. Once that's done I know I can get data into and out of Blender.
  4. Make a DLL encapsulating Nifly
  5. Call that DLL from my Blender addon. 
  6. Keep adding features until done.

 

#1 is mostly done--I have a little script that pretends to be an export/import addon but does nothing. On to #2.

Edited by Bad Dog

175 Comments


Recommended Comments



Bad Dog

Posted

I do want to use this when I go back to skyrim modding, so skyrim-specific stuff is relevant. There, vertex alphas were used on the khajiit heads so the same bit of texture file could be transparent or not depending on which part of the mesh it was used on. I'm not sure it worked that well. Vertex colors were used by the hair shader to keep the hair color off jewelry, and that's maybe more important. 

 

Fact is, getting all the settings on a shader right is so complicated I think in the end you're doing a copy/paste anyway, even if by hand. And the niftools exporter is (was, last I checked) so finicky if you didn't get every setting right you either got a mess or it refused to export. What I mostly miss is importing the texture with the nif so you're good to go without a bunch of fiddling. BUT Blender doesn't read the latest dds formats so you're often screwed anyway.

Blaze69

Posted

5 hours ago, DocClox said:

I think the Skyrim animated load screens did it by making a video clip and then wrapping it in swf and using it as a replacement for the loading menu. That's going by the animated wench loading screen anyway.

Nah, MadMansGun has actual "physical" animated load screens; they aren't flat panes that show moving images/textures/SWFs, they have all the appropriate bones and stuff in them and they have a nif animation. If anything is used as reference for animated nifs and load screens, it should be those.

 

17 minutes ago, Bad Dog said:

vertex alphas were used on the khajiit heads so the same bit of texture file could be transparent or not depending on which part of the mesh it was used on. I'm not sure it worked that well.

It did not, and that whole "fur layer" thing was absolutely retarded and only made the neck seam more obvious and eyesore-y.

 

Quote

Vertex colors were used by the hair shader to keep the hair color off jewelry, and that's maybe more important. 

Are they, though? I thought all jewelry was kept as separate meshes without the hair tint shader/tag, but I guess I'll have to check.

Bad Dog

Posted

Look at the braided hair with rings at the ends of the braids.

MadMansGun

Posted

2 hours ago, Blaze69 said:

.

speaking of, you still have not re-uploaded the optional Selachii menu replacers.

DocClox

Posted

Here's a candidate for your Rogues Gallery of Misbehaving Nifs. TorsoRoboBrain.nif

 

image.png

 

I don't know it it's supposed to start like that or not, but I couldn't get it to load so it was aligned with the skeleton.

 

The head is pretty strange as well, although it looks like the glass canopy is maybe intended to slide open like that.

 

image.png

 

[edit]

 

That canopy is skew-whiff from the top down, as well.

 

image.png

 

Bad Dog

Posted

Ugh. Focused down on getting FFO headgear converted and managed to create a helmet that drove the exporter bonkers and that led to going down a crazy rabbit hole. Turns out when I export multiple morphs for a shape I wasn't recalculating the normals, which is a problem if the morphs are extreme. Which mostly they aren't, but still. Then I discovered the Blender will fucking lie to you about where the verts are unless you jump through a bunch of hoops and it took me a while to figure out what the hoops were. Then I realized that if I recalculate the normals for the different morphs I'm probably going to lose any custom normals, which I want because of the neck seam problem.

 

I ended up deciding that if you have custom normals that's probably what you wanted, even if you're exporting morphs. If you don't have custom normals, then they can be recalculated based on the morph. And I'm just not doing sharp seams at all (which are represented as split normals) because I can't see any consistency in what blender is handing me. You can still get a sharp seam either by beveling the edge or splitting the mesh, so I think that's acceptable.

 

Anyway I have all that handled and some reasonable progress on headgear as well. I've got a range of heads loaded in one file and I'm just loading up one hat after another. Useful because the flight cap has to go under the helmets, so they have to be big enough to accommodate. But it's going fast--fix the morphs, one export, copy in the shader, and done.

Bad Dog

Posted

Discovered that the raider's sackcloth headgear uses vert colors for no very good reason. Regardless, took it as an opportunity to implement vertex colors. New version up.

Bad Dog

Posted

Oh-kay, posted the export/importer over at Nexus. 

DocClox

Posted

Downloaded and endorsed. :)

 

 

Bad Dog

Posted

Returned to work on this thing. Got international characters working, skeletons that weren't imported by my importer, and I've started on shaders. The two handwork things that I still have to do is paste in the shader from another nif and paste in any extra data nodes. So those are high on the list to do.

 

Also I have a few skeletons that don't import correctly, particularly the super mutant skeleton. So I'll be doing that too before I get heavily back into the FO4 mod.

 

image.png

 

This guy's textures were imported along with the nifs. The only work I had to do was set one option for the alpha.

Bad Dog

Posted

New version up! Handles shaders correctly for import and export.

Bad Dog

Posted

Lots of progress being made. Can now handle all the usual sorts of extra data. Got held up looking for bugs in the SE export, but that's handled. Now I just need to teach blender about extra data, fix bugs and clean up, and this round of updates is done.

Bad Dog

Posted

I'm now doing import/export of skyrim models to field test the exporter (and to fix up my skyrim mods). Finding all kinds of fun things. First I made the heads disappear when you zoom in, then I made the ears not follow the head so they broke free and sorta floated around when you turned your head. I don't even know how that's possible. But it's gone now. I think.

 

I redid the mouth expressions, maybe unnecessarily because did I have a panther mouth already? Anyway, there's a separate panther mouth now. I was trying to have one mouth for all the citrus heads but it's just too hard to keep everything in sync. I'm having to redo the head chargen morphs because I was stupid and didn't import them before making changes. Just as well, means I can go in and make them all how I like them. 

 

BUT I'm exporting the nif directly to the game folder and I don't have to clean it up at all, so that's a win.

Bad Dog

Posted

REQUEST FOR HELP

 

So.

 

Animations.

 

Getting requests to support animations in this tool, which would be awesome, but I don't know what it means yet. Thoughts:

  • Pynifly makes a fully-connected skeleton but that's not the same thing as an animation rig. What more should be added to make a full rig? I know there are targets and constraints but I don't know a lot about how they're used. Is there a good rig out there that I could/should copy? Is that a kosher thing to do?
  • 3DS Max has the only animation exporter using an ancient plugin. It occured to me to wonder whether I could hook blender into that plugin. Thing is, a plugin has two parts: a bit of external code that's made separately from the host program, so at some level it doesn't know what the host program is or how it works. Then there's another piece embedded in the host program which calls the external code. As I understand it, the external code is havok and can't be duplicated. But by the nature of the beast, it can't be dependent on 3DS Max. The part that's embedded in Max must follow standard protocols for Max plugins, and those are often in plain text, tho I know nothing about Max. If I can reverse-engineer the Max side to see how the plugin gets called, I don't know why it would be impossible to build a blender plugin that calls the external code the same way.
  • I've made a short sneak replacer using a series of tools put together by Anton. What does the havoc thing give you that his method does not?

So the request: If you're an animator and want to see this work, let me know. I don't need you to do much of the heavy lifting but I'm going to need a lot of advice and to be pointed at the right tools. I'm also going to need help iterating attempts--without experience it can be hard to tell if your own stuff is broken or if you haven't understood how things are supposed to work and that's broken.

 

Feel free to promote this message in other communities. I don't know where the animators hang out.

MadMansGun

Posted

3 hours ago, Bad Dog said:

Pynifly makes a fully-connected skeleton but that's not the same thing as an animation rig. What more should be added to make a full rig? I know there are targets and constraints but I don't know a lot about how they're used.

in my case i don't even use rigs, i animate the skeleton directly.

3 hours ago, Bad Dog said:

I've made a short sneak replacer using a series of tools put together by Anton. What does the havoc thing give you that his method does not?

with max/havok 2010 a simple hit run then done export process without having to deal with hkxcmd after the fact.

Bad Dog

Posted

Huh. So with hkxcmd, you get everything the max/havoc thing would get you? 

 

Because another option, if hkxcmd gives just as good results but is more of a pain to use, is to put a wrapper around it so it can be called from an exporter by spinning up a subprocess with a command line.

Bad Dog

Posted

Huh. Thx. Okay, I'll announce it when I'm ready to do serious work.

 

Right now the supermutants are driving me right up the wall. They import correctly the first time, not the second; turns out to be related to the fact that the code I stole from OS only allows for a single reference skeleton. I fixed that bug of course the supermutants have their *own* skeleton. Now I have to decide if the best answer isn't simply to give the tool a way to load a different skeleton than human vanilla. 

Blaze69

Posted

8 hours ago, Bad Dog said:

Now I have to decide if the best answer isn't simply to give the tool a way to load a different skeleton than human vanilla. 

I have no idea how feasible this is (if at all), but... how's this sound?

 

Include some of the vanilla skeletons as reference just like the human one is, and either have a dropdown menu in the Import window that lets you pick which one, or try to guess from the nif itself somehow. That way both humanoids and "commonly used/edited" creatures would be supported right away.

 

And as for other nonstandard skeletons, the user could load the skeleton nif itself first which would be imported as an armature and then load the actual mesh nif into that armature. Only downside I can think of (again, assuming it's even possible) would be the fact that you'd end up with a full armature instead of just the bones used by the mesh, but I'd say that's not a dealbreaker.

Bad Dog

Posted

Yeah, those are both good ideas. There's really no reason not to load the skeleton first.  Right now I want to make sure the bone locations in the nif are used in preference to those in the reference skeleton. Good news is that I got everything torn apart and put back together and it seems to be working.

Bad Dog

Posted

13 hours ago, AmazingFun said:

As a newbie , i have used this for sse to export some static model, but all of exported nif have a ninode as root node instead of bsfadenode. so i operated in nifskope manually.

is there a better way to solve this?

 

Right now, no, and there should be. (Nothing wrong with changing the type in nifskope, but I want the exporter to do it in one step.) I'll put it on the list.

 

No discord channel. Is that a thing? Should there be? How many discord channels can the world hold?

Bad Dog

Posted

BTW, re ^upthread, I solved the problem with non-human skeletons by honoring the bone locations in the nif or blender armature over the reference skeleton. Ousnius says it should work that way already but either I broke it or it doesn't work in the exact way I'm doing things. With that change I'm able to handle the draugr and supermutant models correctly.

Bad Dog

Posted

My brain is melting and dribbling out my ears. I decided to go ahead and do collisions because wtf collisions should be done, and it's actually going kinda sorta okay in that I can read and write simple (box) collisions.

 

BUT I want it to work naturally from blender->nif, which means the shape you make in blender has to actually provide collisions at the right place in the nif. AND it turns out that collisions attach to something else in the nif and that something provides the base for whatever transform they use. WHICH means I have to figure out the center of the shape you're using for the collision, determine its offset in reverse back to whatever thing you're using as a base--either a bone, or a mesh, or the root node--and get the transforms right so they can be written to the collision. And collisions express their transforms in quaternions, which I hadn't had to fuck with yet.

 

I messed about with it some but finally decided the only way it was going to work was if I really understood quaternions absolutely and completely and had proper support for them in the code. So I've spent the last 3-4 days tearing apart my transforms code and putting it back together with quaternions integrated in. At this point I have that layer working again and now just need to ACTUALLY DO THE WORK of getting the collision transform right. 

 

Onward and upward...


×
×
  • Create New...