Jump to content
  • entries
    12
  • comments
    636
  • views
    10,888

The Effin Export/Import Project


Bad Dog

9,045 views

Edit: For the record, this tool is implemented and has its own thread here: 

 

I can't really do what I want to do with FO4 if I can't do proper weight painting and I don't have full control over the nifs I create. So I've decided to have a look at making proper import/export tools between Blender and FO4. I'm using this blog to track thoughts and progress notes along the way.

 

There are two possible routes:

  1. Extend the niftools package. That is currently being actively worked on, tho there was a long hiatus for the past few years, which meant I was stuck with old versions of Blender and a multi-step conversion process. The recent versions support the latest Blender and maybe have better export--I haven't yet seen a version which exports properly but I haven't tested the latest. Upside of this method is it ties into an existing project and extends the tools that have been around forever. Downside is this stack is very complicated--translation generated from XML files, read/write through pyffi, and so forth. I'm told the XML for FO4 has been written but pyffi needs updating to use it. Presumably higher levels of the stack would need work too? Dunno. 
  2. Bodyslide/OS has export/import to all games and they stay up to date all the time. Their nif code is cleanly separated into a package called Nifly, which is used by BS/OS, Cathedral tools, and Nif Optimizer. So it seems like robust code with an active community behind it. Downside is I'd have to access it from a Blender python addon, and I have to figure out how to walk the Blender representation of a model so as to push it at Nifly.

 

I've looked a little bit at both options. My feeling right now is that the hard part of the niftools route is digging into XML, generated code, and pyffi. The hard part of the Nifly route is figuring out how to bolt it into Blender and parse the Blender data model. That seems more straightforward. It's all known stuff--Blender addons and such. So until I hit a roadblock, I'm going to explore that.

 

The biggest issue is whether I can call the C++ Nifly code from a Blender addon. I hunted through other addons and found an import/export addon that does exactly that. So in theory I just copy their method.

 

So here's the project:

  1. Figure out how to make a Blender export/import addon
  2. Figure out how to make a DLL
  3. Make a dummy DLL and call it from a dummy export/import addon. Once that's done I know I can get data into and out of Blender.
  4. Make a DLL encapsulating Nifly
  5. Call that DLL from my Blender addon. 
  6. Keep adding features until done.

 

#1 is mostly done--I have a little script that pretends to be an export/import addon but does nothing. On to #2.

Edited by Bad Dog

175 Comments


Recommended Comments



Okay, I have a github up at https://github.com/BadDogSkyrim/PyNifly. If you just want to try the import/exporter, io_scene_nifly.zip is at the top level. Right-click, save link as... and you're good to go. I'll create a proper download page soonish, but I want to exercise it some more and I don't want a bunch of support requests until the FFO stuff is better baked.

 

Don't expect too much. But I just made all the tails for FFO, so it actually is getting useful.

Link to comment

I'm getting "invalid archive" reports when I try and open the zip. Windows can't do it, nor can 7zip.

 

Is it just a zip of the io_scene_nifly folder?

 

[edit]

 

Opened the zip with notepad++ - turns out it's HTML. Looks like a github page.

Link to comment

Ok, not a total surprise. I have it set up so it will work whether or not you have a development environment and of course I do, so hard to test the case where there isn't one.

 

Dunno why the zip got borked.

 

Will look and update.

Link to comment

All right, new version up. Let me know if it finds the addon path correctly.

 

Turns out if you want the zip, you have to click on it then hit the download link on the new page.

Link to comment

Well, you zip file is still borked. What it's doing is zipping up the github error page shown if you just click on the zip.

 

image.png

Link to comment
24 minutes ago, Bad Dog said:

All right, new version up. Let me know if it finds the addon path correctly.

Still throws an error, but now it's this one:

Traceback (most recent call last):
  File "D:\Program Files\Blender Foundation\Blender 2.92\2.92\scripts\modules\addon_utils.py", line 351, in enable
    mod = __import__(module_name)
  File "C:\Users\______\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\io_scene_nifly\__init__.py", line 48, in <module>
    from pynifly import *
ModuleNotFoundError: No module named 'pynifly'

 

12 minutes ago, DocClox said:

Well, you zip file is still borked. What it's doing is zipping up the github error page shown if you just click on the zip.

Are you sure you're downloading it properly? You need to go to this link: https://github.com/BadDogSkyrim/PyNifly/blob/main/io_scene_nifly.zip and hit the "Download" button on the right, that should give you the .zip itself.

 

Download.png

Link to comment
41 minutes ago, Blaze69 said:

Are you sure you're downloading it properly?

 

Apparently not. I've been right-clicking and typing "save as", as per original instructions. (And totally missed the revised instructions in BD's last post ... oh well  ?‍♂️ )

Link to comment

Grr. Gotta figure out how to test that on my machine, but I can't just change the folder name because the system goes haywire. Looking...

 

Link to comment

Hey @Blaze69, it would be helpful to see what's in blender's system console (Window -> Toggle System Console). I print some paths there that might help me figure out whatup.

Link to comment
24 minutes ago, Bad Dog said:

Hey @Blaze69, it would be helpful to see what's in blender's system console (Window -> Toggle System Console). I print some paths there that might help me figure out whatup.

 

Read prefs: C:\Users\____\AppData\Roaming\Blender Foundation\Blender\2.92\config\userpref.blend
addon_utils.disable: io_scene_nifly not disabled
addon_utils.disable: io_scene_nifly not disabled
Modules Installed (io_scene_nifly) from 'D:\\DOWNLOADS\\io_scene_nifly.zip' into 'C:\\Users\\____\\AppData\\Roaming\\Blender Foundation\\Blender\\2.92\\scripts\\addons'
PyNifly addon path: C:\Users\____\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\io_scene_nifly\__init__.py
Nifly DLL at C:\Users\____\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\io_scene_nifly\__init__.py\NiflyDLL.dll
ERROR: pynifly DLL not found
Traceback (most recent call last):
  File "D:\Program Files\Blender Foundation\Blender 2.92\2.92\scripts\modules\addon_utils.py", line 351, in enable
    mod = __import__(module_name)
  File "C:\Users\____\AppData\Roaming\Blender Foundation\Blender\2.92\scripts\addons\io_scene_nifly\__init__.py", line 48, in <module>
    from pynifly import *
ModuleNotFoundError: No module named 'pynifly'

 

14 minutes ago, Bad Dog said:

Okay, fixed and tested on my machine. Give it another try.

Latest commit still reads as "3 hours ago" so I assume it's still the same file as before. Tried it and got the same error. Did you forget to upload it or something?

Link to comment

So everybody hold off. The DLL isn't loading on other computers for reasons I don't know yet. Likely there are C++ runtimes, but there may be other things too. So I have to figure out how to either package the thing with no dependencies or at least figure out what they are.

Link to comment

Just saw the OP and I just wanted to butt in because I'm a bit confused. I've been using the latest version of nif tools for import/export in Blender 2.9x for a while now without issue. Is your goal with making your own plugin to improve functionality with mesh weight morphs and tris like mentioned upthread (Something I would appreciate since those things are a bit of a pain to work with currently)? Or for integrating HKX files into the Blender workflow (Something I've not even tried to do yet because ughh..)?

Link to comment

No, it's really about FO4. But since the nifly code supports all the games, I'm gonna do that too. And then I can extend the functionality to do things like tris and so forth. I don't know anything about HKX files.

 

I'm glad to hear the niftools export/import is working. They were dead for so long, and then the last time I tried to use them I tried a simple round trip of an armor and it didn't work, so I gave up. That was last summer, I think.

Link to comment

Ah ok. I didn't realise nif tools doesn't currently support FO4. I'd love to be able to help, but my programming knowledge is.. eh.. rudimentary to say the least. You don't suppose exporting as another game's nif format and then optimising to FO4 using an external program like nif optimiser or CAO would work?

Link to comment
10 minutes ago, Frosferes said:

You don't suppose exporting as another game's nif format and then optimising to FO4 using an external program like nif optimiser or CAO would work?

Not a thing, FO4 nifs are completely different from LE/SE ones.

 

Incidentally, LE and SE ones are very close which is why you can just run them through CAO. Or even use LE ones in SE, though then you run the risk of the nif having a single byte of data that bitch-ass pussy SE doesn't like and throws a hissy fit and a CTD over, unlike LE which loads it like a champ.

Link to comment
7 minutes ago, Blaze69 said:

Incidentally, LE and SE ones are very close which is why you can just run them through CAO.

 

Got it. I get HKX not being supported by Blender *glares muderously at Microsoft* but in a way I'm surprised that progress on Blender's support for file formats like nif has been so glacial, given its massive explosion in popularity and industry adoption in recent years.

 

As an aside, SE's crash happy tendencies are really annoying. I've been trying to run one of MMG's HCOS title screens and some of them will just crash SE no matter what I do to them

Link to comment

With Blaze's testing, we got the kit fixed. It's now up at the github, ready to use on an experimental basis.

 

I've been using it to make headgear for FFO and the shape key export thing is pretty sweet. I do a shape key for cats, dogs, and ungulates and then one export creates all the files.

Link to comment

OK, finally got a chance to play with this thing. Impressive work. This is going to save loads of time in messing around with obj exports and the like.

 

One question though: are the bones supposed to be linked? I thought I'd try a few poses, but rotating (say) SPINE1 doesn't move any of the other bones. Normally, I'd expect moving that one to move pretty much the whole upper body.

 

I might be able to work around it by loading a skeleton.nif file by way of fbx, and then parenting your weighted mesh onto the new skeleton.

Link to comment

In case anyone is wondering, I haven't forgotten that I'm supposed to be modding FO4. I zipped through the gas masks using my fancy-dancy super-cool import/export tool. I did 3 variants of each--cat, dog, and ungulate--and did them as shape keys so one export produced all three nifs. Copy in the shader and they were done.

 

Then I decided to see if I could do a better baby Shawn (I know, not the most important thing, shut up) and went down a rabbit hole. Baby Shawn has the most ridiculous skeleton ever and it has the whole thing rigged up with parent/child relationships. Anyway the thing wasn't importing properly so I spent a bunch of time doing tests at various levels to see where the problem was. Ended up being a one-line fix to not initialize skinning twice. But hey, got lots more checks, cleaned up the code and had fun doing it. For a given value of fun.

 

I suppose now I really ought to furrify him but I'mma quit while I'm ahead, for tonight anyway.

 

Valehyena has a great model for the super mutants, pumped to start getting that in game. It's going to be a MAJOR hassle because they've got a million ways they can be blown apart and they all have separate meshes and skeletons and such. I dunno even how to test that... spawn a mutant and hit it with a pocket nuke and see if it flies apart? 

 

So there's that, there's more headgear, and I really want to do a second pass on the heads. I can now get them down at the origin and get rid of the neck seam.

 

image.png

Link to comment
On 4/29/2021 at 12:40 PM, DocClox said:

OK, finally got a chance to play with this thing. Impressive work. This is going to save loads of time in messing around with obj exports and the like.

 

One question though: are the bones supposed to be linked? I thought I'd try a few poses, but rotating (say) SPINE1 doesn't move any of the other bones. Normally, I'd expect moving that one to move pretty much the whole upper body.

 

I might be able to work around it by loading a skeleton.nif file by way of fbx, and then parenting your weighted mesh onto the new skeleton.

 

No, they aren't yet supposed to be linked. (The niftools export/import doesn't connect them up either, if they aren't connected in the file.)

 

I spent some time today on that. It's not hard to do but surprisingly tricky to do right. A lot of the FO4 meshes don't use connected bones--clothing might weight to the skin bones but not to the core bones, so you can't connect up a useful skeleton properly unless you bring in the core bones from the skeleton. I can do that, since I inherited the reference skeleton concept from OS, and since I already only export bones that are actually used, the extra bones aren't a problem. But then is the reference skeleton vanilla or XP***? If the first, a fair number of modder's meshes might be left out. If the latter, OMG we're talking a billion jillion bones, maybe overwhelming for most purposes. 

 

Then on export should they be connected or not? Most meshes don't connect them. I don't know if there are advantages one way or the other.

 

And it complicates computing the transforms for the export. I think it's just a matter of looking at the global position of the bone, not local, but I didn't try that today.

 

You can connect the bones yourself in Blender. (Which might screw up the export unless you unconnect them.) Or do what I do and just select multiple bones and rotate around the cursor. 

 

Or, now that I know someone cares, I can take another look at it.

Link to comment

First off, thank you for looking at this. I know it's way off from your use case for the tool.

 

2 hours ago, Bad Dog said:

Then on export should they be connected or not? Most meshes don't connect them. I don't know if there are advantages one way or the other.

 

Well, if it helps, this is the workflow I was hoping for:

  1. Import linked skeleton from nif (skeleton.nif will do)
  2. Import weighted body mesh and parent to skeleton (either as part of 1 or separately)
  3. pose and/or animate skeleton
  4. Export to fbx
  5. try and use DexesHTTP's tool to turn the fbx into hkx
  6. Failing that, import the fbx into Max and try that wat

So, for my purposes at least, you don't need to worry about export.

 

3 hours ago, Bad Dog said:

You can connect the bones yourself in Blender

 

I thought about that. Was discouraged by the fact that there were dozens of the damn things, occupying the same 3d space in a great many cases, and with it not being immediately obvious what's supposed to connect to what. I can do "the foot bone's connected to the ankle bone", but anus bones and thigh skin bones are not so straight forward.

 

That said, there's no reason I shouldn't do it. I can use Max as a reference and save the blend after, so it's a one time job. But if you sort it, that would be fantastic.

Link to comment
6 hours ago, Bad Dog said:

Valehyena has a great model for the super mutants, pumped to start getting that in game. It's going to be a MAJOR hassle because they've got a million ways they can be blown apart and they all have separate meshes and skeletons and such. I dunno even how to test that... spawn a mutant and hit it with a pocket nuke and see if it flies apart?

Honestly I think the best course of action here would be just doing the heads and keeping the bodies vanilla/vanilla-fit, with maybe some edits to the neck/head area of the body so the new head itself fits properly at most.

 

I've seen the model and TBH I don't see anything there that standard SM bodies don't have and/or that can't simply be replicated with a retexture, so this seems like a reasonable idea IMO.

Link to comment
4 hours ago, Blaze69 said:

Honestly I think the best course of action here would be just doing the heads and keeping the bodies vanilla/vanilla-fit

 

Nah. These mutants are gonna be hawt.

 

7 hours ago, DocClox said:

I know it's way off from your use case for the tool.

 

This is good. I want to support what people want to do. I just need guidance.

 

7 hours ago, DocClox said:
  • Export to fbx
  • try and use DexesHTTP's tool to turn the fbx into hkx

 

Wait, what? You can use this fbx thingie to export animations? You suddenly have my attention. There are various things I've been wanting to do with animations (wings, sheath). They just hasn't popped to the top of the list yet.

 

I'll take a second look at importing skeletons.

Link to comment

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. For more information, see our Privacy Policy & Terms of Use