Jump to content
  • entries
    12
  • comments
    636
  • views
    10,898

The Effin Export/Import Project


Bad Dog

9,057 views

Edit: For the record, this tool is implemented and has its own thread here: 

 

I can't really do what I want to do with FO4 if I can't do proper weight painting and I don't have full control over the nifs I create. So I've decided to have a look at making proper import/export tools between Blender and FO4. I'm using this blog to track thoughts and progress notes along the way.

 

There are two possible routes:

  1. Extend the niftools package. That is currently being actively worked on, tho there was a long hiatus for the past few years, which meant I was stuck with old versions of Blender and a multi-step conversion process. The recent versions support the latest Blender and maybe have better export--I haven't yet seen a version which exports properly but I haven't tested the latest. Upside of this method is it ties into an existing project and extends the tools that have been around forever. Downside is this stack is very complicated--translation generated from XML files, read/write through pyffi, and so forth. I'm told the XML for FO4 has been written but pyffi needs updating to use it. Presumably higher levels of the stack would need work too? Dunno. 
  2. Bodyslide/OS has export/import to all games and they stay up to date all the time. Their nif code is cleanly separated into a package called Nifly, which is used by BS/OS, Cathedral tools, and Nif Optimizer. So it seems like robust code with an active community behind it. Downside is I'd have to access it from a Blender python addon, and I have to figure out how to walk the Blender representation of a model so as to push it at Nifly.

 

I've looked a little bit at both options. My feeling right now is that the hard part of the niftools route is digging into XML, generated code, and pyffi. The hard part of the Nifly route is figuring out how to bolt it into Blender and parse the Blender data model. That seems more straightforward. It's all known stuff--Blender addons and such. So until I hit a roadblock, I'm going to explore that.

 

The biggest issue is whether I can call the C++ Nifly code from a Blender addon. I hunted through other addons and found an import/export addon that does exactly that. So in theory I just copy their method.

 

So here's the project:

  1. Figure out how to make a Blender export/import addon
  2. Figure out how to make a DLL
  3. Make a dummy DLL and call it from a dummy export/import addon. Once that's done I know I can get data into and out of Blender.
  4. Make a DLL encapsulating Nifly
  5. Call that DLL from my Blender addon. 
  6. Keep adding features until done.

 

#1 is mostly done--I have a little script that pretends to be an export/import addon but does nothing. On to #2.

Edited by Bad Dog

175 Comments


Recommended Comments



New version up! X0.0.16 has tri import/export. Also better messaging and error handling.

 

Edit: And I see the problem with the BT body, tho not how to fix it. It is the calculation of the global-to-skin transform. They're a little different in the two files. I'm guessing it's rounding errors adding up, maybe because of how the bones were positioned? And it's fine with the game's calculation? Still looking.

Link to comment

there is a possibility to import in fallout 4 the raiders outfit of fallout3/FNV by using your tool ?

Link to comment
1 hour ago, Delzaron said:

there is a possibility to import in fallout 4 the raiders outfit of fallout3/FNV by using your tool ?

I guess you could import the FO3/FNV nifs, pose them to match FO4's base pose, and then either export as OBJ to Outfit Studio for rigging, or rig it in Blender itself and export directly into a FO4 nif. That's how you usually port stuff from other games, it just so happens that "other game" would be FO3/FNV this time around.

 

Do note that porting vanilla FO3/FNV assets to other Beth games is very uncool and very illegal according to Bethesda so you may want to keep that for yourself or at the very least post it somewhere out of reach of Beth's Tactical Lawyer Team. Also FO3/FNV quality is usually too low to work out well in FO4, but you do you I guess.

Link to comment
55 minutes ago, Blaze69 said:

porting vanilla FO3/FNV assets to other Beth games is very uncool and very illegal according to Bethesda

Oh, goddamn. Forgot about that part.

 

But yeah, the way to go would be to rig it in blender by copying the weights from an FO4 body and export direct from there.

 

There was a missing file in that last upload, fixed now. But if anyone got it right off the bat, you should download again.

Link to comment
55 minutes ago, Blaze69 said:

I guess you could import the FO3/FNV nifs, pose them to match FO4's base pose, and then either export as OBJ to Outfit Studio for rigging, or rig it in Blender itself and export directly into a FO4 nif. That's how you usually port stuff from other games, it just so happens that "other game" would be FO3/FNV this time around.

 

Do note that porting vanilla FO3/FNV assets to other Beth games is very uncool and very illegal according to Bethesda so you may want to keep that for yourself or at the very least post it somewhere out of reach of Beth's Tactical Lawyer Team. Also FO3/FNV quality is usually too low to work out well in FO4, but you do you I guess.

 

too bad, I know nothing in 3d creation...

But the fallout 4 vanilla raiders outfits sucks so much... (for me, FNV is better than fallout 4)...

Link to comment
25 minutes ago, Delzaron said:

too bad, I know nothing in 3d creation...

It's as good a time as any to learn, then. ?

 

Quote

But the fallout 4 vanilla raiders outfits sucks so much... (for me, FNV is better than fallout 4)...

Don't think anyone has done the FNV raider outfits (yet?), but the Fallout 4 Capital Wasteland team did recreate the FO3 raider outfits in FO4, and since it was built from scratch without using real FO3 assets, it's Nexus-legal:

https://www.nexusmods.com/fallout4/mods/50007

 

26 minutes ago, Bad Dog said:

Oh, goddamn. Forgot about that part.

Not that cross-game porting being (technically) illegal has stopped me from amassing an ungodly amount of ports myself (including a few FNV outfits in Skyrim), but those can't be released publicly* lest one risks getting in trouble.

 

*-> as I said, nowhere that respects copyright at least. There are places/Discord servers made specifically to share this kind of ported stuff, and then there's also entire RL countries that laugh at the silly concept of "intellectual property" as do any sites hosted in them (cough China cough Russia).

Link to comment

If ownership of both games is a condition for downloading ported content, that should be fine. I think

 

Or am I wrong?

Link to comment

You'd be right if it was solely a question of copyright. The problem is that the EULA for the games and the creation kit specifically prohibits porting assets from one Bethesda game to another..

Link to comment

New version up. X0.0.17 will parent a new mesh to an existing armature if you select it before importing. Any bones in the nif are added to the armature, so you can load one body part after another and end up with a complete skeleton.

 

It took literally two lines of code. I love when that happens--indicates the code is well-structured.

Link to comment
On 5/5/2021 at 11:07 PM, Bad Dog said:

It took literally two lines of code. I love when that happens--indicates the code is well-structured.

 

Yeah, that's what I always say :D

 

No joy with that animation pathway, alas. The FBX import can work, and I've made it work in the past, but damned if I could make it happen this time. The process is widely acknowledged to be finicketty as hell. It's possible that by messing around with scales and units I could make it work. I may well have another attempt before long.

 

What did work was importing the body, and making the pose and the furniture in Blender.

 

image.png

 

And then copying the furniture part to Max and using the F4AK rig to pose the body there to fit the furniture. Then I exported the pose using Havok Content Tools.

 

image.png

 

image.png

 

So not the outcome we were hoping for, but still a lot easier than my previous workflow which involved a lot more copy and paste between Max and Blender and which still managed to be a bit hit-and-miss at the end of the day.

 

I haven't had a chance to text X0.17 yet, what with everything else. I might give that a go now.

 

Link to comment

Can you check that the download for x0.17 is in fact 0.17? It installs OK, but then continues to report itself as 0.14, and it doesn't do the merge skeletons as expected.

 

This is X0.0.17 installed without error:

 

image.png

 

Reported version after install. I quit Blender and restarted just to be sure everything was refreshed

 

image.png

 

This is my test blend - I deleted the body mesh on the most recent version of the furniture item I posed earlier, and then like an idiot, I saved it. This seems like a good test since I should just be able to re-import the body.

 

image.png

 

But this is what I get.

 

image.png

 

 

The little black smudge by the 3D cursor is the actual model and original skeleton:

 

image.png

 

The scale thing is my doing - I scaled the body down because the original import was too big by a factor of 10 when I imported it into OS and nifscope. (Might not have needed to do that if I just exported using nifly - didn't think of that at the time). Anyway, new body is importing at that same size, which could be messing up the skeleton merge.

 

Is it worth adding a scaling factor to the import dialogue, I wonder?

 

 

Link to comment

One more thing: I thought, by way of an expriment, I'd try deleting the armature from that imported mesh, scaling the body, and then parenting it to the posed skeleton. In theory, that ought to move the mesh to the bones and recreate the pose, right?

 

In practice ....

 

image.png

 

Probably I'm doing something wrong, but if nothing else, it's another data point.

 

Interesting fact: a lot of my failed imports into max ended up looking a lot like that. 

Link to comment

One more post, and then I'll stop hogging the thread :)

 

On the subject of animation, over on Nexus forums user ajs52698 has just deposited a huge great info dump on Havok, hkx format and all manner of related subjects. It leads me to believe that we could, potentially, generate HKX XML from Blender and then pack them up using HKX Pack (which does actually exist).

 

I am sorely tempted to take a crack at it.

Link to comment

Be tempted.

 

Yes, I don't really know whatup with the size of blender nifs, but they're always 10x what the other tools expect. I don't know what's behind it. I'm not doing any scaling, just taking the verts as they come and the coordinates are literally what's in the nif. I don't know why if you export to obj (which keeps the coordinates) and then import to OS, it's 10x too large, especially compared to a nif you imported from OS. And I'm using the OS code.

 

I can only guess that for compatibility with other games and obj files OS is scaling nifs down by a factor of 10 on import and re-scaling on output. I could do that but I don't love the idea. There's a loss of precision and just more layers of cruft between you and the model. 

 

I could do an option but I don't love having a million options. I probably will do it before I'm done but I'd like the default to be the most straightforward: in from the game, modify, out back to the game, with no fiddling.

 

Also of course you can scale once you're in blender and that works fine so why not leave it at that?

 

Fabulous that you can get your stuff out to the game. For the armature, you do have it as the active object when you do the import? For blender there can be lots of selected objects but only one active object. It's usually whatever you clicked on last.

 

Parenting to your armature likely didn't work because it's in a pose, so when you parented the mesh picked up the current location of the bones, then said, oops, I'm in a pose and applied the pose a second time. Put the armature in "rest" position (an option on the armature properties page) and do your parenting, then when you return to the pose the body should follow.

 

Whether or no, I exported my FO4 model to Skyrim once to often so I put in a check to set the default based on the armature, if there is one. It's got a few other minor enhancements too. So I'll put that up today.

 

If your system is reporting x0.0.14, that's what you have. Dunno why it didn't install after saying it did. Did you give the file the name PyNifly-X.0.0.17.zip on download? Cuz I didn't. (I have been thinking I should put the version in the name somehow.)

Link to comment

(BTW - I've been hassling with Blender's logging, and I see you're getting the full debug log. Did you turn that on explicitly?)

Link to comment
2 hours ago, Bad Dog said:

Also of course you can scale once you're in blender and that works fine so why not leave it at that?

 

I'm just thinking about the case where you might have a model but need to add a few more bones from an up-to-date skeleton. Do you end up with head bones 10 times the size of the body bones, for instance.

 

In practice, it's rarely going to be an issue, so I'm fine either way.

 

2 hours ago, Bad Dog said:

Parenting to your armature likely didn't work because it's in a pose, so when you parented the mesh picked up the current location of the bones, then said, oops, I'm in a pose and applied the pose a second time. Put the armature in "rest" position (an option on the armature properties page) and do your parenting, then when you return to the pose the body should follow.

 

Sounds good, I'll try that.

 

2 hours ago, Bad Dog said:

If your system is reporting x0.0.14, that's what you have. Dunno why it didn't install after saying it did.

 


 

Yeah. After previous attempts, I was careful not to try and uninstall and made sure I didn't do a damn thing before updating the plugin.

 

 

 

Quote

Did you give the file the name PyNifly-X.0.0.17.zip on download? Cuz I didn't. (I have been thinking I should put the version in the name somehow.)

 

That's just how GitHub served the file.

 

image.png

 

I am getting it from the right page/link, I hope?

 

I'll see if I can get the proper version to install.

 

1 hour ago, Bad Dog said:

(BTW - I've been hassling with Blender's logging, and I see you're getting the full debug log. Did you turn that on explicitly?)

 

Nope. Not unless it's a blender global, in winch case I could have enabled it while messing around with scripts a year or two ago. I can't remember doing it though.

 

[edit]

 

I just cracked open that 0.0.17 zip and looked at the contents

 

bl_info = {
    "name": "NIF format",
    "description": "Nifly Import/Export for Skyrim, Skyrim SE, and Fallout 4 NIF files (*.nif)",
    "author": "Bad Dog",
    "blender": (2, 92, 0),
    "version": (0, 0, 17), 
    "location": "File > Import-Export",
    "warning": "WIP",
    "support": "COMMUNITY",
    "category": "Import-Export"
}

 

So that certainly looks like 0.0.17. And it looks like it installed. Right alongside is_scene_nifly. But there's no PyNifly showing up in the extensions menu, just the old 0.0.14 version.

 

image.png

Link to comment

I just compated against 0.0.16 - there's only two downloads for 17, the source code ones. The 0.0.16 release has an io_scene_nifly.zip asset as well, which is what I installed last time.

 

Would that make a difference?

Link to comment

The github stuff is bizarro-land. I don't know what it's giving you when it gives you pynifly-x0.0.17, tho I think it must be the right thing? When I've downloaded from there I somehow got the io-scene zip instead. 

 

Maybe I'll have the plugin announce its version number in the log just for clarity.

 

Link to comment

It's just 0.0.17 - there's a missing download. It has the two "source code" zips, but not the final package.  You can see it in the screenshot I posted earlier.

 

UNless you see three downloads when you look. That would be weird.

 

[edit]

 

OK. I unpacked X0.0.16, copied over the common files from PyNifly x0.0.17 and then zipped it back up.  That installs and works as expected. Haven't tried the tri file import yet.

Link to comment

Okay @DocClox, x0.0.20 up with some convenience features.

 

Looks like I forgot to attach the zip to 0.16 and so you just got the source code, where things aren't at the right level. Pull io_scene_nifly.zip from this one and it should install right.

Link to comment

Yeah, that installs just fine.

 

Probably unrelated, but my model has developed an infuriating glitch where some parts seem to have a backed in transformation that reapplies itself whenever there's a file operation. The collar part, for instance, really wants to shift itself about 5 units up and 7 degrees off true, and it reverts to this position and orientation every I load or export. Worse, it reverts position before it exports so I can't get a good nif from it.

 

I don't suppose it rings any bells? It's probably just Blender being Blender, but I'm grasping at straws at the moment.

Link to comment

You've checked the shape transform on the object properties? Should not do the glitchy thing but...

 

If all else fails export to obj and re-import?

Link to comment
44 minutes ago, Bad Dog said:

You've checked the shape transform on the object properties? Should not do the glitchy thing but...

 

Yeah, I had some transforms get stuck. Didn't matter how often I cleared or applied them, they kept coming back.

 

I gave up in the end and remade the offending items.

 

Want to see something odd I noticed? Here's my model in Blender  (third iteration)

 

image.png

 

And here it is exported to obj and imported into OS with the rotation corrected:

 

image.png

 

And this is it exported to nif with nifly and imported into OS

 

image.png

 

So it didn't need its rotation corrected, but the z axis is madly distorted for the vertical pipework. And some things, like the nuts and bolts I added, are not rotated, and are displaced from where they should be on the model. Here's a close up.

 

 

image.png

 

Not a big deal for my workflow since I'd want to export the furnoture to obj anyway. But I thought you might want to know.

 

 

Thinking about animation: I think that between that HKX breakdown and Anton's animation tools, I probably have every thing I need. Anton's scripts talk through the animation frames of a Blender file and turn that data into animations which get inserted into a nif file. So  the same basic logic should work to generate HKX XML. Anton's scripts are Skyrim LE only, but someone on the support thread posted the fixes needed to work with the latest Blender object model.

 

I'm going unpack some simple animations, and then see if I can recreate them.

Link to comment

If you want to send me that blender file, I'll have a look. I'm collecting a rogue's gallery of nifs that cause problems. Dunno how the z axis could be deformed without anything else.

Link to comment

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. For more information, see our Privacy Policy & Terms of Use