Potential Fix for NIFs with extra data outside of Scene Root

[varenne]

I've been experimenting with finding a simply fix to address NIFs that have extraneous nodes and data outside of the Scene Root. These have the potential to cause issues in-game and are also problematic when you try to optimize the NIF using Nifskope or PyFFI. (Besides this one from Ozmo, I have also discovered this flaw in some of Colourwheel's NIFs, and all 4 Cup versions; B, D, H and Manga of Forbidden Armor-39626-2-0.)

 

For this example I used Ozmo's BBB For Ivy Armor 2_0-33352.7z and the LArmorDcup_Black.nif.

 

 

 

 

As you can see in the above NIF, it includes data or nodes outside of the Scene Root. This defect or flaw occurs in all of the D-cup NIFs. I find that typically this occurs in armors and clothes as a result of MOD author removing a block, but not the entire block, like when you have to remove the foot and replace it with a shoe. More than likely they used Block | Remove instead of Block | Remove Branch, which removes everything associated or attached to that specific node and branch.

 

 

 

 

 

The first step is to remove those blocks or nodes. The easiest and quickest way is to click on Scene Root so it is highlighted and use Block | Crop to Branch. (I also ran some other spells, but that may not be needed at this point. Yes, using Spells moved to end of processing.) DO NOT run the spell Make All Skin Partitions. I'll show why next.

 

 

 

 

 

If you run the spell Batch | Make All Skin Partions the NIF model will actually vaporize! ***POOF***

 

Fix with PyFFI

What I did was after removing the extra data was to run it through PyFFI, and then run the NifSkope spells. This seems to have worked to correct the NIF as Make All Skin Partitions will now work in NifSkope on this NIF. Testing in-game so far I have found no issues.

IvyArmorDcupPyFFI_Fix_file_examples.7z

 

 

Why do this?

This is in preparation for using gerra6's Mesh Rigger to add the lowerbody OP3 bones, which these models do not have yet, or have them and do not animate in-game (No idea why not). I'm slowly going through those armor & cloth models I do have and use in-game, and adding this where I feel it adds more variety to my game. That's it.

 

[Vaelorian]

You could probably mention that one can use the crop to branch function on the scene root to eliminate everything outside of it in one go. Just for the sake of completion.

 

But yes, this'll be very much required once we get some animations that utilize these bones, and is a good idea to write in advance. Have a +1.

 

-V

[gregathit]

Like Vaelorian, I just crop to screen root and wipe all that crap out.  Never had an issue doing that.

[varenne]

Like Vaelorian, I just crop to screen root and wipe all that crap out.  Never had an issue doing that.

 

Yeah G', tried that, doesn't work 100% of the time, IF you run Make All Skin Partitions in the same session. (I will try again, as it may just be with certain really f'edup NIFs I'm testing and developing my approach with.) There are also added benefits for running PyFFI on a NIF at this point too. I'm always pursuing optimization when and where I can. It is the only way to gradually increase quality levels too, IMO.

 

VERIFIED!  - Using Crop To Branch on Scene Root on my included test NIF immediately followed by Make All Skin Partitions produces the same result; model vaporization.

 

I feel something is happening with the NIF when you cut or delete those extraneous nodes, that NifSkope just cannot deal with, but PyFFI can.

 

My desired goal here though is to see if this can be at some future point added to or merged into gerra6's stand-alone tools, which I like a lot. Having to use NifSkope (spells), AND PyFFI and then finally be able to use the g'6 stand-alone tools, is not what I call a highly productive process. One click or one dble click is easy. Blender is also not easy for many. If we make it simple and easy we can promote crowd sourcing as other less technical members can make use of it.

[varenne]

You could probably mention that one can use the crop to branch function on the scene root to eliminate everything outside of it in one go. Just for the sake of completion.

 

But yes, this'll be very much required once we get some animations that utilize these bones, and is a good idea to write in advance. Have a +1.

 

-V

 

Thanks.

 

I'm actually using it now to add OP3 bones, via Mesh Rigger that are already utilized in many walk, cast and fight animations, not so much for the Lovers animations as they honestly look like crap. When and IF the Buttock Bones become added to future animations, I'm already setup to run Mesh Rigger en masse on a group of armors/clothes. Once I get all these flawed NIFs straightened out.

 

V'

 

[gerra6]

It should be possible to streamline all of those steps into a single tool.

 

Unfortunately, I'm sort of at the poking my code with a stick stage of getting back to working on it, so no promises.

 

The problem that I'm having right now, unrelated specifically to this, is that I'm in the very early stages of refactoring my tool framework to something a bit more streamlined and hopefully less buggy and messy.

[varenne]

Here are my process steps, for when you are ready.

 

1. Open and review NIF using NifSkope for stuff outside of Scene Root.

2. If found click on Scene Root so it is highlighted and use Block | Crop to Branch.

3. Run PyFFI on mesh or proferably a group of them in a folder.

4. Open again in Nifskope and use the following spells

a. Combine Properties

b. Remove Bogus Nodes

c. Update All Tangent Spaces

d. Make All Skin Partitions (with Stripify Triangles enabled or ticked)

e. Update All MOPP Code

f.  Sort By Name

g. Reorder Blocks

h. Reorder Link Arrays

 

The last three I use to make it my standard human-readable format. With this I can quickly review and find elements within the NIF for future edits and rework.

 

At this point I consider it ready to use the stand-alone tools; Mesh Rigger, etc.

[gerra6]

A note on #c, unless there has been a new release in the last few months, the Nifskope Pyffi update all tangent spaces spell is broken.

 

Oops:  That'll teach me to write stuff late at night.  Sorry about that, my brain misprocessed Nifskope as Pyffi.  Sigh.

 

Basically, when tangent spaces are calculated, a combination of the face normals and orientation of the 2d UV space is used to calculate the sign.  *Unfortunately* the pyffi update tangent spaces function (called by the spell) merges vertices on either side of a UV seam into a single hybrid vertex.

Now, if you are calculating the vertex normal, this is a very good thing. Since the vertex normal is a composite vector of the face normals for each adjacent face, treating seam vertices as a single unit allows you to calculate smooth seams.

On the other hand, when it comes to the tangent space calculation, things turn into a clusterfuck. Since the UV orientation on either side of a seam can be wildly different in completely unpredictable ways, the calculated sign factor of each component of the tangent space suddenly becomes completely random relative to the components on the other side of the seam.

This bug is what causes the shiny seam ugliness.

Now, I did update the niftools git with a potential fix awhile backs, but I accidentally updated the master rather than my branch (I suck at git) and made a bunch of work for neomonkeus. Big oops there. Honestly, I'm not sure if my fixes ended up incorporated or not.

[movomo]

By broken, which tool did you mean? A typo of pyffi accidentally typed to nifskope, or you've discovered a new bug in the c++ niflib?

 

If the latter is the case, you definitely should make a stand alone update_tangent_space only tool to renedy all the faulty nifs that were made by nifskope to what they should be. Somebody should.

 

Because, there should be at least one tool that does the thing in the right way...

[varenne]

@ gerra6 - I'll assume that "Nifskope update all tangent spaces spell is broken" is all versions of NifSkope then, and shelve using it till a fix is applied. Think I'll take a look at GIT and see if it was...

 

"This bug is what causes the shiny seam ugliness."

 

I've actually seen this when reviewing meshes in Blender (and wondered about the cause of it), but not in-game.

[gerra6]

My apologies, my brain farted.  Dum Brane.

The Pyffi spell that calls the update tangent spaces function is broken, because the pyffi function was broken as of the last time I downloaded Pyffi (back in March). Note: I discussed the nature of the issue with Ghostwalker and uploaded a fix to the git repo, but I screwed up the git push and am sort of hanging my head in shame until I work up the nerve to go back there. So...it is entirely possible that this has been fixed, if there is a more recent version of Pyffi out there.

Nifskope, although developed by the same group, uses an entirely different set of functions. The Nifskope spell that updates tangent spaces *does* function properly.

So...Nifskope update tangent spaces = Good. My tool framework update tangent spaces = Good. Pyffi update tangent spaces = Bad (shimmering seams).

[varenne]

Okay then. Signals Nifskope Update Tangent Spaces production mice to get back to bloody work!    Such a bunch of slackers that they are!

 

I and maybe a few others have upgraded recently to current PyFFI, needed for movomo's Python project. Either I or anyone in that category could run a quick test to see if it still is a bug or has been fixed. It'll jump right out at you if you load a model into Blender.

[gerra6]

I'll go download it and check the source code.  See if my fixes ever made it in.

 

*Edit*
 

I just took a look at the latest master code in the git repo.  Pyffi still has the bug.

[varenne]

No worries here. I've sort of figured out a decent albeit manual/semi-automated process for now. When ever you feel like getting back into things we can revisit it then I guess.

[varenne]

@ movomo - Seen your recent post on Niftools Forum, Batch update tangent spaces help!

 

So do you know if this this Clean Nif.bat is the same one in ~Tools_beta_0_81?

 

Also read where this fix will be in the next PyFFI release, "soonish".

[movomo]

Also read where this fix will be in the next PyFFI release, "soonish".

In japanese kanji reading way you write it "soon" and read it "forever". :huh:

well, we never know if it's real soon or if it's actually under progress at all. I guess all we can do is to wait. They are all smart people and know what they are doing, after all.

 

um.. what's the matter with clean_nif.bat btw? All that does is to redirect the .py file to stand-alone python package's interpreter. Anyway, I *think* I'm using that same 0.81 version. I don't remember correctly, as I'd deleted that huge portable python package and only left the core scripts and the kg package.