Blaze69 Posted May 28, 2019 Posted May 28, 2019 So, I've been tinkering with FO4 a bit, and more specifically character meshes and textures, and while I think I've gotten the hang of it despite the weird setup (meshes with bone transforms that "lift" them from their "base" position below ground in the nif, normals with inverted blue channels, specs that use the red and green channels for different stuff but leave the blue one unused, etc.), I've hit a snag when it comes to working on normal map seams across meshes, i.e. at the neck transition between body and head. So far I've only worked on character assets for Skyrim, which uses Object Space normals instead of the Tangent Space ones used by FO4, but those were pretty simple to bake. All I needed to do was to place the high-poly mesh in the right spot and select which low-poly/ingame meshes to bake to, and the normal maps would come out flawless and 100% seamless if everything was properly set up. Problem is, I can't seem to get that to happen with FO4 at all no matter what I try. AFAIK Tangent space normals take the vertex normals from the mesh itself into account (unlike Skyrim's, which didn't, and which even required removing tangent normal data from the nif to prevent tangent-caused seams), so the normal maps won't be "visually" seamless like diffuse textures or Object Space normals would. But no matter what settings I use when baking, they are not seamless ingame either once properly applied as normal maps. I don't think it's a mesh issue, because the vanilla head vertex normals don't match those of the body when loaded without textures either but become pretty much seamless when checked in NifSkope or ingame with the textures applied; that would mean the normals can indeed be baked in a way to make the transition seamless even if the vertex normals themselves are not, but I can't find the way to do it with my custom textures and meshes. Any ideas on how to go around it? I have the high-poly mesh done and ready, and it worked fine when baking object space normals; it's the FO4 tangent ones that are giving me trouble. I can provide pics and/or a more detailed explanation of the process I've used so far if it helps.
Andy14 Posted June 6, 2019 Posted June 6, 2019 Try xNormals with your both meshes (high and low). Adjust the margin settings for seamless normals.
Blaze69 Posted June 6, 2019 Author Posted June 6, 2019 31 minutes ago, Andy14 said: Try xNormals with your both meshes (high and low). Adjust the margin settings for seamless normals. Tried that, didn't make a difference. It would still have a seam ingame and in NifSkope/Outfit Studio. In the end I opted for something else: I loaded the head mesh into Blender, edited the vertex normals at the seam to match those of the CBBE body neck (using the TiNA addon it's only a matter of seconds) and then used the GIMP normalmap plugin to create a flat and seamless normal using the diffuse textures as a base (which are 100% seamless, and thus so should be the generated normal). The blended vertex normal coupled with the seamless normal texture seemed to do the trick, as the neck transition is now finally seamless. Probably not the best solution, but it's the only one that worked for me. I'd still appreciate it if anyone could port me to the "proper" way to bake normals, though, as this is just a "dirty" workaround and it's not how tangent normals are supposed to be made in order to be seamless AFAIK. Thanks for the suggestion anyway.
SQr17 Posted June 10, 2019 Posted June 10, 2019 TiNA for Blender is an excellent find, works really well. I have a question for you if you don't mind; after using TiNA in Blender, how did you get the updated (head) model into the game files without messing up the normals again? Outfit studio seems to regen normals by default... Thanks.
Blaze69 Posted June 10, 2019 Author Posted June 10, 2019 4 hours ago, SQr17 said: I have a question for you if you don't mind; after using TiNA in Blender, how did you get the updated (head) model into the game files without messing up the normals again? Outfit studio seems to regen normals by default... There's an option in the "Shape" menu from the top bar in OS that reads "Smooth Seam Normals". Uncheck that one and check the "Lock Normal" ones. That seems to do the trick, but unfortunately it doesn't seem to work for OBJ import into existing nifs (i.e. by doing that you force it to keep the normals in the original nif, and the ones from the imported OBJ are ignored). So I had to go for a workaround: Import the mesh with the edited normals into OS as an OBJ. With the just-imported shape selected, make the changes to the normal options in the Shape menu. Import the same OBJ again, and say "Yes" to updating vertex position and UVs. The vertex normals should actually update to the fixed ones and remain that way. Load the original non-edited head nif as a reference and transfer weights from the ref to the fixed-normals mesh (should work fine as they are supposed to be the exact same mesh). Save the fixed-normals and newly-rigged mesh as a nif. Make sure to make any changes to the nif data you need (like shader properties or material file). Again, there may be a better and/or cleaner way to do this, but that's the one that has worked for me so far.
SQr17 Posted June 10, 2019 Posted June 10, 2019 Ok, I followed your instructions and did it....WOW, I think you have finally solved it, the seams are almost completely gone! (i just used flat #808000 color at the seams on the normal maps to test). It's not all great though, Outfit Studio is pretty wonky with normals and it also somehow manages to reorder vertices when using OBJs, so weight transfer goes to hell, (yes, Keep vertex order is selected in Blender for Import and export). I will try to do this using FBX and if I am successful, it's going into the next version of HiPoly Faces. We can now fix the wrist seams too...damn. Fine work, Blaze69. EDIT: No dice. Doesn't work with FBX and OBJ vertex order is imported incorrectly, don't even need to go through Blender for this issue. Were you able to import the head mesh obj correctly in OS, weights and everything?
Blaze69 Posted June 10, 2019 Author Posted June 10, 2019 2 hours ago, SQr17 said: EDIT: No dice. Doesn't work with FBX and OBJ vertex order is imported incorrectly, don't even need to go through Blender for this issue. Were you able to import the head mesh obj correctly in OS, weights and everything? Yeah, I was. I read somewhere that OS wants the vertices sorted in a particular way (can't remember which, exactly) so if your mesh has the "wrong" vertex order, it won't be recognized as the same mesh as the pre-export one no matter what you do because OS will always reorder them to the way it likes when importing in the first place. In my case, the mesh I was working on already has the OS-friendly order so it wasn't a problem, but as you said it's not the case with all meshes out there. If it's just the bone weights you want, you can use OS' standard weight copy tool and set it to 1 target per vertex and a small search distance; as long as the mesh shape and vertex position is the same (regardless of order), those settings should give you a 1:1 copy of the original weights. Only problem is that may mess up the TRIs, which would need to be edited as well somehow to have the exact same vertex order as the OS-exported-and-reimported OBJ. EDIT: check PaulGreen's edited TRI export plugin in this post, it may be just what you need to reorder the TRIs so they match the head.
SQr17 Posted June 10, 2019 Posted June 10, 2019 Doing that is my last resort. It will probably work but it's messy as hell. Still testing a few things, let's see how it goes.... Any chance you are going to release the seamless textures as a mod? If not, would you be willing to let me take a look at them (privately, if you prefer)? Totally fine, if you don't want to. BTW, I fixed the CBBE wrist seams.
Blaze69 Posted June 10, 2019 Author Posted June 10, 2019 15 minutes ago, SQr17 said: Any chance you are going to release the seamless textures as a mod? If not, would you be willing to let me take a look at them (privately, if you prefer)? Totally fine, if you don't want to. I wouldn't be against doing so, but the thing is, they are not for humans, and the custom race they are made for uses a custom head completely different from the vanilla FO4 human heads (it's all in one piece instead of the back being separate, it uses a completely different UV layout, it has a different shape, etc.) so they would be useless for anything human-related. If you are wondering how I created them, I simply took some seamless diffuse maps, ran them through the GIMP Normalmap plugin (I think it was this one) and then inverted the blue channel to get them to match FO4's default tangent space normal format. As for the seamless diffuses themselves, I simply loaded both body and head into Mudbox alongside their original textures and used the "Clone" tool to paint/blend the textures across the neck seam.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.