HKXPack/HKXAnim - Animations in FO4

[ShadeAnimator]

 

Aren't the 'face bones' and *_skin bones (found in the skeleton) used for morphing the body/face when you create you character?

 

_skin bones of the body are used for scaling and moving body parts for the weight system they added with FO4.

They should NOT be animated. The parents of those bones are the ones that actually get animated.

 

Face Bones are used only during character creation.

 

 

Ah, cool, thanks. Thats awesome. 

If you have any more information that can help us figure stuff out, please, share

[ShadeAnimator]

Okay, take a look at my last post here, I think I got everything right finally. Just needed to add all the joints and rename some.

[MaikCG]

Is there an analogue hkxcmd for F4? from Skyrim of course it does not work due to the version. Or how I can import hkx xml file derived from HKXPack?

[ShadeAnimator]

Is there an analogue hkxcmd for F4? from Skyrim of course it does not work due to the version. Or how I can import hkx xml file derived from HKXPack?

 

Import where?

 

HKX pack works both ways, it can convert xml to hkx and hkx to xml.

In command-line mode you can specify the operation and input output files.

In my GUI it will automatically detect file's extension, and make the proper conversion

[DexesTTP]

What he said. HKXPack is a HKX binary => XML file and reverse, and actually it is hugely inspired by HKXCmd (I actually made some parts of it in collaboration with Figment)

 

 

In my GUI it will automatically detect file's extension, and make the proper conversion

 

 

That makes me think, for what it's worth you can just do java -jar hkxpack.jar <file> and it'll detect the format and convert it to the other (hkx makes a XML and xml makes a HKX). This is an undocumented feature that I added for the lolz

[ShadeAnimator]

What he said. HKXPack is a HKX binary => XML file and reverse, and actually it is hugely inspired by HKXCmd (I actually made some parts of it in collaboration with Figment)

 

 

In my GUI it will automatically detect file's extension, and make the proper conversion

 

 

That makes me think, for what it's worth you can just do java -jar hkxpack.jar <file> and it'll detect the format and convert it to the other (hkx makes a XML and xml makes a HKX). This is an undocumented feature that I added for the lolz

 

Haha, and I wrote it myself for the GUI Just a few lines.

It is actually quite handy in my opinion. I do tend to automate things as much as possible, and even though some people may say that something it is unnecessary, but in my experience it always pays out, and not even only in the long run. At least mid run

[MaikCG]

Import where?

 

 

What he said. HKXPack is a HKX binary => XML file and reverse, and actually it is hugely inspired by HKXCmd (I actually made some parts of it in collaboration with Figment)

 

hkx/xml need to convert to *.kf or better *.fbx

[DexesTTP]

 

hkx/xml need to convert to *.kf or better *.fbx

 

 

Oh, that.

 

Well, tough luck. This isn't HKXPack's job but HKXAnim's which is exactly what we've been talking about for the last 10 pages or so

 

This doesn't exist yet because we didn't crack the animation format, and it won't exist for a while because I've hit a roadblock last time I worked on it and I don't have enough time to study the new material properly

 

Also, as of now we're intending to only use .dae as intermediary format (although apparenlty FBX exported in ascii would be usable).

[ShadeAnimator]

 

Import where?

 

 

What he said. HKXPack is a HKX binary => XML file and reverse, and actually it is hugely inspired by HKXCmd (I actually made some parts of it in collaboration with Figment)

 

hkx/xml need to convert to *.kf or better *.fbx

 

 

Conversion fbx\hkx and backwards is currently under developent by DexesTTP. Not FBX though, Collada. Which is similar to FBX. And later maybe FBX too.

 

Hm... From what I can see in xml, there are at least two types of HKX files, pure animation ones, and skeleton definition. Animation files only contain information about animation, no object\joint information at all, not even names.

 

skeleton definition hkx file however contains joint names and initial t-pose positions, but not their hierarchy. And it also contains physics related information, from what I can see. Just thinking out loud.

 

Hm, DexesTTP, it's actually interesting information, and I wonder.. wouldn't it be easier, instead of writing a hkx=>fbx converter, to write a max script to parse xml file and create everything that's needed? For example a script to import just animation from xml file on existing skeleton in the scene seems relatively straight forward... Not sure if maxscript has built in xml parser however, uh, I miss Python in max.

 

Hm, actually maxscript can use dotNet, and it has xml parser, and there is a code-friendly wrapper of dotNet on scriptspot.. hah

[MaikCG]

but it is not the same Skyrim? yes, new version, but the same groups of tracks and same spline compression

[Guest]

but it is not the same Skyrim? yes, new version, but the same groups of tracks and same spline compression

 

No.

Skyrim = Havok 2010

FO4 = Havok 2014

[DexesTTP]

 

Hm, DexesTTP, it's actually interesting information, and I wonder.. wouldn't it be easier, instead of writing a hkx=>fbx converter, to write a max script to parse xml file and create everything that's needed? For example a script to import just animation from xml file on existing skeleton in the scene seems relatively straight forward... Not sure if maxscript has built in xml parser however, uh, I miss Python in max.

 

Hm, actually maxscript can use dotNet, and it has xml parser, and there is a code-friendly wrapper of dotNet on scriptspot.. hah

 

 

Yeah, I mean that's basically what I am doing with HKXAnim - just outside of a plugin. The whole idea is to take this file and fill the blanks, then convert it to HKX using HKXPack.

 

If you need, I can give you a list of all the format of the contents to add and a list of all the data to generate. I don't however have a clear idea of the preprocessing needed (I believe the animation data need to be "baked" (not sure of the word, I mean combined with the skeleton pose beforehand) but there may be other things, like rotations, needed before export).

 

I myself never worked with Max plugins, and my experience with Blender plugins (with the Blender Nif exporter form NifTools) wasn't really pleasant. So that's why I wanted to stick to the "external tool" idea. But creating a plugin directly is a good way to go about this, and IIRC I mentioned it a few months back already

[MaikCG]

 

 Animation files only contain information about animation, no object\joint information at all, not even names.

 

 

Joint names and their order = rig bones for HCT which we use when exporting

 

 

 

No.

Skyrim = Havok 2010

FO4 = Havok 2014

 

Uderstand. Indeed they completely changed the order of the recording and the size of the blocks responsible for the storage of information? Although I am far from programming, so it is possible. illogical, but everything is possible

[DexesTTP]

but it is not the same Skyrim? yes, new version, but the same groups of tracks and same spline compression

 

Basically, hkxcmd was a small program that did four things :

- Open a hkx file with Skyrim's Havok API

- Take the animation data using Skyrim's Havok API.

- Save the animation data by using the NifTools API

- Save the file by using the NifTools API

 

Basically, HKXPack is a huge program that does a lot of things :

- Open the HKX file as binary data

- Generate a map of all data structure available in the binary file

- Associate each piece of binary data with the corresponding data structure

- Generate all underlying elements

- Fill all underlying elements

- Convert the decoded data structure to a XML format

- Store the XML format.

 

As you may see, there is no "call X API" in the workflow, because there is no api used by HKXPack. The reason is because there IS no API for Fallout 4's Havok nor (stable) NifLib api for Fallout 4 files.

 

And we can't just use Skyrim's API because it recognizes only data in 4 bytes (32x) while everything Fallout is 8 bytes (64x) so it won't fit.

 

As always, making something so that it would fit is a possibility, but nobody made it ¯\_(ツ)_/¯

[ShadeAnimator]

 

 

Hm, DexesTTP, it's actually interesting information, and I wonder.. wouldn't it be easier, instead of writing a hkx=>fbx converter, to write a max script to parse xml file and create everything that's needed? For example a script to import just animation from xml file on existing skeleton in the scene seems relatively straight forward... Not sure if maxscript has built in xml parser however, uh, I miss Python in max.

 

Hm, actually maxscript can use dotNet, and it has xml parser, and there is a code-friendly wrapper of dotNet on scriptspot.. hah

 

 

Yeah, I mean that's basically what I am doing with HKXAnim - just outside of a plugin. The whole idea is to take this file and fill the blanks, then convert it to HKX using HKXPack.

 

If you need, I can give you a list of all the format of the contents to add and a list of all the data to generate. I don't however have a clear idea of the preprocessing needed (I believe the animation data need to be "baked" (not sure of the word, I mean combined with the skeleton pose beforehand) but there may be other things, like rotations, needed before export).

 

I myself never worked with Max plugins, and my experience with Blender plugins (with the Blender Nif exporter form NifTools) wasn't really pleasant. So that's why I wanted to stick to the "external tool" idea. But creating a plugin directly is a good way to go about this, and IIRC I mentioned it a few months back already

 

 

No, calling it a plugin is not actually correct, I am talking about a script.

In terms of 3d package, at least how it's usually used among people I talk to, a plugin is considered to be a C++ extension of a program, which also often means that it is version-dependent. At least SDK version.

 

But a script, like MaxScript or Mel is 90% of the time NOT dependent on version of the software, and it is.. well, a script. It is not compiled. You can always open it and fix things. Etc.

 

Python in maya (always) and max (since 2017) is somewhere between scripts and plugins. Power of plugins, with full API access, but at the same time it's not compiled (well, it generates pyc files on the fly), and does not depend on version either.

 

So, as you can see, using script languages or python would make compatibility issues go away. You won't need to compile stuff for each max\maya version.

 

But I don't think that it would be possible to read binary data with maxscript or mel... Python can do that though.

 

What I think about is actually physics. You can't store physics data in fbx, as far as I know. But if it was a script, you'd be able to create everything while reading the file... Again, I'm not trying to put you offtrack, just thinking.

 

 

but it is not the same Skyrim? yes, new version, but the same groups of tracks and same spline compression

 

Basically, hkxcmd was a small program that did four things :

- Open a hkx file with Skyrim's Havok API

- Take the animation data using Skyrim's Havok API.

- Save the animation data by using the NifTools API

- Save the file by using the NifTools API

 

Basically, HKXPack is a huge program that does a lot of things :

- Open the HKX file as binary data

- Generate a map of all data structure available in the binary file

- Associate each piece of binary data with the corresponding data structure

- Generate all underlying elements

- Fill all underlying elements

- Convert the decoded data structure to a XML format

- Store the XML format.

 

As you may see, there is no "call X API" in the workflow, because there is no api used by HKXPack. The reason is because there IS no API for Fallout 4's Havok nor (stable) NifLib api for Fallout 4 files.

 

And we can't just use Skyrim's API because it recognizes only data in 4 bytes (32x) while everything Fallout is 8 bytes (64x) so it won't fit.

 

As always, making something so that it would fit is a possibility, but nobody made it ¯\_(ツ)_/¯

 

 

You rock, man, reverse-engineering all this shit from binary data, thats a skill

And I think what MaikCG wants is to import existing animation into max to edit it. It would actually be very handy for animators. it's not always possible to find the source file of animation, but hkx are all there. And you can't get sources of vanilla animations

[MaikCG]

Yes, edit the existing.
But it can be done easily and without an original, I'm curious to see how extra bones work.
How work the twist arms in vanilla animations, I did in the rig twists rotated more or less beautiful, but most likely it is different from the original
How move RibHelper, so may be i can make them automatic
How to move the weapon bones on arms and why they are generally included in the skeleton
How to move the camera bone and root. in Fallout3 movement animation has been a shift, in Skyrim in-place (and speed specify by engine), in Fallout4 again movement animation has been a shift
etc.
See original is very good for brain health

 

https://github.com/Highflex/havok2fbx

not worked for me

may be can help

[ShadeAnimator]

Twist joints usually just twist proportinally to the wrist or elbow, the only thing that can be different are percentages. Or it can be done in engine.

 

Rib helper - well, you can see in my demo video that it works. You can just rotate it and it works If it has lockTranslation on in xml file, then you can't move it, at least unless you edit the skeleton hkx file.

 

not sure what you're concerned about here.

 

As for camera, yeah did not figure that one out yet i think, there are some nodes that i don't know yet how to use.

 

no, that havok2fbx will not work with Fallout4

[invalidfate]

...

skeleton definition hkx file however contains joint names and initial t-pose positions, but not their hierarchy. And it also contains physics related information, from what I can see. Just thinking out loud.

....

 

    Actually, there is an integer array before the bone-name one, describing the parent index of the bones.  However, havok seems only care about their ordering when reading animation hkx, so you just need a rig.txt to specify that when exporting.

 

 

 

I used you rig bones file. Names are case sensitive?

Something wrong

 

https://www.youtube.com/watch?v=A3a5A01rNuQ

 

 

    The bone file just define used bones and their ordering; from the video, it seems the bone mapping is almost correct but some transformation issue presents.  

 

...

 

https://github.com/Highflex/havok2fbx

not worked for me

may be can help

 

   The program is written with once-free havok 2014 32-bit SDK, so you just need HCT 2014 64-bit to convert FO4 64-bit hkx to 32-bit one first, and then use the 1st FO4 skeleton to convert 1st/3rd FO4 hkx to fbx, but unfortunately, some hkx cannot be read, crashing the converter program.

[MaikCG]

 

   The program is written with once-free havok 2014 32-bit SDK, so you just need HCT 2014 64-bit to convert FO4 64-bit hkx to 32-bit one first, and then use the 1st FO4 skeleton to convert 1st/3rd FO4 hkx to fbx, but unfortunately, some hkx cannot be read, crashing the converter program.

 

 

perfect!!!

i import vanilla run

[DexesTTP]

perfect!!!

i import vanilla run

 

Nice

 

 

So, just to check. You guys are currently able to create new animations for FO4, right ?

 

I mean, they work in-game ?

 

 

Is there any functionalities you are missing, like, right now to create new animations for Fallout 4 ?

 

 

Just checking, because if not I'd rather work on stuff other than HKXAnim for a while

[MaikCG]

Yep, i can I'm at all surprised that this has not been done half-year ago (nif-plugin Dec 2015, havok2fbx Nov 2015, HCT 2014).

[Guest]

Yep, i can I'm at all surprised that this has not been done half-year ago (nif-plugin Dec 2015, havok2fbx Nov 2015, HCT 2014).

 

In effect nobody was aware that Creation Kit allows to add new animation files by generating on the fly the behaviors.

 

So everybody was trying to do something like Fore did for Skyrim.

[DexesTTP]

Yep, i can I'm at all surprised that this has not been done half-year ago (nif-plugin Dec 2015, havok2fbx Nov 2015, HCT 2014).

 

Where were YOU half a year ago to do it ?

 

(that was a joke  ) In all seriousness, Highflex let little instructions about how to do that, and even people that managed to reproduce that (like I guess InvalidFate did ?) never told anyone how - at least not with enough information for people to reproduce it.

 

In addition, I'm not an animator (I'm a dev) and so searching through 3DsMax options to find who to export stuff isn't exactly what I'm good at, especially since I don't have 3DsMax installed So I chose to focus on what I knew was necessary long term (a Havok API) and worked to make that happen, without even trying to create animations.

 

Also, and as CPU said, I was originally planning on creating AF4 - a FNIS equivalent for Skyrim - until it became apparent through FO4Edit's progress that there was a "native" way to add animations. The thing is, even now we don't know if AF4 is really unnecessary - as nobody that know what to look for bothered to check if they can do what they need - so we are still in the dark about that.

 

So, if nobody comes up with something for me to do that is completely necessary right now then I'll resume what I started to do (taking a break from HKXAnim until at least October) and I'll enjoy the new mods that'll go out during this time. Because I'm sure that in a few months we will enjoy some real stuff

[ShadeAnimator]

 

 

Yep, i can I'm at all surprised that this has not been done half-year ago (nif-plugin Dec 2015, havok2fbx Nov 2015, HCT 2014).

 

Where were YOU half a year ago to do it ?

 

(that was a joke  ) In all seriousness, Highflex let little instructions about how to do that, and even people that managed to reproduce that (like I guess InvalidFate did ?) never told anyone how - at least not with enough information for people to reproduce it.

 

In addition, I'm not an animator (I'm a dev) and so searching through 3DsMax options to find who to export stuff isn't exactly what I'm good at, especially since I don't have 3DsMax installed So I chose to focus on what I knew was necessary long term (a Havok API) and worked to make that happen, without even trying to create animations.

 

Also, and as CPU said, I was originally planning on creating AF4 - a FNIS equivalent for Skyrim - until it became apparent through FO4Edit's progress that there was a "native" way to add animations. The thing is, even now we don't know if AF4 is really unnecessary - as nobody that know what to look for bothered to check if they can do what they need - so we are still in the dark about that.

 

So, if nobody comes up with something for me to do that is completely necessary right now then I'll resume what I started to do (taking a break from HKXAnim until at least October) and I'll enjoy the new mods that'll go out during this time. Because I'm sure that in a few months we will enjoy some real stuff

 

 

 

 

uhm, really? I uploaded 2 videos with working animations for fo4 for Human and fucking Deathclaw and you did not realise that YES we can import animations in the game? Come o-on.

 

The only thing left was importing existing animations FROM the game INTO max for tweaking, and it seems it is now possible as well. So basically yes, we have all we need to add animations intp the game. And new skeletons, but that will require a bit more work.

 

Sure, you can take a rest. I'm documenting everything me and MaikCG find out, as well as anyone else participating in this. So soon we should know if threr is anything that we really need. HKXPack was needed for sure. Also HKXAnim can reduce export/iteration time i think.

 

 

MaikCG, how do you convert 64bit hkx into 32bit exactly?

[MaikCG]

means of HCT 2014