fore Posted February 20, 2012 Posted February 20, 2012 HKXs are just an archive format' date=' like .zip or .bsa. Even if hkxcmd can't be updated to understand the format of the system bethesda used, it could still just dump things it doesn't understand to raw data in the .xml file and recompile it. The real "problem" is trying to edit the hkxs in a hex editor to start with. [/quote'] I hadn't realised that hkxcmd could successfully repack hkx files at that level. That could certainly make life a lot easier. No, this is a total misunderstanding. Especially hkx is NOT an archive format. Havok stores 2 totally different things in hkx. For one, there are animations and (animation related) skeleton. This is comparable to .kf and .nif in Oblivion. Tools and documentation to handle these formats are freely available (although not to the desired extent), because they are part of Havoks free Animation and Physics, and Content Tool. And this information was used for hkxcmd. But BESIDES that, these hkx files are used as container for a trimmed and compressed, game-specific binary version of the behavior project files, probably generated by a bethesda internal plugin to the proprietary HBT. Both formats seem to have almost nothing in common, and people who have REALLY looked into that, don't see any similarity. In my opinion I doubt that we will ever be able to understand hkx behavior, unless there will be some information available which we don't know about today. Cracking with current information seems at least much, much harder than it was to crack NIF.
prideslayer Posted February 20, 2012 Posted February 20, 2012 HKXs can store a lot more than just those two things. Models, textures, cloth definitions, AI... basically anything can be stored in an HKX. It is the archival (distribution, packed, whatever you prefer to call it) format for all of the objects that Havok can create or use. All that means is that it can store many different things. You don't have to package all of those things in an HKX, but you can. hkxcmd uses the SDK libraries to extract the record types that the SDK understands via hkSerializeUtil, so behaviors are not included. However, if the HKX record format overall can be figured out, then that is enough. There's no need to understand what every structure looks like, so long as there's an overall structure they all conform to, which seems likely. All that is needed is to understand the size of the items in the container, and then the data can be serialized and unserialized in a raw format for items that aren't understood. It is definitely going to be possible, without a *ton* of work, to come up with a union struct that can map every record type in the HKX, without having to actually understand all the fields. The goal of modifying behaviors is distinct from the goal of simply extracting and repackaging them.
fore Posted February 21, 2012 Posted February 21, 2012 HKXs can store a lot more than just those two things. Models' date=' textures, cloth definitions, AI... basically anything can be stored in an HKX. It [i']is[/i] the archival (distribution, packed, whatever you prefer to call it) format for all of the objects that Havok can create or use. All that means is that it can store many different things. You don't have to package all of those things in an HKX, but you can. hkxcmd uses the SDK libraries to extract the record types that the SDK understands via hkSerializeUtil, so behaviors are not included. However, if the HKX record format overall can be figured out, then that is enough. There's no need to understand what every structure looks like, so long as there's an overall structure they all conform to, which seems likely. All that is needed is to understand the size of the items in the container, and then the data can be serialized and unserialized in a raw format for items that aren't understood. It is definitely going to be possible, without a *ton* of work, to come up with a union struct that can map every record type in the HKX, without having to actually understand all the fields. The goal of modifying behaviors is distinct from the goal of simply extracting and repackaging them. Yes, I was unprecise. It can contain more that 2 things (which I meant related to animation). But it certainly doesn't contain AI packages (unlike behavior), textures. I don't understand your point about reading the record structure of HKX. What does this help you when the records constist of KBs of unstructered binary data? Have you ever looked into one of the xml files converted with hkxcmd? Tons of global type definitions, numbered references (no semantics at all), and 50% and more hex data. But this is not the only problem. If we had to decrypt this animation hkx bulk date, we might even succeed. Because there are tools (CCCAsset2, HCT's 3DS export) where we could compare and iterate generated output whith what we feed it. With behavior it's different. We don't even have a tool where we can compare input and output. All we have are those (relatively few) existing behavior file. And without knowing, what kind of information they represent. So what do you want to understand?
jester18 Posted February 21, 2012 Posted February 21, 2012 I must admit I am completely out of my league here. I did a quick search and didn't find any reference so wasn't sure if you had already found it but managed to get the folowing to work in the console. player.sae idlecowering player.sae idlestop
DocClox Posted February 21, 2012 Posted February 21, 2012 The thing about deserialisation, I suppose is it all depends on whether the information is stored in the file or not. I mean some list formats are like this struct element { int size; void *data; } struct archive { int count; struct element[]; }; And they're great because you can work out the size of each element and you don't need to know what's in it. Others though might look like this struct element { int record_type_id; double some_mumber; char flag_array[16]; int size_of_blob; void *arbitrary_data_blob; }; In this case the deserialisation depends on looking up a function to read the record format corresponding to record_type_id from the code. So the length data isn't stored for the record as a whole. There is a size for the arbitrary blob, but it's buried deep enough in the struct that you're unlikely to identify it as such. And of course there could be several such blobs. Which isn't to say that you can't hack the format, but it can be a lot more work than you might expect of a simple archive format. Especially if you work from a limited data set. If course, we don't know which model the hlx files use ... but judging from the attempts in this topic to find the embedded size data, I'm tending to assume the deserialisation info is in the executable code rather than the archive itself. On a related note: whichever way we do this, we'll need custom idle objects in the game. We can't (so far as I can see) add them with Papyrus, so we're either going to have to add them by hand in the CK, or else ... has anyone used the batch edit features in TESVSnip? I've seen references to batch mode either in the docs or the github repo, but I can't see how to access it. It could allow us to build an esp from scratch that just contained idles for the new animation routines. is there a npc command for sae Probably you can click on the npc and then type sae without the player. prefix. [edit] This sounds like it might be useful if we need to automate a hundred new Idle records. Or add a new package to every NPC in the game, maybe, perhaps
Rosewind Posted February 21, 2012 Posted February 21, 2012 I must admit I am completely out of my league here. I did a quick search and didn't find any reference so wasn't sure if you had already found it but managed to get the folowing to work in the console. player.sae idlecowering player.sae idlestop When was that console code discovered? lol saves a lot of time instead of swapping idle animations override thingy
prideslayer Posted February 21, 2012 Posted February 21, 2012 Yes' date=' I was unprecise. It can contain more that 2 things (which I meant related to animation). But it certainly doesn't contain AI packages (unlike behavior), textures. I don't understand your point about reading the record structure of HKX. What does this help you when the records constist of KBs of unstructered binary data? Have you ever looked into one of the xml files converted with hkxcmd? Tons of global type definitions, numbered references (no semantics at all), and 50% and more hex data. [/quote'] I have. The point is that if you only want to change the names of things, you do not have to understand their content. You can serialize the HKX to XML, and then change just the things you recognize, and re-"compile" it back to native format without ever having to understand what all that binary data is. It really IS a "zip file". It's a container for havok "stuff" the same way a zip file is a container for files. What we're doing here with this renaming business is like trying to open a zip in a hex editor and change the name of the files inside it. Of course there are going to be problems with that. What I'm suggesting is unzipping the file, renaming the files it contains, and then rezipping it. You do not need to understand the contents of the files themselves, and likewise, you do not need to understand WHAT all that binary "junk" in the serialized HKX is. But this is not the only problem. If we had to decrypt this animation hkx bulk date, we might even succeed. Because there are tools (CCCAsset2, HCT's 3DS export) where we could compare and iterate generated output whith what we feed it. With behavior it's different. We don't even have a tool where we can compare input and output. All we have are those (relatively few) existing behavior file. And without knowing, what kind of information they represent. So what do you want to understand? I don't care about doing any of that stuff, my comments were only regarding why the names have to be the same length when hex editing the HKXs, and the reason is simple: It's a packed structure, exactly like any other kind of container file, and you can't change the 'length' of stuff without screwing up all the other offsets in the file. You might be able to make shorter names by null padding the space, but this is just how it goes when you're hex editing a binary file. If the HKX is serialized, then you can rename everything in it to your hearts content, and it will recompile just fine. You don't have to understand (or touch) all that binary data that isn't properly understood in order to just serialize and unserialize it.
prideslayer Posted February 21, 2012 Posted February 21, 2012 DocClox, you're picking up what I'm laying down. I actually expect that the format is closer to the second one you have, with the addition of some union magic (which makes things both easier and harder). If their structure is something like your 2nd example struct element { int record_type_id; double some_mumber; char flag_array[16]; int size_of_blob; void *arbitrary_data_blob; }; We can code it exactly like that, and serialize and unserialize it, without ever knowing (or caring) what that arbitrary_data_blob actually contains. That is/was my point to fore. It would be nice to really understand what all the records are, but that's a pipe dream. Even the fields in NIF/KF format are still not entirely understood. The important thing is that we don't need to understand all of it, just enough to serialize and unserialize it is enough. It'll be tough to crack regardless, since generating sample behaviors isn't something we can easily do either. However, we have a LOT of them to compare to one another, which is all that most cryptanalysts have to start with, and they are doing something far more difficult.
philipcollin Posted February 21, 2012 Posted February 21, 2012 is there a npc command for sae Probably you can click on the npc and then type sae without the player. prefix. sae = sendanimevent so you can click on the npc, type sendanimevent idleflutestart and itll play ont hat npc
DocClox Posted February 21, 2012 Posted February 21, 2012 struct element { int record_type_id; double some_mumber; char flag_array[16]; int size_of_blob; void *arbitrary_data_blob; }; We can code it exactly like that' date=' and serialize and unserialize it, without ever knowing (or caring) what that arbitrary_data_blob actually contains. That is/was my point to fore. [/quote'] Oh coding it like that is trivial ... once we know the format. It's the business of discovering the format that sounds like a lot of work. Then again "on no account should the one who says a thing is impossible be allowed to interrupt the one who is busy doing the thing in question". I think I'll bow out gracefully from this part of the discussion
Rosewind Posted February 21, 2012 Posted February 21, 2012 This might sound really stupid but, you know how you can duplicate animations in the Gameplay/animations window. Is there anyway to "replace" the duplicate with out overriding the original? Not really up to speed with that side the creation kit.
prideslayer Posted February 21, 2012 Posted February 21, 2012 This might sound really stupid but' date=' you know how you can duplicate animations in the Gameplay/animations window. Is there anyway to "replace" the duplicate with out overriding the original? Not really up to speed with that side the creation kit. [/quote'] No. This wouldn't be a 25 page thread figuring out how to get this done if it were that easy..
Rosewind Posted February 21, 2012 Posted February 21, 2012 This might sound really stupid but' date=' you know how you can duplicate animations in the Gameplay/animations window. Is there anyway to "replace" the duplicate with out overriding the original? Not really up to speed with that side the creation kit. [/quote'] No. This wouldn't be a 25 page thread figuring out how to get this done if it were that easy.. I know but no one mentioned it that I can remember of, been reading this since the beginning. Some times the most simple thing can be over-looked though.
prideslayer Posted February 21, 2012 Posted February 21, 2012 I'm starting to wonder if there IS an easier way to do all this. I may be completely off base, but I don't yet see any reason why we couldn't create an entirely new behavior from scratch, and assign all of our animations to that. There's been concern over hijacking an existing animation, tied to an existing behavior, and rightly so. The behavior can have all kinds of garbage in it we don't know about, unrelated to the animation we want to play. For simple animations like these (sex animations), it seems we could avoid that issue by creating our own behavior. The only hurdle to that is the availability of the behavior tool -- bethesdas behavior source not required. This is assuming that havok won't complain about using a behavior generated by an older version of its tools. I'm also forced to wonder why they don't make their behavior source files available, those are certainly not covered by any redistribution rules that Havok has in place. They (bethesda) know that the HBT is out there and that people have it, even if it's an old version. They may be 'useless' from the standpoint of recompiling them since we lack the modern tools, but they are far from useless from an educational standpoint, and may provide real clues into decoding the compiled behaviors.
littleman Posted February 21, 2012 Posted February 21, 2012 Perhaps it was designed to stop folks here at LL and JP sites. What would be needed to test this theory of yours Pride?
DocClox Posted February 21, 2012 Posted February 21, 2012 Perhaps it was designed to stop folks here at LL and JP sites. What would be needed to test this theory of yours Pride? Well, we need the old version of havok behavior from the intel site. Not sure where, but there's a link in the thread in two or three places. Someone who understodd havok bahavior well enough to use it to make an animation. Something simple would do - something that isn't in the game already. In fact we could do with two of them so we can make sure the format packages sepatate animations into one file - but we don't need that for the first test. Then we need to plug the animation into the game. I think we'll still need a second race to make it work, but we can do that. And then we see what happens. Anyone want to give it a shot?
cyanure Posted February 21, 2012 Posted February 21, 2012 May not be usefull,but who know? http://www.youtube.com/watch?v=RGnSqPsdj40 http://www.youtube.com/watch?v=sclYyTiqRrw http://www.youtube.com/watch?v=aaT__JhTI2E as ever if there is aproblems I remove the links addendum http://www.develop-online.net/news/30855/Havok-offers-up-Behaviour-Tool-online
caveat73 Posted February 22, 2012 Posted February 22, 2012 Not sure if anyone's seen this... Animated Prostitution - Beta He claims there's no replacer. I tried it and it seems to work... http://skyrim.nexusmods.com/downloads/file.php?id=10748
Imperialarg Posted February 22, 2012 Posted February 22, 2012 Not sure if anyone's seen this... Animated Prostitution - Beta He claims there's no replacer. I tried it and it seems to work... http://skyrim.nexusmods.com/downloads/file.php?id=10748 It works. Lol seems legit. And so it begins!
RogueMage Posted February 22, 2012 Posted February 22, 2012 - Hairstyle changes (maybe change of race?) - sometimes sexpartner is invisible - extremly stiff maybe a first step.
fore Posted February 22, 2012 Posted February 22, 2012 - Hairstyle changes (maybe change of race?) - sometimes sexpartner is invisible - extremly stiff maybe a first step. He does change race. But not only that. He also makes a COMPLETE copy of "character" into "charactersexy". Where at least the mt_idle is changed. 2285 files. If that's what's needed to make one sex animation .....
Teechan Posted February 22, 2012 Posted February 22, 2012 *looks inside zip folder* My goodness...look at all those .pex ... Terrible joke, I know. *ducks the rotten tomatoes*
labrat Posted February 22, 2012 Posted February 22, 2012 It's more progress. Things are really moving fast despite the hurdles that have been put in our way. Hopefully some of that bloat will get refined away, and the snargs get ironed out. Different people have different skills. This guy is good at working out the implementation details but someone else might be a better animator, or more efficient coder, and all their efforts will pay off. The level of collaboration in a common cause here is a shining example that should be adopted in schools, universities, businesses and government. If they discovered an alien planet full of butt-nekkid beauties we'd have faster than light travel within a year.
cyanure Posted February 22, 2012 Posted February 22, 2012 So far,seem it change the eyes color and sometime the haircuton the female and suffer of the "Bestheda related" grey face for the men,sure it is a little "simple" with one animation,but that s a start and a good one I think... if I may,someone could send him an invite to LL?(my excuse if it as already been done)
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now