crashfix Animation Limit Crash Fix

[Aki K]

Can someone explain to me what this mod does like I'm an idiot?

 

I get that it prevents a crash of some kind related to animations.  But I don't really understand the description.  Does it completely remove the limit or just raise it?

[Nymra]

18 minutes ago, Aki K said:

Can someone explain to me what this mod does like I'm an idiot?

 

I get that it prevents a crash of some kind related to animations.  But I don't really understand the description.  Does it completely remove the limit or just raise it?

It allows you to use as many animations as FNIS can provide (26000 with 7.5 XXL). 

That is what I understand

[MadMansGun]

4 minutes ago, Nymra said:

It allows you to use as many animations as FNIS can provide (26000 with 7.5 XXL).

*pending the other limits listed in post #12.

[mrsrt]

My own tests show very good picture too. I saw the game render became more smoother, but didn't think it's related to the patch until last finding. It is especially noticable with 120+ fps and g-sync. Overall, the game behave very good, less stutterings, no crashes, no bugs seen within about 10 hours of tests in total. I was afraid to create a memory leak with skipping of the clean up, but nope, nothing noticeable within an hour of staying at one place with render active.

 

Future plans: as the code has extensive execution during gameplay and important affection on game performance, I'll continue with experiments to find out what else and how deep the part of code can be safely cut with optimization purpose, when I have time.

If no negative impact by the mod will be noticed within beta, I'll make an SSE version when the mod will come to its final state. 

[galgat]

I have been able to increase my animation load order considerably above what I could before, and have encounter no real problems other than an increase in time and sometime unresponsive SLAnimLoader when re-registering sexlab data, and resetting the json file during the update of animations with SLAnimLoader.

 

This was only apparent to me when redoing SLAnimLoader on a second or third attempt. Also only in one profile, so it could be a priority problem on the left side of MO2, as I use Loot for the actual load order. This only happened on one profile so far so I suspect it is a priority of mods problem.

 

EDIT >> That profile use's DDI4.0 and not 4.3, so the reserved slots in 4.0 could have something to do with that. A mod in that profile desires 4.0 as apposed to 4.3 or above.

[user9120975435]

Bloody amazing mod, works brilliantly! 102% CTD and loads no problem.

 

Big kudos from me, this deserves to be pinned!

[velocipreys]

Thank you so much for making this mod man!

[GenioMaestro]

I can't believe what I see... You did it... Amazing... Fantastic... Very well done... THANK YOU...

 

Ussing only fill mods i can load 26200 animations in character of type b whitout any parameter.

Spoiler

Skeleton(hkx) female: Default (99 bones)   male: Default (99 bones)
Patch: "GENDER Specific Animations"  
Patch: "SKELETON Arm Fix"  

Reading FILL-b-2k-0 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-1 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-10 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-11 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-12 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-13_200 V?.? ...     ChAnims:200     CTD:2,3%     pOpt:1,5%
Reading FILL-b-2k-2 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-3 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-4 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-5 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-6 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-7 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-8 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FILL-b-2k-9 V?.? ...     ChAnims:2000     CTD:23,0%     pOpt:15,3%
Reading FNISBase V7.5 ...     ChAnims:0     CTD:0,0%     pOpt:0,0%
Reading FNISCreatureVersion V6.1 ...     ChAnims:0     CTD:0,0%     pOpt:0,0%

All Anim Lists scanned. Generating Behavior Files....
 0 GENDER modifications for Animations\male
 0 GENDER modifications for Animations\female

Create Creature Behaviors ...

 26200 animations for 16 mods successfully included (character).
ChAnims: 26200  CTD:300,8%  pOpt:200,3%  max: 8711  LC: 78685 (max. 26162)

>>Warning: Too many custom animations. Your Skyrim is likely to crash (CTD, crash to desktop)<<

 

Whitout your patch my game made CTD with only 8707 animations of the same type and parameters.

Whit your patch the character block not have limit. Works in the same way as the creatures.

 

But I can't understand why. I will do more tests and report any progress.

[cosworth]

Congratz and thanks a million, you kinda tamed our untamed Skyrim.?

[GenioMaestro]

I have done a detailed investigation examining the HKX files read by the game.

I am using ProcessMonitor to filter access to files with the process name "TesV.exe" and the path ends with "hkx" and the game loads all HKX files before starting the render. "all" means each behavior for characters and creatures and each individual HKX file for each stage of each actor in each animation.

 

I guess all those files are read by the Havok engine included in the game because it uses a FASTIO_NETWORK_QUERY_OPEN read function while the animationdatasinglefile.txt and animationsetdatasinglefile.txt special files are read differently using IRP_MJ_READ and I guess they are read using a patch created by Beth developers.

 

When we click on New Game, the game reloads some HKX files, but ONLY some HKX files, probably because Beth developers need to make additional patches in Havok's default management, and that process is the cause of all the problems.

To start, when we click on New Game, the game does not read any creature files except draug and tail.

That may be the reason why creatures animations have no limit.

Next, read each individual AA animation along with the default behaviors for walking, running, magic, idle...

Also, read all the character behavior files, probably looking for special animations, maybe AA.

When examining the patched function, it seems that Beth's stupid developers, when they process each and every one of the animations included in each behavior file, when the animation does not have AA or any special animation, they skip it by putting 0 within its matrix internal for AA animations and that causes overflow and CTD depending on how many animations we add.

With the patch that stupid function is not executed and the internal matrix for the AA animations is not overloaded and the game does not do CTD for excessive animations.

The only drawback of the patch may be when the user has special animations that do not need to be processed by FNIS to be included in the game. Perhaps in this situation the patched function skips those animations. I don't have any of that, but anyone who uses those special animations can do a quick test.

[mrsrt]

1 hour ago, GenioMaestro said:

That may be the reason why creatures animations have no limit.

I think the creature "limit" exists too, but never passed. Take a look at this post: https://www.loverslab.com/topic/134659-beta-le-animation-limit-crash-fix/?tab=comments#comment-2836278

For example, for horses I got 000000D8 (216), for something else I got 00001045 (4165) what's already big enough. These values comes from the same place I patched:

00BE6CB7     0FBF86 A800000>MOVSX EAX,WORD PTR DS:[ESI+A8]

I think creature animations has additional separations, maybe by skeleton, maybe by something else and because of these separations we never passed the creature "limit".

 

What was really surprised me is that the function executes during rendering and has direct influence on game performance. I'm looking now at what else I can cut from this code, but no positive results yet. 

[fore]

On 12/8/2019 at 11:45 AM, mrsrt said:

As I understand, Fore did some restrictions and optimizations with registry to prevent passing the internal game limit. Now the limit is skipped, so, if my solution will pass beta, probably Fore can completely remove or largely increase his limit in FNIS. It will need some tests ofc, but for now need to listen feedback at least for 2 weeks if my decision to skip code was really safe. 

 

It's not really that easy. I cannot simply remove the limit, change a few parameters and everything is ok. FNIS wasn't designed to be extended indefinitely (actually it all started with hundred lines of Excel script). And every time I extend the limit it takes 2 or 3 releases to figure out which one of the dozens of arrays might overflow as well.

 

And it will also effect performance drastically. Especially for users with "potatoes".

 

This game is 8 years old, and user have learned to live with 11 to 13k. If they have 26k now it is twice as much. Or more, when I make it a round 30k. But isn't that REALLY enough?

[fore]

20 minutes ago, mrsrt said:

I think the creature "limit" exists too, but never passed. Take a look at this post: https://www.loverslab.com/topic/134659-beta-le-animation-limit-crash-fix/?tab=comments#comment-2836278

For example, for horses I got 000000D8 (216), for something else I got 00001045 (4165) what's already big enough. These values comes from the same place I patched:

00BE6CB7     0FBF86 A800000>MOVSX EAX,WORD PTR DS:[ESI+A8]

I think creature animations has additional separations, maybe by skeleton, maybe by something else and because of these separations we never passed the creature "limit".

 

What was really surprised me is that the function executes during rendering and has direct influence on game performance. I'm looking now at what else I can cut from this code, but no positive results yet. 

I have tested successfully with 26k for character PLUS 2 time 28k for 2 different creatures.

[V3rius]

15 minutes ago, fore said:

This game is 8 years old, and user have learned to live with 11 to 13k. If they have 26k now it is twice as much. Or more, when I make it a round 30k. But isn't that REALLY enough?

Its never enough!  We thought 1k is enough. Then 7. Then 8. Then 12. There was a time to stop. We passed it long ago so keep going.

 

Next stop: 2,147,483,647

 

[fore]

1 hour ago, GenioMaestro said:

Next, read each individual AA animation along with the default behaviors for walking, running, magic, idle...

Also, read all the character behavior files, probably looking for special animations, maybe AA.

 

AA (if you talk about FNIS Alternate Animations) are nothing but replacements of vanilla animations. The place which makes AA's beeing read (for pre-cache) is in the animationSETdatasinglefile.txt. I have added entries for each the AA's at every place where I already found an entry for the base vanilla file. And that was usually several time per animation file.

 

But this type of reading was not done during start, as far as I remember. It was always done (for the respective set of animations) when the player changes the weapon.

[fore]

On 12/9/2019 at 5:29 PM, mrsrt said:

I continued the research and found very interesting details.

These results may say that if we really can safely skip the part of code we aren't just fixing the CTD and increasing loading speed, but also boosting up FPS and overall game performance. 

 

Unfortunatelly, strings stored as char arrays in memory, so you'll not see any text in the dump. However, if you want to give it a try I attached the block the loop tries to clear. The first pointer points at 35610108, player's counter is 19CB, so the last pointer should be at 35614E69 (each itr +3). 

 

No, sorry. There is nothing I feel familiar in any way.

 

I'm also afraid that this might be a ghost chase. 19cb/6603 is your number of character animations? Where do you get this number from? I'm asking because I never found "number of character animations" an important criteria. (except in the list of animations in default(fe)male.hkx. This 26162 is a "behavior value". And when reading animations (for pre-cache) the game will always read only a small subset.

[mrsrt]

2 hours ago, fore said:

It's not really that easy. I cannot simply remove the limit, change a few parameters and everything is ok. FNIS wasn't designed to be extended indefinitely (actually it all started with hundred lines of Excel script). And every time I extend the limit it takes 2 or 3 releases to figure out which one of the dozens of arrays might overflow as well.

 

And it will also effect performance drastically. Especially for users with "potatoes".

 

This game is 8 years old, and user have learned to live with 11 to 13k. If they have 26k now it is twice as much. Or more, when I make it a round 30k. But isn't that REALLY enough?

Before the patching I was thinking about alternative ways to cure the problem. I had several ideas like animation batching or forcing the game to load animations directly without behaviors, and several else. All of them requires tons of work and after that thinking I came to the same question, do we really need that?

When patch was created I started to search for a lot of animations, I had about 10.3k, and I was need to find 16k more to test it out. And, ofcourse, I didn't find them throughout whole Nexus & LL and simply used your fill-up files (rly thanks for them). After that the answer of my question was obvious. 

With my post I just meant that the game shouldn't CTD anymore because of animation count and FNIS can completely ignore the factor during anim processing and developing.

 

As for me, I was living with 8.7k anims in total, installed a mod that boosted the amount to 10.3k and got BE6DD3 CTD. With face like that I started to work on it

Spoiler

Only 10.3k!

 

2 hours ago, fore said:

I have tested successfully with 26k for character PLUS 2 time 28k for 2 different creatures.

Can you please publish these fill-up files for creatures? I'd like to see what will happen inside. I have several thoughts why it's possible.

No need already.

[GenioMaestro]

1 hour ago, mrsrt said:

I think the creature "limit" exists too, but never passed.

8 minutes ago, mrsrt said:

Can you please publish these fill-up files for creatures? I'd like to see what will happen inside. I have several thoughts why it's possible.

https://www.loverslab.com/topic/94228-sse-conversion-tracking-dec-8-4279/?do=findComment&comment=2795127

 

The creature animations never was the problem and we not find any limit in creature animations.

For long time we sum the animations from character + creatures thinking that the problem was the total number of animations. But that idea has been discarded because, until now, we can not find a way for affect the limit of the character animations ussing creature animations. And seems that your patch remove that limit.

[mrsrt]

8 minutes ago, fore said:

19cb/6603 is your number of character animations? Where do you get this number from? I'm asking because I never found "number of character animations" an important criteria. (except in the list of animations in default(fe)male.hkx. This 26162 is a "behavior value". And when reading animations (for pre-cache) the game will always read only a small subset.

This number I got directly with debugger in that place I patched. I don't know what that value means exactly, but can say for sure this is the one that makes problems and changes dependently of player's anim count/behaviours. I got exactly that value when simply removed character's animation and behavior folders, txt's I didn't touch.

I'll try to make some additional tests now. 

[GenioMaestro]

I make a interesting discover and probably find the answer to one of the most hard problems that @fore had when computing the space used by each animation type.

 

Why the "mag movement and magcast movement didn't have any impact on the animation limit"

 

The answer: Because the file magic_readied_direction_behavior.hkx is NOT included inside 0_master.hkx

 

I compare the names of the behavior files loaded before the game start and the names of the behavior files loaded when press New Game and .... the file magic_readied_direction_behavior.hkx is NOT read:

Spoiler

 

Seems that the function for preload the AA animations only process the behaviors files included inside 0_master.hkx and not process each one of the behavior files.

 

For example, I delete my fill mods in actors\character\animations and FNIS not process it but i not delete the corresponding FNIS_FILL-b-2k-xx_Behavior.hkx in the folder actors\character\Behaviors.

You can see how that files are read when the game start but are not read on New Game:

Spoiler

 

In that way, my suposition that the mods with animations that not need be procesed by FNIS can have problems not have any sense. That behavior files are read before the game start but are not procesed by the preloader of AA animations. That is the default functionality of the game and not every animation mod need be procesed by FNIS.

 

In the same way, i have one idea.

If the game load every behavior file before start the render and the animations works, then, is not necesary include every behavior file inside 0_master.hkx and ONLY is necesary include the behavior files that have AA animations or special animations because need be procesed by the AA preloader.

That operation can solve in a definitive way the CTD caused by excesive animations.

[mrsrt]

Test 1 - No animations

 

The meshes folder completely removed.

Loading calls:

Spoiler

0000000D
0000000D
0000000D
00000007
00000007
00000005
00000005
00000007
00000007
00000007
00000007
00001A47
00001A47
00001026
00001026

 

 

Even with no additional animations there come quite big values: 1A47 (6727) and 1026 (4134). These values are static and doesn't change between game instances (unless I change something). I'm not sure, maybe my bsa's contain and replace something and this is not the real "clean" values, however, no additional animations added by FNIS for sure.

 

Test 2 - Add troll animations

 

I took default troll animations from sexlab for the test.

Loading calls:

Spoiler

0000000D
0000000D
0000000D
00000007
00000007
00000005
00000005
00000007
00000007
00000007
00000007
00001A47
00001A47
00001026
00001026

 

 

Nothing changed within these calls. 

 

Test 3 - Add player animations

 

For the test I took RohZima's anim pack: https://www.loverslab.com/files/file/5224-rohzima-slal-pack-june-2019/

 

Loading calls:

Spoiler

0000000D
0000000D
00000007
00000007
00000005
00000005
00000007
00000007
00000007
00000007
00001D89
00001D89
00001026
00001026

 

 

Player's value change: 1A47 -> 1D89. +834 (dec). 

 

Test 4 - Add the rest creature animations

 

Loading calls:

Spoiler

0000000D
0000000D
0000000D
00000007
00000007
00000005
00000005
00000007
00000007
00000007
00000007
00001D89
00001D89
00001026
00001026

 

 

No change. Yes, now we can confirm, creature anims does not participate here at all. 

 

Test 5 - Add a fill-up file

 

Loading calls:

Spoiler

0000000D
0000000D
0000000D
00000007
00000007
00000005
00000005
00000007
00000007
00000007
00000007
0000255E
0000255E
00001026
00001026

 

 

Player's value change: 1D89 -> 255E. +2005 (dec).

 

Test 6 - Add TkDodge anims

 

Loading calls:

Spoiler

0000000D
0000000D
0000000D
00000007
00000007
00000005
00000005
00000007
00000007
00000007
00000007
0000255E
0000255E
00001045
00001045

 

 

The 2 values AFTER player's ones was changed: 1026 -> 1045. +31 (dec). An another perspective bottleneck.

[fore]

9 hours ago, GenioMaestro said:

I make a interesting discover and probably find the answer to one of the most hard problems that @fore had when computing the space used by each animation type.

 

Why the "mag movement and magcast movement didn't have any impact on the animation limit"

 

The answer: Because the file magic_readied_direction_behavior.hkx is NOT included inside 0_master.hkx

 

I compare the names of the behavior files loaded before the game start and the names of the behavior files loaded when press New Game and .... the file magic_readied_direction_behavior.hkx is NOT read:

 

Seems that the function for preload the AA animations only process the behaviors files included inside 0_master.hkx and not process each one of the behavior files.

 

For example, I delete my fill mods in actors\character\animations and FNIS not process it but i not delete the corresponding FNIS_FILL-b-2k-xx_Behavior.hkx in the folder actors\character\Behaviors.

You can see how that files are read when the game start but are not read on New Game:

 

In that way, my suposition that the mods with animations that not need be procesed by FNIS can have problems not have any sense. That behavior files are read before the game start but are not procesed by the preloader of AA animations. That is the default functionality of the game and not every animation mod need be procesed by FNIS.

 

In the same way, i have one idea.

If the game load every behavior file before start the render and the animations works, then, is not necesary include every behavior file inside 0_master.hkx and ONLY is necesary include the behavior files that have AA animations or special animations because need be procesed by the AA preloader.

That operation can solve in a definitive way the CTD caused by excesive animations.

 

Interesting find, but wrong conclusion.

 

I never noticed that magic_readiead wasn't called from 0_master.hkx. Because not all other behavior files are called from 0_master. Only 12 of 17. Idlebehavior.hkx, for example, is called from mt_behavior.hkx (via "hkbBehaviorReferenceGenerator")

 

But that doesn't mean that magic_readiead is used in a different way. It means that magic_readiead isn't used AT ALL. Interestingly they have (unnecessarily) called magicbehavior.hkx TWICE. Probably wanted to call magic_readiead, but used the wrong name.

 

Still doesn't explain why those mag/magcast animations aren't counted. Because they are all referenced in magicbehavior as well.

[GenioMaestro]

2 hours ago, fore said:

 

Interesting find, but wrong conclusion.

 

I never noticed that magic_readiead wasn't called from 0_master.hkx. Because not all other behavior files are called from 0_master. Only 12 of 17. Idlebehavior.hkx, for example, is called from mt_behavior.hkx (via "hkbBehaviorReferenceGenerator")

 

But that doesn't mean that magic_readiead is used in a different way. It means that magic_readiead isn't used AT ALL. Interestingly they have (unnecessarily) called magicbehavior.hkx TWICE. Probably wanted to call magic_readiead, but used the wrong name.

 

Still doesn't explain why those mag/magcast animations aren't counted. Because they are all referenced in magicbehavior as well.

Well, i'm not expert in animations and i not have your deep knowneledge about the animations system.

But i'm a profesional developer and my experience allow me imagine how internally works the animations.

But take good note that i imagining and, of course, i can be very wrong in some situations.

 

First, i must say that the file magic_readiead.hkx is readed ONLY before the game start.

But that file is not referenced inside any other HKX file and that make me imagine that, really, the animations included inside magic_readiead.hkx are not used by the game and can not be played in any way.

As you say, that can have all the sense because that animation are repeated inside magicbehavior.hkx but the point that you not understand is WHY the animations inside magicbehavior.hkx not use space.

 

I alrready answered the question: Because the file magicbehavior.hkx is not referenced inside 0_master.hkx

 

The file magicbehavior.hkx is referenced inside 1hm_behavior.hkx whit this XML code:

Spoiler

        <hkobject name="#3354" class="hkbBehaviorReferenceGenerator" signature="0xfcb5423">
            <!-- memSizeAndFlags SERIALIZE_IGNORED -->
            <!-- referenceCount SERIALIZE_IGNORED -->
            <hkparam name="variableBindingSet">null</hkparam>
            <!-- cachedBindables SERIALIZE_IGNORED -->
            <!-- areBindablesCached SERIALIZE_IGNORED -->
            <hkparam name="userData">0</hkparam>
            <hkparam name="name">1HM_MagicBFR</hkparam>
            <!-- id SERIALIZE_IGNORED -->
            <!-- cloneState SERIALIZE_IGNORED -->
            <!-- padNode SERIALIZE_IGNORED -->
            <hkparam name="behaviorName">Behaviors\MagicBehavior.hkx</hkparam>
            <!-- behavior SERIALIZE_IGNORED -->
        </hkobject>

 

I take it from the original animation file of the game, expanding Skyrim - Animations.bsa and decompresing the behavior HKX files ussing hkxcmd.exe for get the XML code.

 

In the same way, as you say, not ALL the HKX files are included inside 0_master.hkx

The file Idlebehavior.hkx is included inside mt_behavior.hkx ussing another hkbBehaviorReferenceGenerator.

In the same way, others HKX files are not included inside 0_master.hkx but, as they are referenced inside others HKX files, the game load it and process it whitout need a reference inside 0_master.hkx

 

And that is my main point. I try explaing it in the most easy way.

Seems that the Preloader of the AA animations process each one of the HKX files referenced inside 0_master.hkx but not process the rest of the master HKX files of the game like mt_behavior.hkx

Seems that is not strictly necesary reference each FNIS_xxx_Behavior file inside 0_master.hkx

Probably, the animations can works in the same way if you include it inside mt_behavior.hkx

 

Simply making that, the Preloader of the AA not process animations that not need AA and the animation limit automatically disapear because the game make CTD when the Preloader of the AA process each one of the animations included inside each FNIS_xxx_Behavior file included inside 0_master.hkx

 

Seems that the CTD is caused because the internal array of AA animations is overloaded, because the function is bad developed, and the patch created by mrsrt exactly skip the section of code that cause the overload.

And seems that happend because not ALL the behaviors files need be referenced inside 0_master.hkx

 

I think that only SOME of the FNIS_xxx_Behavior files need be referenced inside 0_master.hkx, probably only the FNIS_xxx_Behavior that have AA animations need be included, but the rest of the the FNIS_xxx_Behavior can be referenced inside other master HKX file of the game like mt_behavior.hkx

 

Is the only posible explanation about WHY the animations included inside magicbehavior.hkx not consume any space. Because the file magicbehavior.hkx is included inside mt_behavior.hkx and not exist inside 0_master.hkx

[mrsrt]

2 hours ago, GenioMaestro said:

Seems that the CTD is caused because the internal array of AA animations is overloaded

At the beginning I was thinking that it's a simple block overflow, coz exceptions point on access violation. During the research I found that it's a 16mb block that was placed together with others pretty similar blocks, they look like chunks allocated consistently. Their count was unpredicted, but avg pattern was like: more anims -> more blocks, what may mean that they will be allocated dynamically when needed. I successfully even increased their size to 32mb, but, ofc, it didn't help with the problem.

It looks more like block(s) for some data with raw access by offsets. As for the overflowing counter, this is the real reason of the CTD which should take attention. 

I also notice once again, this code has extensive execution not only while loading but also during frame processing, dependently on npc/creatures around up to several calls per one frame. 

[Rod Broward]

On 12/9/2019 at 1:11 PM, thecoon101 said:

If this actually works without huge bugs this is probably the best new skyrim mod that came out this year on this site. Too bad i don't play LE anymore i wish a mod like this came to SE

cope sissy