HDT Physics Extensions Crash Fix

March 30, 2020

HDT Physics Extensions Crash Fix

Introduction:

This SKSE plugin fixes hdtPhysicsExtensions.dll+0x15E2EC crash (generally confused as a load game crash because it crashes right after loading your first save).

The crash happens due to a race condition when calling TlsGetValue (which returns 0/NULL when uninitialized), followed by HDT PE not checking for this possibility, doing arithmetic on a null pointer and finally dereferencing the result of this arithmetic, thus causing the game to crash for accessing an invalid memory address.

The solution was to make the function return immediately if the result of TlsGetValue is NULL, thus preventing it from trying to access an invalid memory address and crash the game.

Additional Fixes:

hdtPhysicsExtensions.dll+0xEF30: Crash happens because the game might return a null pointer for an object reference, which HDT PE does not check for; Solution was to execute the code to release this object reference from HDT, which is executed when the object reference is no longer in the expected cell.
hdtPhysicsExtensions.dll+0xD5A5C: Another crash that happens because HDT PE does not check for the possibility of TlsGetValue returning a null pointer; Solution was to skip part of the code that relies on the returned pointer.
hdtPhysicsExtensions.dll+0xC89E8: Another crash that happens because HDT PE does not check for the possibility of TlsGetValue returning a null pointer; Solution was to skip part of the code that relies on the returned pointer.

Installation:

Place hdtPhysicsExtensionsCrashFix.dll in the same location as hdtPhysicsExtensions.dll.

Known Issues:

Other memory addresses relying on TlsGetValue without a sanity check might need to be patched aswell.

I'll need you to report back to me with the crashing addresses.

Submitter

Hawk9969
Submitted

03/30/2020
Category

HDT Physics
Requires

HDT Physics Extensions 14.28
Special Edition Compatible

No

March 30, 2020

P.S. Made an exception for my "no upload to LL" rule because of the severity of the issue.

I am still quite surprised that this race condition affects so few.

27X · March 30, 2020

23 minutes ago, Hawk9969 said:

I am still quite surprised that this race condition affects so few.

Pretty simple.

HDTPE is hella obsolete.

March 30, 2020

43 minutes ago, 27X said:

Pretty simple.

HDTPE is hella obsolete.

If you play as a female, sure.

HDT-SMP sucks if you play as a male wanting physics on NPCs. Only solution for me was running it for naked actors only and it was still quite heavy when I had multiple dead naked females around (clear a bandit's hideout and loot everything).

Running an i7-4770k @ 4.3ghz with an OC'd GTX 1660, can't keep my FPS at 60 in several areas, and that's with PE.

I also see a lot of people still running HDT PE, but I've only found two references of this specific crash, which puzzles me. I mean, it's a race condition, the crash is arbitrary and only happens at the first load, but still, others must have crashed and just shrugged it off.

27X · March 30, 2020

Nope.

Per usual your opinion and facts are not interrelated, and SMP's collision framework is both extensible and flexible. You not doing the research and relying on people desperate to get them patreon dollars with half assed and at this point completely inbred copypasta instead of creating and tailoring controller xmls for what your exact system is (and isn't) capable of is entirely on you.

SMP is as intensive as it's allowed to be, and no more or less.

As for the crash, it's fairly common to 'player unique' and anything else that expands or appends the amount of playable race/character mods, and knowing bethesda's typical level of coding, it's probably caused by a check on expected conditions based on an absolute value that only the vanilla game should return rather than a delta/sum, considering there are at least four patches/mods/fix collections covering this type of shit with havok initialization, actor states at initialization, object placement and reading custom race/npc records in a timely fashion before the engine barfs up its guts respectively.

March 30, 2020

6 hours ago, 27X said:

As for the crash, it's fairly common to 'player unique' and anything else that expands or appends the amount of playable race/character mods, and knowing bethesda's typical level of coding, it's probably caused by a check on expected conditions based on an absolute value that only the vanilla game should return rather than a delta/sum, considering there are at least four patches/mods/fix collections covering this type of shit with havok initialization, actor states at initialization, object placement and reading custom race/npc records in a timely fashion before the engine barfs up its guts respectively.

Can you please read the mod's description before commenting? It has nothing to do with Bethesda; I've described exactly what causes the crash.

The crash happens entirely within the image of hdtPhysicsExtensions.dll, when calling a Windows API function with a defined behavior for returning NULL.

Quote

The data stored in a TLS slot can have a value of 0 because it still has its initial value or because the thread called the TlsSetValue function with 0. Therefore, if the return value is 0, you must check whether GetLastError returns ERROR_SUCCESS before determining that the function has failed. If GetLastError returns ERROR_SUCCESS, then the function has succeeded and the data stored in the TLS slot is 0. Otherwise, the function has failed.

HDT PE simply does not check, much like the reason for most of the vanilla crashes, lack of sanity checks.

hdtPhysicsExtensions.dll:

void hdtPhysicsExtensions_15E2E0(void* param1, void* param2)
{
    DWORD tlsIndex = *((DWORD*)hdtPhysicsExtensions_871D0C); // Always 0x2F in my debugging and crashes.
    eax = TlsGetValue(tlsIndex);
    ecx = *(eax + 0x2C); // Crashes when TlsGetValue returns NULL, regardless of it being an error or just being uninitialized.
  
    ... // Rest of the function's body.
}

hdtPhysicsExtensionsCrashFix.dll:

void hdtPhysicsExtensions_15E2E0(void* param1, void* param2)
{
    DWORD tlsIndex = *((DWORD*)hdtPhysicsExtensions_871D0C); // Always 0x2F in my debugging and crashes.
    eax = TlsGetValue(tlsIndex);

    if (eax == NULL) // We now check EAX's value and skip the rest of the function if it's NULL.
        return;

    ecx = *(eax + 0x2C);
  
    ... // Rest of the function's body.
}

As for the rest, I am not going to bother arguing with someone over here again, a place where I don't even feel at home.

I'll ask that you go argue your PE vs SMP crap elsewhere as this is not the point of this mod/thread. I know you like arguing with people; Please, find someone else, I am not interested.

TheLoverLabCriminal · March 31, 2020

so this will prevent crash from hdt pe right ?

mrsrt · March 31, 2020

Can confirm the problem is actual for some people. However, a big part of players aren't affected by the crash for some reason. Probably, it depends on given data (meshes, physic xmls) which processing postpone the problematic TLS call or it's simply platform dependent.

By skipping a part of the function we may get unpredicted behavior and/or crashes on another TLS call, as you already mentioned. To avoid this we can use some kind of lock instead of return:

void hdtPhysicsExtensions_15E2E0(void* param1, void* param2)
{
    DWORD tlsIndex = *((DWORD*)hdtPhysicsExtensions_871D0C);

    while ((eax = TlsGetValue(tlsIndex)) == NULL) // Wait until value is initialized
    {}

    ecx = *(eax + 0x2C);
  
    ... // Rest of the function's body.
}

P.S. Thanks again for your work and don't bother with scums over here that spread nothing but flood and toxicity.

TheLoverLabCriminal · March 31, 2020

also a question how can we know if we are affected from the crash im using bhunp idk if it will work with it

March 31, 2020

2 hours ago, mrsrt said:
By skipping a part of the function we may get unpredicted behavior and/or crashes on another TLS call, as you already mentioned. To avoid this we can use some kind of lock instead of return:
void hdtPhysicsExtensions_15E2E0(void* param1, void* param2)
{
    DWORD tlsIndex = *((DWORD*)hdtPhysicsExtensions_871D0C);

    while ((eax = TlsGetValue(tlsIndex)) == NULL) // Wait until value is initialized
    {}

    ecx = *(eax + 0x2C);
  
    ... // Rest of the function's body.
}

That was the first thing I'd tried.

Quote

Looping back to the function's call didn't help, as it would now deadlock the game on a null pointer (probably because whatever calls this function has locked shared resources).

I had jz hdtPhysicsExtensions.dll+0x15E2E0 instead of jz skip (ret 8).

It doesn't work, this is the thread that initializes the value.

Quote

Retrieves the value in the calling thread's thread local storage (TLS) slot for the specified TLS index. Each thread of a process has its own slot for each TLS index.

I know there are other TlsGetValue calls within hdtPhysicsExtensions.dll, which is why I am waiting until a crash occurs or someone else gives me the minidump or address of the crash. No point in patching them all if they do not get called when they are uninitialized.

As for unpredicted behavior, I do not think there is any lasting one. I went through the function body entirely, the register values are not used post it and it does not push anything to the stack that is not used within the function itself.

The effect is probably local and refreshed on the next call (seems to be called every frame), which wouldn't have any effect anyway on an uninitialized slot.

I tried looking at the source code, but there is only one reference for TlsGetValue, an inline static member function that is never referenced, in a class that doesn't seem to be used anywhere within the code and having a "We know nothing about this" for its comment. https://github.com/kytulendu/hdt-pe/blob/master/hdtPhysicsExtensions/HookMemory.h#L45

Those TlsGetValue calls are probably added by the Windows API, and without the pdb file, all we've is the machine code and guessing out of experience. I would need the same SDK version, compiler and toolset (v110 according to the project files) to generate the same code.

11 hours ago, TheLoverLabCriminal said:

so this will prevent crash from hdt pe right ?

Only for the crash listed in the description.

I am willing to patch more addresses if it's related to uninitialized TLS values aswell as crashes that do not require me to install any new mods to reproduce them.

March 31, 2020

16 minutes ago, TheLoverLabCriminal said:

also a question how can we know if we are affected from the crash im using bhunp idk if it will work with it

Enable Crash Fixes or Cobb's Bug Fixes crash log.

Crash address should be 0xXXXXE2EC.

Cobb's Bug Fixes will outright tell you the module that caused the crash and its physical address (hdtPhysicsExtensions.dll+0x15E2EC).

You can also load the minidump generated by SKSE (My Games/Skyrim/SKSE/Crashdumps) into Visual Studio and look for the address and exception there.

Exception should be an access violation (generally by accessing 0x0000002C) and address should be 0xXXXXE2EC.

mrsrt · March 31, 2020

2 hours ago, Hawk9969 said:

It doesn't work, this is the thread that initializes the value.

Hm, did you try to place the lock before TlsSetValue call where the accessing value initializes? Unfortunatelly, I cannot experiment with this as I don't encounter the bug and cannot reproduce it.

2 hours ago, Hawk9969 said:

As for unpredicted behavior, I do not think there is any lasting one. I went through the function body entirely, the register values are not used post it and it does not push anything to the stack that is not used within the function itself.

The effect is probably local and refreshed on the next call (seems to be called every frame), which wouldn't have any effect anyway on an uninitialized slot.

Well, nevermind then if it works with no problem. No point in changes.

2 hours ago, Hawk9969 said:

Those TlsGetValue calls are probably added by the Windows API

Looks like a compiler's optimization.

March 31, 2020

53 minutes ago, mrsrt said:

Hm, did you try to place the lock before TlsSetValue call where the accessing value initializes? Unfortunatelly, I cannot experiment with this as I don't encounter the bug and cannot reproduce it.

Lock primitives are pointless here and I don't even think they are used directly. Both TlsSetValue (called by Skyrim) and hdtPhysicsExtensions' code will happen at the same thread.

hdtPhysicsExtensions' code is called by Skyrim from a hook added by its dll. The race condition lies in hdtPhysicsExtensions' code not checking whether its own thread has already set the value or not; Thread-safety is not an issue here.

It's kind of hard to do anything but return on NULL without properly knowing what the function does and the reason for its existence. It seems to be hdtPhysicsExtension' hook into the thread's Havok memory and the crash seems to happen when said memory has not yet been initialized.

April 1, 2020

Could I get some reports on personal results with this fix?

I've tried some heavy exploration with lots of: save -> quit game -> load save, and I have not crashed so far.

This was an arbitrary but consistent crash that I've been experiencing since early 2018 (back when I first finished my initial Skyrim LE setup), sometimes it would crash 5 or more times in a row when trying to load my save after launching the game.

Looks like I won't need to patch any more addresses.

P.S. I guess I've the MSM and incompetent governments to thank for shutting down the world over flu. Finally had the time to sit down and properly debug and test a solution.

steve8240 · April 3, 2020

I think the TlsSetValue problem plagued more people than you think. Technical questions/problems rarely attract help but will for sure will attract negative attention and people for the most part just suck it up rather than address it.

I appreciate the fix and will pay attention to my crash log and will update you to the best of my abilities, I am no programmer so bear with me.

I am showing my support because rarely do you run into people actually want to read your crashdump files. You are a real gem, Thank you.

April 6, 2020

On 4/3/2020 at 1:08 AM, steve8240 said:

I appreciate the fix and will pay attention to my crash log and will update you to the best of my abilities, I am no programmer so bear with me.

I am showing my support because rarely do you run into people actually want to read your crashdump files. You are a real gem, Thank you.

Thanks, but it's no trouble at all; Not only that, but most people aren't knowledgeable enough to find a DLL's physical addresses.

If you were to send me the crash address from Crash Fixes alone, I would've to start guessing because the DLL gets loaded dynamically at runtime at an arbitrary address within the main process.

By sending me Cobb Bug Fixes' crash log or the minidump generated by SKSE, I can either get the module and physical address right away or I can get the module and physical address by checking which module is loaded at crash address.

Cobb Bug Fixes' crash log is the easiest for normal users to check; If you are running that, only send me the log if the crash log says hdtPhysicsExtensions.dll was the source of the crash.

April 13, 2020

Updated to 1.1.0.

Since I haven't had a crash since, this update is merely a restructure of the code as to allow for further and easier patching later on (if needed).

Doesn't really matter if you update it or not, the fix is literally the same.

Leaving this here in case I am unable to patch new addresses myself.

1) Add the segment signature to include/Signatures.h. Must be 5 bytes minimum for the jmp instruction to override.

Use the DEFINE_SIGNATURE macro. Passed address will be detoured with a relative 32-bits jump.

Example:

namespace Plugin::Signatures
{
    ...

    DEFINE_SIGNATURE(0x00ABCDEF)
    {
        0x90, // nop
        0x90, // nop
        0x90, // nop
        0x90, // nop
        0x90  // nop
    };
}

2) Define the detour in include/Fixes.h, adding any necessary overridden instructions from the original address location and fixes.

Use the DEFINE_FIX macro while passing the same address as the one used in DEFINE_SIGNATURE.

If you need to return to the original control flow after executing your code, use the JUMP_BACK_RESERVE macro. The dummy mov instruction and address will be replaced with a jmp instruction and the real dynamic address at runtime. ONLY USE THIS MACRO ONCE!!! Use labels if you need to jump more than once.

Example:

namespace Plugin::Fixes
{
    ...

    DEFINE_FIX(0x00ABCDEF)
    {
        // Original overridden instructions
        // Crash fix
        __asm
        {
            JUMP_BACK_RESERVE
        }
    }
}

3) Add your fix to Plugin's constructor in Plugin.cpp.

Use the CREATE_FIX macro while passing the same address as the one used in DEFINE_SIGNATURE and DEFINE_FIX.

The second argument for the macro should be true if you've used the JUMP_BACK_RESERVE macro within your detour, false otherwise.

Example:

Plugin::Instance::Instance(void)
{
     ...

     CREATE_FIX(0x00ABCDEF, true)
}

4) Recompile.

King-Crimson · April 13, 2020

Thanks for the fix,

I use SMP in Skyrim but PE in Enderal and with this fix it now works without random load crashes and with using the Hair mods that instantly crashed the game before.

So as far s I'm concerned, this works great, considering I brute-forced everything I could into Enderal (including Devious Devices and More Nasty Critters).

@Hawk9969 By the way, where do upload normally? Your general rule is "no LL", so where can one find your other mods? I'm honestly interested in seeing more...

April 14, 2020

12 hours ago, XenonS3 said:

@Hawk9969 By the way, where do upload normally? Your general rule is "no LL", so where can one find your other mods? I'm honestly interested in seeing more...

Nowhere. Doesn't seem like there is a place where I won't have to deal with "copyright" bullshit for mods.

Unless it's something that I consider essential with no other alternative around, I'll keep them for personal use only, since that's the reason for me to mod Skyrim in the first place.

King-Crimson · April 14, 2020

7 hours ago, Hawk9969 said:

Nowhere. Doesn't seem like there is a place where I won't have to deal with "copyright" bullshit for mods.

Unless it's something that I consider essential with no other alternative around, I'll keep them for personal use only, since that's the reason for me to mod Skyrim in the first place.

Yeah, you're probably right to do so.

Nexus is basically a big jerk-off-circle for the big guys and everyone else is oppressed under their stupid view on copyright and even here on LL people are quick to jump at your throat for ... nothing (just look at the first comments on this thread and the general hostility...). Sigh...

Have fun and thanks for sharing these essentials.

decaluka · April 14, 2020

Just wanna say thanks for the fixes and for your mods in general! I use a few and they've always worked just fine!

April 15, 2020

1 hour ago, Tyrygosa said:

Just upload them to google drive or mega or some other repository and make threads about them here. Most of us, regular mod users, could care less about this copyright shit, but QOL and fixes mods are always appreciated.

They will remove the link.

Here is the example that made me give up on LL:

They did exactly that, removed the link. They didn't even send me a message, and I only came to realize it was removed when I had a notification that I was quoted in that thread.

All my published mods in here are my own work; When it comes to improving someone's else work, I just post in their own thread as to not take any credits away from them. Only exception for this was Cumshot Improvements, which the original author requested for me to upload it as a new mod.

I'll not publish new (own) works in here because:

Could contain an asset from someone else. Like my SexLab - Cum Overlays mod that uses textures from another mod (I am not a visual artist myself). I tried contacting the author, he read my message but never replied to my message.
Out of principle. Why support those who try to hinder you at every step (Captcha, copyright stupidity, etc)?

Mind you, I am not throwing a fit and removing all my work, but I've decided it is not worth to support my work in here anymore. If it's something important, like a crash fix, I'll make exceptions to this rule and publish for the sake of contributing back.

My work is licenseless and you can do anything you deem fit with them aswell as publish them anywhere.

I do however feel like a person in China speaking against the CCP everytime I write these posts. I might just vanish someday.

Spyro84 · April 16, 2020

16 hours ago, Hawk9969 said:

Like my SexLab - Cum Overlays mod...

I'm curious about this mod. What is it about? I ask knowing that it's your own personal mod.

April 16, 2020

11 hours ago, Spyro84 said:

I'm curious about this mod. What is it about? I ask knowing that it's your own personal mod.

It uses NiOverride overlays to replace the default cum applying from SexLab.

It allows for better looking effects and placements (mouth, face, chest, vagina and anus vs face/chest, vagina and anus).

It's the same as applying those overlays yourself with SlaveTats, but it's done automatically at your configured orgasm event and does not require SlaveTats a all.

Spoiler

https://imgur.com/a/nNyXiMm


int Function GetSceneCumType(sslBaseAnimation animation)
    int cumType = CUM_NONE

    ; First we check the tags for some more specific types.
    if (animation.HasTag("CumInMouth") || animation.HasTag("EatingCum"))
        cumType = Math.LogicalOr(cumType, CUM_MOUTH)
    elseif (animation.HasTag("Facial"))
        cumType = Math.LogicalOr(cumType, CUM_FACE)
    endif
    if (animation.HasTag("ChestCum"))
        cumType = Math.LogicalOr(cumType, CUM_BREAST)
    endif
    if (animation.HasTag("Creampie") || animation.HasTag("VaginalCum"))
        cumType = Math.LogicalOr(cumType, CUM_VAGINA)
    endif
    if (animation.HasTag("AnalCreampie") || animation.HasTag("AnalCum"))
        cumType = Math.LogicalOr(cumType, CUM_ANUS)
    endif

    return cumType
EndFunction

int Function GetPositionCumType(sslBaseAnimation animation, int stage, int position)
    int cumType = CUM_NONE

    int cumID = animation.GetCumID(position, stage)

    if (cumID < 1 || cumID > 7)
        return cumType
    endif

    if (cumID == 7)
        cumType = CUM_ALL
    else
        if (cumID == 2 || cumID == 4 || cumID == 6) ; Oral, Oral Vaginal, Oral Anal
            if (animation.HasTag("Boobjob") || animation.HasTag("Titjob") || animation.HasTag("Titfuck")) ; Check for boobjob.
                cumType = Math.LogicalOr(cumType, CUM_BREAST)
            else
                cumType = Math.LogicalOr(cumType, CUM_FACE)
            endif
        endif
        if (cumID == 1 || cumID == 4 || cumID == 5) ; Vaginal, Oral Vaginal, Vaginal Anal
            cumType = Math.LogicalOr(cumType, CUM_VAGINA)
        endif
        if (cumID == 3 || cumID == 5 || cumID == 6) ; Anal, Vaginal Anal, Oral Anal
            cumType = Math.LogicalOr(cumType, CUM_ANUS)
        endif
    endif

    return cumType
EndFunction

I've offered it around to some authors (including the author of the textures), but got literally zero replies from them.

jpee1965 · April 16, 2020

Thanks for this HAWK, hewg fan!

Sign In

HDT Physics Extensions Crash Fix

Recommended Posts

Guest