Jump to content

Help test fix for save game string table overflow


Recommended Posts

Hi, I added a potential fix for this problem in my next version of Crash Fixes plugin.

 

Simple explanation is like this: if the count would go over 65520 then instead write 65535 as count and then write the real count again as 32 bit value. During loading it does the same, if the count read is 65535 then it reads real count again as 32 bit value. Same for the pointers that point to string entries in the table. Obviously this means that the save file format is no longer vanilla IF your string buffer is over 65520 and you save game with this setting enabled. In that case save file tools and vanilla game will no longer be able to load the save game and you must use this setting to be able to access that save.

 

I made only short tests but it seems to be working, with a string count of 133733 I can save game, reload and it's 133733 and the strings didn't get messed up in any way.

 

There's a plugin that helps you overflow the string count and later see if they did end up being the same (enable papyrus logs):

http://www.nexusmods.com/skyrim/mods/74390/?

 

This is a debug build of my plugin so it writes to CrashFixPlugin.txt in Skyrim root folder the following:

01 May 15:39:06 Read from storage #1: 34527
01 May 15:39:21 Write to storage #1: 133733
01 May 15:39:21 Write to storage #2: 133733
01 May 15:39:29 Read from storage #1: 65535
01 May 15:39:29 Read from storage #2: 133733
01 May 15:39:39 Write to storage #1: 133671
01 May 15:39:39 Write to storage #2: 133671
Here I loaded game without having the overflow active in that save game. Then saved game, it writes new string count 133733, then it writes it second time because it is over 65520. Next I reloaded the save so it reads 65535 and then reads again the second value. Then I saved again.

 

Don't test on valuable saves or make backups before!

 

Please post your results here, if it works that's helpful information too.

 

Edit: forgot to mention it doesn't fix the problem on saves that are already broken!

Crash Fixes v8 BETA1.zip

Link to comment

Excellent work.

 

I can reproduce with ease this problem with just 20 mods, where one is done by me just to test strings.

 

Link to comment

Just did a quick test there. Installed 2 versions of the string breaker mod, 40,000 script properties + 99,000 string variables + base game strings = around 170,000 strings. 

Saves loaded fine. I am impressed!

Will test with more conventional mods when I get a chance.

Link to comment

OK, did a test with a test mod that was adding 65536 strings. Including the ones from the other mods and the vanilla the total number was more than 100,000.

 

The mod works fine, no problems in saving and loading, and after loading (so the init of the strings is already done) then strings keep their values.

 

For me it is working.

Link to comment

Hmm I hope it doesn't have side effects, had to patch game code at around 30 places so might have missed something or otherwise screwed up. :D

 

Another fun thing: removed the corrupter mod, tested loading save with 133671 strings, saved again and 34605 strings now. After it was 34605 the format goes back to vanilla so the save game can be opened again without this plugin or with save tools.

Link to comment

I only did a "mechanical" testing.

By having a few quest scripts and alias scripts with all of them creating (using a macro in NP++) a bunch of strings with a pre-configured content.

Never the same content in two strings.

 

All string creation was on the "OnInit()", with some utility.wait(Utility.randomFloat(3.0, 30.0)) just to avoid clogging in the initialization.

 

Then a test done by hooking a key, that tests the value of the strings and properties.

 

All strings seems to be keeping the correct values inside.

 

Link to comment

I'll test this when I'm on my Skyrim machine, but in terms of actually using it I tweak mods too often and need to be able to open the save in a save editor, so this unfortunately isn't a solution I can take advantage of (until someone makes a save editor that can read the change, I suppose).

Link to comment

I'll test this when I'm on my Skyrim machine, but in terms of actually using it I tweak mods too often and need to be able to open the save in a save editor, so this unfortunately isn't a solution I can take advantage of (until someone makes a save editor that can read the change, I suppose).

Is Savetool open source? It seems like a pretty minor change to the file format, probably easy to patch Savetool. I don't have access to a PC today so I can't check it out myself.
Link to comment

I'll test this when I'm on my Skyrim machine, but in terms of actually using it I tweak mods too often and need to be able to open the save in a save editor, so this unfortunately isn't a solution I can take advantage of (until someone makes a save editor that can read the change, I suppose).

 

yeah i use the hell outta the Save Game Cleaner and Pap Transfer programs too. so loosing that would suck but i bet it wouldn't be long until a cleaner that would work, would be made.

 

edit: well he said...

"Another fun thing: removed the corrupter mod, tested loading save with 133671 strings, saved again and 34605 strings now. After it was 34605 the format goes back to vanilla so the save game can be opened again without this plugin or with save tools."

==================

 

so maybe saves made w/ this CAN be opened w/ cleaners?

Link to comment

Cleaners will only fail to open it if my mod saved in a custom format, and it only saves in a custom format if the string count goes over 65520. Note: save tools will fail to open the save game ANYWAY if the string count goes over, whether or not you use this plugin's setting. Only difference is that the save will be useless completely instead of still being able to play it. :)

Quick question: how does one use this?  Load it in MO, start a new game, and big string saves now work?

Yes, don't even need to start a new game. This is my regular crash fix plugin (http://www.nexusmods.com/skyrim/mods/72725/?) with added feature of allowing big string save games + debugging so you can see string counts in your save games. You can configure this in the INI file StringCount32=1, it's experimental so I posted here for testing. If people say it works ok then I release it normally as next version of crash fix mod.

Link to comment

Cleaners will only fail to open it if my mod saved in a custom format, and it only saves in a custom format if the string count goes over 65520. Note: save tools will fail to open the save game ANYWAY if the string count goes over, whether or not you use this plugin's setting. Only difference is that the save will be useless completely instead of still being able to play it. :)

Quick question: how does one use this?  Load it in MO, start a new game, and big string saves now work?

Yes, don't even need to start a new game. This is my regular crash fix plugin (http://www.nexusmods.com/skyrim/mods/72725/?) with added feature of allowing big string save games + debugging so you can see string counts in your save games. You can configure this in the INI file StringCount32=1, it's experimental so I posted here for testing. If people say it works ok then I release it normally as next version of crash fix mod.

 

 

You... you might just be my favorite :)

Link to comment

Wow.  As a fly-on-the-wall end user, I've been lurking in markdf's thread just amazed at the whole process.  I admit I was feeling a bit down when I read that Sexlab (easily the prime culprit) was not going to be re-written any time soon, but I have to say I had no idea you were about to come galloping in to the rescue!

 

My current level of testing expertise is limited to crossing my fingers while waiting to see if my hopelessly cross-wired modlist pops like a soap bubble on load (something that happens less often (never?) since Crash Fixes), so I'll have to sit out while the advanced crew stress tests the fix.  Here's to looking forward to v8!

Link to comment

Right now @Markd is proposing a tool to reduce the strings (I did not checked it yet.) and @h38fh2mf is proposing a more radical solution, to just bypass the save game limit of strings.

The string number seems to be not a problem in the game. But it is a problem when saving/loading.

 

I did a single test (pretty extensive) on @h38fh2mf solution, and it seems to work just fine.

Link to comment

I added a link to this in the String Table Bug thread. Hopefully that will bring in some more testers. :-)

 

I'm really surprised that this approach works, but I'm glad it does. I stand by my patcher, but this is a great solution too -- and probably more useful for anyone except hardcore mod enthusiasts.

Link to comment

 

I'll test this when I'm on my Skyrim machine, but in terms of actually using it I tweak mods too often and need to be able to open the save in a save editor, so this unfortunately isn't a solution I can take advantage of (until someone makes a save editor that can read the change, I suppose).

Is Savetool open source? It seems like a pretty minor change to the file format, probably easy to patch Savetool. I don't have access to a PC today so I can't check it out myself.

 

This appears to be open source.
Link to comment

 

 

I'll test this when I'm on my Skyrim machine, but in terms of actually using it I tweak mods too often and need to be able to open the save in a save editor, so this unfortunately isn't a solution I can take advantage of (until someone makes a save editor that can read the change, I suppose).

Is Savetool open source? It seems like a pretty minor change to the file format, probably easy to patch Savetool. I don't have access to a PC today so I can't check it out myself.

 

This appears to be open source.

 

 

oh Lordy yes! this'd would be a life saver for people with in progress playthrus. many of us use the Save Cleaners all the time, if even just to check stuff

 

i see this 32=1 method as a boon for ease of use and in-progress games (the string bug is so insidious because it's not known!), and the Restringer method for properly "building" your game

 

Link to comment

Great work. Been adding a bunch of new mods to my load order, and I can actually load a save game with sl and all my animations and all new mods. Some good news: the tool in MO that tells you some info about your saves(playtime, plugins, etc.) seems to still work with the extended string function. So at least not all save tools are broken by it.

Link to comment

OK, did a test with a test mod that was adding 65536 strings. Including the ones from the other mods and the vanilla the total number was more than 100,000.

 

The mod works fine, no problems in saving and loading, and after loading (so the init of the strings is already done) then strings keep their values.

 

For me it is working.

 

Do you get any Papyrus log errors? In the original String Table Bug thread, a few people reported that once they reached the string limit, they started getting a tremendous number of Papyrus errors while they were playing, and that the papyrus spam coincided with producing corrupted saves.

 

My own experience has been that nothing unusual shows up in Papyrus when the limit is reached, but I'm curious what you've found.

Link to comment

 

Do you get any Papyrus log errors? In the original String Table Bug thread, a few people reported that once they reached the string limit, they started getting a tremendous number of Papyrus errors while they were playing, and that the papyrus spam coincided with producing corrupted saves.

 

My own experience has been that nothing unusual shows up in Papyrus when the limit is reached, but I'm curious what you've found.

 

 

Doing my test (a really specific one with a mod create just for testing this) I have pretty much no papyrus errors (the usual ones from my load order), nothing special.

Link to comment

 

 

Do you get any Papyrus log errors? In the original String Table Bug thread, a few people reported that once they reached the string limit, they started getting a tremendous number of Papyrus errors while they were playing, and that the papyrus spam coincided with producing corrupted saves.

 

My own experience has been that nothing unusual shows up in Papyrus when the limit is reached, but I'm curious what you've found.

 

Doing my test (a really specific one with a mod create just for testing this) I have pretty much no papyrus errors (the usual ones from my load order), nothing special.

 

Any difference in script latency from having a much larger string table?

Link to comment

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. For more information, see our Privacy Policy & Terms of Use