Corrupted Saves

[Earen]

 

BSFixedString (its SKSE implementation) doesn't release its strings. It might be not an issue, IF the counter doesn't get saved. I remember I was asking the team about this, http://forums.bethsoft.com/topic/1512976-wipz-skyrim-script-extender-skse/page-7?do=findComment&comment=23931061 .  That's why I began use modified one

 

Plain english version: currently SKSE and most of plugins don't release strings, each time  a function implemented by SKSE or some plugin returns new string ( aka BSFixedString("some-new-string") ) the counter gets increased, but it won't be decreased later. Since most of plugins don't introduce new strings it's not an issue (?).  IMO, amount of strings just reflects how much huge your setup is.

 

Actually pretty much all mods that have some scripts inside will introduce new string.

 

Not a big problem, but you can find strings also from mods that are uninstalled.

 

Check with a simple mod thatt add a papyrus script with

String test = "ThIs Is A New String 0192837465"

 

Load the mod, be sure the script is run, save, uninstall the mod.

 

Then do another mod with"

 

String whallabahalla = "this IS a NEW string 0192837465"

 

 

And check the result.

You will get the capitalization of the previous mod.

 

 

Huh, interesting. And your string is simple stack variable? Damn. I hoped they don't save

 

Or maybe they simply reuse the string because they had no time to remove previous one? What if you'd re-saved the save with first mod uninstalled (and the second not installed yet) - imho this would be a bit more pure way of doing it. Btw, Apropos generates tons of new and relatively random strings - save files would explode..

 

> Actually pretty much all mods that have some scripts inside will introduce new string.

 

At least (there is hope) pure Papyrus decreases per-string reference counter properly - it doesn't get decremented by SKSE's BSFixedString destructor

[sheson]

Huh, interesting. And your string is simple stack variable? Damn. I hoped they don't save

 

Or maybe they simply reuse the string because they had no time to remove previous one? What if you'd re-saved the save with first mod uninstalled (and the second not installed yet) - imho this would be a bit more pure way of doing it. Btw, Apropos generates tons of new and relatively random strings - save files would explode..

 

> Actually pretty much all mods that have some scripts inside will introduce new string.

 

At least (there is hope) pure Papyrus decreases per-string reference counter properly - it doesn't get decremented by SKSE's BSFixedString destructor

 

If nothing uses the string anymore they are removed. So "clean save" works.n As far as I can tell, the real source must be gone, e.g. the script.pex - not just the esp/data that uses that script.

 

Every property name is a string. Consider a simple script like this

 

Scriptname AlasseaErevanChatter1 extends ObjectReference  

Actor Property playerref Auto

quest property robbersrefuge3dnpc auto

Scene Property RRefugeScene2 Auto

Event OnTriggerEnter (ObjectReference akActionREF)

  if akactionref == playerref && robbersrefuge3dnpc.getstage() <= 30

    RRefugeScene2.Start()

  endif

endevent

 

All the bold words are "Strings" in this context. Obviously the 3 ObjectReference, playerref, akActionREF are used by many scripts, so they are fine.

[Nepro]

i did not know about cores where do you change the core settings for skyrim ??

 

Task manager -> find TESV.exe -> right click -> set affinity -> choose what cores do you want to use.

 

I recommend to do this before you first load a save game in the main menu. I use OneTweak which makes things easier to access things outside the skyrim window.

[zqzqz]

Thank fuck  I found this thread, I was going CRAZY wondering why my save files were all completely fucked after 5-6 hours of play. Looks like I was running just on the edge of the limit.

 

Are there any other super string heavy mods people have found?

 

Edit: SkyComplete Legendary used: ~1000

This is an MCM menu that tracks quest completion, and uses many strings in the process.

 

 

[Guest emilio]

if i disable some offending mods like interesting npcs, can I lower the string count and resume a playthrough?

[4nk8r]

if i disable some offending mods like interesting npcs, can I lower the string count and resume a playthrough?

 

Yes, but it won't correct any saves that are already damaged.  Based on the discoveries so far, those saves are damaged beyond repair.  You'll need to go back to your last good save and resume from there.  Also, removing/disabling mods mid-game introduces its own issues if you don't go about it "correctly" (and sometimes even if you do).  The phrase "clean save" really doesn't exist for Skyrim.

[Iroha]

markdf,

Looks like culprit of save corruption in heavy-modded skyrim has been found (I mean hard-coded short limit of string count in papyrus section). It will be nice if you will update 1st post with results of discussion and some temporary workarounds (some notice about different threading ini-tweaks that just do nothing good, and can even bring CTD or strange game behavior worth to be marked), and also nice if you will update 1st post in case new info on save corruption problem will pop up.

Anyway, thank you for the thread, I'm calmed a little now.

4nik8tor,  vyper00 thank you too, guys

 

Cheers

[malilie]

I've been having an issue with corrupted saves -- specifically, the issue where the saves are so corrupted that they crash SaveTool and cause Script Scalpel to lock up. I've included an image of Script Scalpel's locked state, showing that it believes there are 1.8 billion script definitions in the file.

 

The issue starts after registering animations; either by enabling creature animations in the Sexlab MCM, or by registering animations in the SLAL Loader MCM. From that point on, any savegame will be corrupted. Unregistering the animations doesn't fix it.

 

As more mods are enabled, it takes fewer and fewer animations registering to trigger the issue. It doesn't seem to be connected to any specific mods, . And it seems specific to mods that have Sexlab.esm as a master; if I unregister every non-sexlab mod, the issue still occurs. But if I unregister enough sexlab mods, the issue goes away. The limit seems to be about 10 sexlab mods, regardless of how heavy the rest of my load order is.

 

It would be wonderful if this issue could be fixed. I've tried to characterize it as best I can and hopefully I've provided adequate supporting information to track down the cause.

 

And yes, I know it's a heavy, heavy load order. But it's completely stable until I start registering animations! Which is weird.

 

 

markdf,

Looks like culprit of save corruption in heavy-modded skyrim has been found (I mean hard-coded short limit of string count in papyrus section). It will be nice if you will update 1st post with results of discussion and some temporary workarounds (some notice about different threading ini-tweaks that just do nothing good, and can even bring CTD or strange game behavior worth to be marked), and also nice if you will update 1st post in case new info on save corruption problem will pop up.

Anyway, thank you for the thread, I'm calmed a little now.

4nik8tor,  vyper00 thank you too, guys

 

Cheers

 

A second from me to please update the OP if you could.  I'd also like to offer my thanks to the people in this thread who've been sharing information!  I had, without knowing about this thread, figured out that my new playthrough (less than two hours in) was running into this problem.  I had even narrowed it down to knowing the save would get corrupted after enabling creature animations in sexlab.  At least I know why and can uninstall some mods and in the meantime keep an eye on possible fixes

 

Sadly all the mods that have been posted as adding lots of strings hit my load order, and I enjoy them all

 

[markdf]

 

Sadly all the mods that have been posted as adding lots of strings hit my load order, and I enjoy them all

 

What I've been doing is just adding in mods temporarily, keeping savegame from before I added it so that I can revert after finishing the mod. It's great for big quest mods like From The Deeps, Forgotten City, Beyond The Reach, etc. Those mods each tend to add between 500 and 1000 strings, so keeping them out of your permanent load order can really help.

 

[malilie]

tldr - I tested how many strings different mods created in a new game save results below

 

Did a little research tonight trying to find out some of the worst offenders for those of us who can't help technically much.  I am working on starting a new build, I was reading through the current thread of the week on /r/skyrimmods and got inspired by some smaller immersion mods I hadn't heard of.  For the test, I used Skyrim Unbound, which has a good spot to save before you even make a character so I used that as my baseline.  Just a note, I have over 300 mods running with a bunch of merges >.>

 

The tests were performed the same way with the same mod load order, minus the ones removed for the testing.  After starting a new game with Skyrim Unbound, I waited looking at Helgen until all the status messages went away, then made a save with Jaxonz Save Namer.  After each test I restored my mod load order from a Mod Organizer backup and removed a different mod for testing.  String amounts were checked using TESVESS

 

Anyways, on to the results!

(This a new game save, nothing touched in MCM, no mods 'activated' in any way)

(Some of these results are not single mods, because of dependencies and merges)

(I even included ones with small numbers or zero just for the curious)

•   328 - Civil War Overhaul
• 3203 - Interesting NPCs
• 3892 - Legacy of the Dragonborn + Another Sorting Mod 2016
• 1158 - Frostfall & Campfire
• 2665 - Ultimate Deadly Encounters (Sands of Time)
•   166 - Genesis Dynamic Spawns
•     28 - Revenge of the Enemies
•     34 - Populated Cities Reborn + Populated Roads reborn + Populated Solstheim
•       0 - Inconsequential NPCs
•   830 - SPERG
•   555 - Skyrim Immersive Creatures
•   786 - iNeed
•   164 - Open Cities
•     93 - Expanded Towns and Cities
•   554 - Complete Alchemy Overhaul + SPERG + Another Sorting Mod 2016
•     99 - OBIS
•     41 - Moonpath to Elswyr
• 1131 - Immersive Armors + OBIS + SPERG
•   104 - Real Names
•   578 - Hunterborn + Bathing in Skyrim + Interactive Wash Basins
•   326 - Arissa the Wandering Rogue
•       0 - When Vampires Attack
•       0 - Run for Your Lives
• 4678 - Lovers Lab Mods (Large Merge) (No Animations Registered) (List in Spoilers)

 

 

• Blush When Aroused
• Estrus Chaurus +
• Estrus Chaurus Spider Addon
• FurtherLoversComfort
• Non SexLab Animation Pack
• Nusbie Voices
• Scent of Sex
• ArmorousAdventurs
• Animations Loader
• Aroused
• Cumshot
• Defeat
• SexLab Framework
• FunnyBiz Animations SLAL
• Monstermod V13 Patch
• Sexualized True Wolves of Skyrim
• Sleeping Necro Assault
• SexLab Tools
• SOS
• SOS for UNP
• ZaZ Animation

 

 

[Monoman1]

Helpful info, and keep it coming but lets not turn this into a mad witch hunt

The absolute worst offender here is bethesda afterall

[Centrophy]

Whelp, 20 hours into a new setup where I removed a lot of the larger mods and the save gets corrupted. 

 

 

 

I would expect the game to CTD if string pointers start to wrap and point at the wrong strings at run time. Evidently the game is fine at run time as far as I can tell from the posts.

 

 

An observation. I think I was wrong when I said it was just the save file that was restricted to the 65.5k (est) string limit. If one takes a look at the papyrus logs in a save at 65.1k there are no real papyrus errors. Once the strings reach that magic number over a few minutes of gameplay the papyrus log starts ballooning in size with "does not exist" and "missing" errors. This will of course mess some things up while playing and ultimately lead to a CTD once the script engine can't take anymore. I currently have a save file at 65.1k if someone needs something close to the limit to test the hypothesis.

 

 

BSFixedString (its SKSE implementation) doesn't release its strings. It might be not an issue, IF the counter doesn't get saved. I remember I was asking the team about this, http://forums.bethsoft.com/topic/1512976-wipz-skyrim-script-extender-skse/page-7?do=findComment&comment=23931061 .  That's why I began use modified one

 

Plain english version: currently SKSE and most of plugins don't release strings, each time  a function implemented by SKSE or some plugin returns new string ( aka BSFixedString("some-new-string") ) the counter gets increased, but it won't be decreased later. Since most of plugins don't introduce new strings it's not an issue (?).  IMO, amount of strings just reflects how much huge your setup is.

 

 

This seems the most relevant piece of info concerning this problem. Besides the fact that most mods do in fact add some strings which was discussed in later posts. The solution is to have SKSE release unneeded strings. Otherwise, I think hacking the game possibly using a SKSE update that changes the stringlimit to 32bit will be the only solution.

 

I guess I'm just going to wait until a fix is out. I'll keep watching this thread. Good luck, everyone.

 

Edit: Removed my frustration fueled tantrum and added some more observations.

[malilie]

Helpful info, and keep it coming but lets not turn this into a mad witch hunt

The absolute worst offender here is bethesda afterall

 

Eeek!  I hope no one thinks I'm trying to start a witch hunt.  I just thought other people might benefit, since I'm putting the work in anyway.  For those of us who have to take a scalpel to our cherished mod list

 

Anyway, here's some more!

 

• 1792 - Anna NPCs
•   670 - Dev Aveza
•   626 - Familiar Faces
•   667 - Falskaar
•   931 - Holidays
•     12 - Ethereal Elven Overhaul
•     35 - Book Covers of Skyrim
•     76 - Gildergreen Regrown
• 1253 - Helgen Reborn
•     79 - Hidden Hideouts
•     86 - A Quality World Map + Colored Map Markers + High Quality 3d Map
•   119 - Immersive Patrols
•   113 - Inns and Taverns
•   109 - No Snow Under the Roof
•     84 - SkyFalls + SkyMills
•     36 - Skyrim Sewers
•   205 - Skyrim Monster Mod
•   145 - Sneak Tools
•   353 - Sofia
•   110 - The Choice is Yours - Improved Dialogue Options
•     29 - Travelers of Skyrim
•     82 - Unlimited Bookshelves
•   309 - Violens
•   347 - Wyrmstooth
•   369 - Moon & Star
•   128 - Huge bunch of merged hair mods
•   114 - TK Children
•     55 - Diverse Guards

[domp]

So did anyone opened a thread about this issue on official bethesda forums,  maybe and just maybe they ll read it and give an answer yes or no (%99 chance to no ) just askin cause  i dont wanna make a double post there

[Centrophy]

So did anyone opened a thread about this issue on official bethesda forums,  maybe and just maybe they ll read it and give an answer yes or no (%99 chance to no ) just askin cause  i dont wanna make a double post there

 

I noticed that someone posted about save corruption in the skse dev thread. No responses to that though.

[Monoman1]

Can you provide a link to that thread please. I found a thread on the step forum started by darklynx but can't find the skse thread.

 

http://forum.step-project.com/topic/9683-i-found-a-new-bug-or-maybe-is-a-not-solvable-old-one-in-the-save-files/

 

He posted in January, seems he figured it out before us.

[markdf]

Can you provide a link to that thread please. I found a thread on the step forum started by darklynx but can't find the skse thread.

 

http://forum.step-project.com/topic/9683-i-found-a-new-bug-or-maybe-is-a-not-solvable-old-one-in-the-save-files/

 

He posted in January, seems he figured it out before us.

 

I have no idea if there's a thread on the SKSE forums, I just emailed the developers directly.

[chipstick]

I had posted a thread linking back to this in the skyrim general forum just to increase awareness of this issue, but it took iroha and malile's posts to remind me that I didn't even post here just to say thank you to everyone for raising and isolating the issue. So...

 

Thank you. To markdf, vyper00 and everyone who's contributed to this. I appreciate it greatly.

[domp]

I had posted a thread linking back to this in the skyrim general forum just to increase awareness of this issue, but it took iroha and malile's posts to remind me that I didn't even post here just to say thank you to everyone for raising and isolating the issue. So...

 

Thank you. To markdf, vyper00 and everyone who's contributed to this. I appreciate it greatly.

ı cant seem to find ur post on bethforums could u provide link plz and thanks for making others aware of this issue everyone we need to make this issue reach the devs of skyrim and maybe reach skyrim youtubers to make it visible more what do you guyz think?

[Monoman1]

Awareness couldn't hurt. If not for a fix for the issue then for modders awareness to try use variables more efficiently or implement better methods. 

[Kimy]

 

 

The biggest difference will be finding ways to reduce numbers of scripts. Interesting NPCs uses 3400 strings before even leaving qasmoke. It has 6800 scripts, most of which have at least one property. There could potentially be HUGE gains to be made there.

 

Like all of those auto-generated dialogue scripts that do nothing except advance a quest stage. Isn't there any other way to do that? Like without a script? I don't mess with the dialogue system much, but I feel like we could get rid of some of those scripts.

 

 

There is no way to do anything in dialogue or a scene without a script, all dialogue will do is dialogue and nothing else and scenes can only do dialogue or perform AI packages. Believe me, if there was a way to not have to do a script in every piece of dialogue or scene where something needs done I would do it as it is a tedious pain in the ass to do.

 

 

That and not even going into the issue that nobody would want to touch a mod the size of Interesting NPCs at this point and basically rewrite the entire thing from scratch. In the end, while this information is certainly helpful to track down why some saves are getting corrupted after a while, there is no way to write a sufficiently large quest or feature mod without using a ton of scripts. And let's be honest, not too many new, large Skyrim modding projects are getting started at this point.

 

In the end, it boils down to the user to make an informed decision about what heavy-weight mods to use. We always knew that we couldn't install a infinite number of performance hungry mods without stuff imploding left and right, and now we can do so more accurately. That's why the information is valuable.

[markdf]

We always knew that we couldn't install a infinite number of performance hungry mods without stuff imploding left and right, and now we can do so more accurately. That's why the information is valuable.

What's so agonizing about this issue is that it's not just performance hungry mods; some of the worst offenders for performance (like real shelter, footprints, headtracking, and wet and cold) add very little to the strings count. That's part of the reason this was such a pain to isolate until Monoman1 finally nailed it down.

[Monoman1]

 

 

*snip*

 

Hmm. You have a point of course and I don't think anyone is blaming mod authors. 

It's not really a performance issue though (my game doesn't grind to a halt), more of a capacity issue. I can be sitting in qasmoke with script latency less than 40ms and still get corrupt saves. 

 

While the information is useful we can not at this point say what a good starting point for a new game is. What sort of a cushion do you need for a good play through? How many strings will be created as the game progresses is completely unknown. How many strings short of the limit can we push it. And how many hours before the 16 bits are expended. It's extremely tricky. Start a new game now and maybe 50 hours in it craps out. So you uninstall more mods start a new game and play for another 50 hours. 

 

I've played very little skyrim since this issue has been identified. 

 

[markdf]

Simplifying MCMs and moving some settings to inis or jsons, storing some data in jcontainers instead of properties, that will probably help.

[markdf]

I've played very little skyrim since this issue has been identified.

Me too, I've taken a break to (finally) play through Oblivion, but I'll be back to Skyrim when Frostfall 3.1 is released. Skyrim is like smoking, you never REALLY quit.