Jump to content

Recommended Posts

I'm stable at 9.2 mb.

 

If the size of the Papyrus section is the issue, does that mean that animations are being stored in Papyrus's memory space? If so, that may be a bug in Sexlab (or a bug caused by Sexlab's interaction with other mods). Animations have better places to be stored than Papyrus's very small limited memory space.

 

With no additional animations added, the papyrus size of working saves are sitting around 8.3MB with a form count around 21K.  It's still a new game, so I haven't accumulated much data yet.  I don't have any hard evidence that is the case, but that is the part that indicates an overrun when examined with TESV ESS Files Editor.  It could be the form count as well/instead.

 

Would you be willing to examine a few of your corrupt saves with TESV ESS Files editor (both before and after the new ENB/oost ) and post the results?  I've already discarded all my old saves after performing some serious surgery on my mod loadout over the last few weeks, so I don't have any comparison data, but I'll post info about anything new I encounter with the new ENB/oost.

Link to comment

 

I'm stable at 9.2 mb.

 

If the size of the Papyrus section is the issue, does that mean that animations are being stored in Papyrus's memory space? If so, that may be a bug in Sexlab (or a bug caused by Sexlab's interaction with other mods). Animations have better places to be stored than Papyrus's very small limited memory space.

 

With no additional animations added, the papyrus size of working saves are sitting around 8.3MB with a form count around 21K.  It's still a new game, so I haven't accumulated much data yet.  I don't have any hard evidence that is the case, but that is the part that indicates an overrun when examined with TESV ESS Files Editor.  It could be the form count as well/instead.

 

Would you be willing to examine a few of your corrupt saves with TESV ESS Files editor (both before and after the new ENB/oost ) and post the results?  I've already discarded all my old saves after performing some serious surgery on my mod loadout over the last few weeks, so I don't have any comparison data, but I'll post info about anything new I encounter with the new ENB/oost.

 

 

I will.

 

Link to comment

Just found a corruption.

 

Stable at 9.2mb, 67901 forms, papyrus length 9166472, 5 active scripts, 119390 script instances

Corrupted: 9.4mb, 67982 forms (0.1% increase), papyrus length 9343099 (2% increase). Range check error.

 

That difference corresponds to enabling the DynDoLod.esp (the "lightweight" static version that only creates 6 script instances).

Link to comment

Just found a corruption.

 

Stable at 9.2mb, 67901 forms, papyrus length 9166472, 5 active scripts, 119390 script instances

Corrupted: 9.4mb, 67982 forms (0.1% increase), papyrus length 9343099 (2% increase). Range check error.

 

That difference corresponds to enabling the DynDoLod.esp (the "lightweight" static version that only creates 6 script instances).

 

I'm not clear on this. Is DynDoLod tipping the balance into corruption or the other way?

Link to comment

 

 

So, with new ENB/oost and an additional 108 animations via SL Loader, my save is still healthy at 8.6MB and 32.5K forms.

 

Do you have an SSD or a HDD?

 

 

SSD

 

 

Is there some significance to using an SSD or HDD outside of seek/access times?  To be clear, my game and OS are on an SSD, but my saves are stored on a larger HDD.  I just copied the Skyrim folder to the SSD and set up a junction to the new location so Steam still thinks it is on the HDD, though I did create and actively use shortcuts that link directly to the real location on the SSD.

Link to comment

This is cut and pasted. These ini settings have to be balanced according to my calculations by your machine. If you enb you need to balance settings in enblocal. I noticed when I upgraded real vision from an older version the papyrus stress levels for my machine changed. It looks as though your disk speed can change values also SSD drives allowing transfer speeds and allow larger Papyrus memory allocations. The newer super fast memory modules that are coming out will probably have a huge impact on this also. So it looks as though increasing memory size leads to slower framerate and longer load times and visa-versa. This is a limit of the program itself. I imagine that the program is limited in taking advantage of multi processing also for in enb which does requires you to change those settings according to your machine. The solution is getting faster memory,harddrive, processor, and a graphics card capable of running Skyrim at 120-160 frame rate and adjusting fUpdateBudgetMS fPostLoadUpdateTimeMS in order to allow papyrus a larger size.

But you will still hit a limit again at some point. Maybe leaving less used rubbers, tampons, and other skyrim crap laying around and also making sure other thing such as used victims are cleaned up might have effect. Or having a team available to rewrite skyrim programming if you have millions laying around to pay your team. I am no programmer so this is just observation of experience.

 

 If you're having lots of script lag check your papyrus logs. It will let you know if the script system is overloaded and dump a bunch of data to the log file. If it doesn't think the system is overloaded (can happen if you're seeing lag due to deadlocking scripts) you can force a dump using the DPS command in the console.

When in doubt, check the wiki smile.png

Two of the CPU settings (fUpdateBudgetMS, fExtraTaskletBudgetMS) will harm your framerate if the script system actually uses the entire budget. Remember that to run at a solid 30fps the game only has 33ms total to do everything it wants to do that frame. If you want the game to run at 60fps, the game only gets 16ms. (And so on for higher framerates)

So you can see that if you tell Papyrus that it has 20ms for script updates that (if it uses all 20ms) you can never hit 60fps, and it's going to be hard to even hit 30 (the game will only have 13ms left over to do everything else).

The default of 1.2 should be fine for most people, even with modded setups. If you do increase it, keep it to small values or don't be surprised if your game framerate drops or hitches during periods of high script activity. (And if it's a combat mod, that would be during combat!)

The last CPU-related setting (fPostLoadUpdateTimeMS) will not affect your framerate, but will affect your load times. Unlike the other settings, Papyrus will use the entire time even if it has nothing to do. Increasing this will only help make sure cells and such are set up if there is a load screen. For example, the civil war can stress this because it has to set up a ton of things in the cities for the war when you enter through their load doors.

Papyrus (Skyrim.ini)
First section is the most important. Papyrus, which is the scripting system Skyrim uses, is very fragile. The most popular tweak for it will overload it, and can hurt performance, and overwork the CPU.

Replace anything you have here (in Skyrim.ini) with the default settings (vanilla master Skyrim.ini uses these numbers):
[Papyrus]
fUpdateBudgetMS=1.2
fExtraTaskletBudgetMS=1.2
fPostLoadUpdateTimeMS=500.0
iMinMemoryPageSize=128
iMaxMemoryPageSize=512
iMaxAllocatedMemoryBytes=76800
bEnableLogging=0
bEnableTrace=0
bLoadDebugInformation=0
bEnableProfiling=0

Notice that the MS values is ~660x smaller than what some recommend (800)?  To put it simply, 60fps requires around 16.6ms max to be spent on rendering each frame. Allowing up to 800ms of CPU time to one task is not a good idea, it needs to share CPU time with other things. Also notice the small memory values. Setting these larger is not a good idea, just read the link below to know why.

If you are running a script-heavy setup (you shouldn't be), and your scripts start to misbehave/perform badly, you could increase the fUpdateBudgetMS value in increments (to 1.6, then 2.0 at max). This will help prevent the additional time in fExtraTaskletBudgetMS being allocated to the CPU, instead adding a smaller amount to additionally always be allocated to the CPU. This will ensure all scripts have sufficient time to run.

Explanation of these settings is here: http://www.creationkit.com/INI_Settings_(Papyrus)

Bethesda Developer confirming this:
http://forums.bethsoft.com/topic/1487930-getting-a-lot-of-script-lag-going-over-10000-ms-sometimes/?p=23340131

Link to comment

Don't think the issue can be attributed to script lag or disk latency, at least not directly. I've got very little script lag and still get the corruption problem. This can happen while sitting in the alternate start cell at 60fps solid and script lag usually well below 50ms. I'm using one of these for skyrim: http://www.newegg.com/Product/Product.aspx?Item=N82E16820147467

5qslUvC.png

 

It's currently limited too by the PCIe 2.0 slot I have it in but don't have any spare 3.0 slots. Saves dumped on a seperate SDD. And I still have this problem. 

 

Edit: When people are trying out the new enb boost are you disabling the threading tweaks in Skyrim.ini / Prefs? Boris seems to be very much against them. 

Link to comment

Edit: When people are trying out the new enb boost are you disabling the threading tweaks in Skyrim.ini / Prefs? Boris seems to be very much against them. 

 

I've never used the threading tweaks. They seem to cause nothing but problems, and it seems like they are just placebos. Any improvement in performance is purely in the user's mind.

 

Link to comment

I'm not using MO and I have the same save file corruption.

 

Yeah, this is definitely an issue with the Skyrim engine itself. It seems like there's no real solution, unless the SKSE team finds something that they can patch or Bethesda releases the Skyrim source code.

Link to comment

Yes and whenever I adjust fUpdateBudgetMS it delays or increases when the corruption occurs so my post +59 applies directly. I seem to have found a balance on a setting of 1.5 that allows play to occur for longer periods without corruption. If you read my post you should have it set between 1.6 and 2.0 for heavily scripted machines not to go over 2.0. I am upgrading my video card to a geforce 970 in about 2 weeks it will be interesting to see what effect it has on duration before corruption occurs for I dont think it will get rid of it totally. With it set at 1.5 it seems to mostly occur after heavy creature spawns and larger fights.

Link to comment

Just on the save file papyrus size. I have an old save here. Papyrus size: 14288.66kB, changeformcount: 122307. And not corrupt. Papyrus size there is pretty large. 

That's a 23MB save. I remember having saves around 35MB on my first play through of Skyrim years ago that never had this issue. No idea what the papyrus size was then though. 

Is there any way to tell the impact a particular mod has on Papyrus size? Like remove a mod, clean the save and check the difference? 

If we had a way to quantify how much a mod would add or find out exactly at what point a save breaks we might at least be able to avoid the problem.

Link to comment

I noticed my Papyrus size changes mostly from active scripts. It raises or lowers accordingly by a large amount. Specifically when I fast travel to a new area or a big battle. Early on play when Skyrim first started would not have included all the storage utilities we now have and load times for additional mods so as allow larger papyrus sizes that were not doing as much. Simpler scripting. Scripts now monitor everything  and I would not be surprised to see a script come out to monitor cow fart impacts on Skyrim atmosphere sometime in the near future. :D 

Link to comment

Are you guys using the SKSE.ini fixes?

 

[General]
ClearInvalidRegistrations=1
[Display]
iTintTextureResolution=2048
[Memory]
defaultHeapInitialAllocMB=768
scrapHeapSizeMB=256
 
Also have tried the start a new game then try and load your saved game work around?
Link to comment

 

Are you guys using the SKSE.ini fixes?

 

[General]
ClearInvalidRegistrations=1
[Display]
iTintTextureResolution=2048
[Memory]
defaultHeapInitialAllocMB=768
scrapHeapSizeMB=256
 
Also have tried the start a new game then try and load your saved game work around?

 

Yes. This is a very different issue. These aren't just saves that can't be loaded. The papyrus section of the save files is completely corrupted. These save files wont even load in Script Scalpel (which locks up) or SaveTool (which crashes almost instantly).

 

Link to comment

Uninstalling More Nasty Creatures seems to have fixed this for me. I've enabled creature animations, I've registered every single SLAL animation that I could find, I even registered the Zaz animations for good measure. So far so good.

 

These gains are sometimes temporary, so I'll followup later if it keeps working.

Link to comment

I have played with this corruption problem since November. I believe its the number of scripts saved to the save file. If I uninstall a heavily scripted complex mod, it allows more animations to be registered. I run an extremely complex\heavily scripted load order. Last count I had over 350 Mods merged down to 255. The game runs pretty solid except for the occasional Skyrim random "time for me to crash". I Think we have just hit the limitations of papyrus. Using script scalpel, if stored script definitions exceeds 8200 or so my saves become corrupted.

Link to comment

I have had this problem on a number of saves now sadly, but I think I have found the reason why these saves get corrupted,
so I'm sharing my findings here.
Now I'm no expert on this stuff, but I hope I can explain:

It has to do with how Papyrus Data is saved in the .ess file. I had a look at one of my broken saves with TESVESSE
( http://www.nexusmods.com/skyrim/mods/27119/? ), since it's the only tool that doesn't immediately crash.

On a working save you can see the number of saved Papyrus Strings there when looking at the Papyrus data, which for my working save
was at 65499. Now, why is this interesting...

Looking at how the .ess file is laid out ( see http://www.uesp.net/wiki/Tes5Mod:Save_File_Format/Papyrus ),
the first thing after the header is strCount and strings. Since strCount is saved as "uint16", the maximum possible value is 65535,
which in the save file itself would be saved as hex value right at the beginning after the header, for me it's like

header: 04 00,
then strCount: DC FF .
You can see this in TESVESSE.

The problem is, what happens after it reaches FF FF. I looked in a corrupt file and it says 19 00, which would mean a strCount of 25.

So I guess Skyrim just decides to start counting again from 00 00 after FF FF. Therefore, since in reality there is a
lot more Strings there, but the file only counts 25, the game can't read the save correctly, same goes for the usual save editors.

So I don't think it's fixable, unfortunately, it's just another limitation of how Skyrim stores save data...
I guess at least you can look how close to the limit you are and try to stay below.

At the moment I'm trying to see which mods add the most strings, so I can hopefully prevent this issue from occuring again.

Man, sorry for the wall of text, hope this is helpful at least.

Link to comment

I have had this problem on a number of saves now sadly, but I think I have found the reason why these saves get corrupted,

so I'm sharing my findings here.

Now I'm no expert on this stuff, but I hope I can explain:

 

It has to do with how Papyrus Data is saved in the .ess file. I had a look at one of my broken saves with TESVESSE

( http://www.nexusmods.com/skyrim/mods/27119/? ), since it's the only tool that doesn't immediately crash.

 

On a working save you can see the number of saved Papyrus Strings there when looking at the Papyrus data, which for my working save

was at 65499. Now, why is this interesting...

 

Looking at how the .ess file is laid out ( see http://www.uesp.net/wiki/Tes5Mod:Save_File_Format/Papyrus ),

the first thing after the header is strCount and strings. Since strCount is saved as "uint16", the maximum possible value is 65535,

which in the save file itself would be saved as hex value right at the beginning after the header, for me it's like

header: 04 00,

then strCount: DC FF .

You can see this in TESVESSE.

 

The problem is, what happens after it reaches FF FF. I looked in a corrupt file and it says 19 00, which would mean a strCount of 25.

So I guess Skyrim just decides to start counting again from 00 00 after FF FF. Therefore, since in reality there is a

lot more Strings there, but the file only counts 25, the game can't read the save correctly, same goes for the usual save editors.

 

So I don't think it's fixable, unfortunately, it's just another limitation of how Skyrim stores save data...

I guess at least you can look how close to the limit you are and try to stay below.

 

At the moment I'm trying to see which mods add the most strings, so I can hopefully prevent this issue from occuring again.

 

Man, sorry for the wall of text, hope this is helpful at least.

 

Thanks for posting vyper and thanks for the link to that tool. I was not aware of it's existence. 

 

Very interesting. You could be on to something there. On a bad save, when I expand "Global Data table 3" and click on papyrus I get a popup error: "Range Check Error". 

 

Papyrus section of a good save from around the same time lists a lot of animations down near the end for me. Each stage of an animation has an entry which would explain why the number animations seem to have a heavy impact on this issue. Could some clever person identify what the section is defining. They look like string variables to me. Then the question is can we move animations from using string variables to something else...?

 

I'm assuming increasing the limit from FF FF would be more than difficult. 

 

And one more question. Why would games gradually go bad? Is the number of string definitions not static if you're not changing your load order

 

Edit: Actually they look like all variable types. Not good. So it looks like the maximum number of variable definitions are being exceeded. I'll be looking through this section and trying to identify mods adding more than their fair share of variables. It seems MCMs are another big hitter due to each entry for an MCM option requires a lot of variable definitions. 

Link to comment

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. For more information, see our Privacy Policy & Terms of Use