Corrupted Saves

[markdf]

Those _var strings aren't properties, they are autovariables (variables generated automatically to store the contents of properties). They outnumber all other strings too.

 

That patch that reduced the Skyrim basegame string count by 21000? 20000 of that was autovariables.

 

Changing the base game's autovars is not practical, as the error logs show. But it works great on mods. In my tests it reduced Legacy of the Dragonborn from 4000 strings to 600 with no errors.

[migal130]

Those _var strings aren't properties, they are autovariables (variables generated automatically to store the contents of properties). They outnumber all other strings too.

 

Perhaps i'm lacking information about the way the declarations work internally.  I know what my own mod's properties are called (properties declared with "auto" and some not)  I also know that in almost all cases I do not have any vars with that same name.  Yet, I see my property names in the strings table with "_var" appended to the end of the name.  Are you saying my properties are tracked internally as some kind of autovar?

 

I'm not sure that matters for the purposes of testing.  If those strings exist because they are derived from a property, then a property is responsible, as opposed to a var declared in my script.

 

We can say with relative certainty that private declared vars such as those used in MCM scripts reliably show up in a manner similar to properties, correct?  The only difference I've seen between those kinds of vars and properties in my own saves is that an MCM var is always in the table (possibly because the MCM quest is always running), while I'm not sure all properties are at all times.

[zippy57]

Presumably doing it manually would be quite tedious and prone to error, but would it be possible to write some form of TES5Edit script to remove the auto flag from all properties? It may be worth the time to check whether or not there's an actual performance impact related to removing that flag.

[Monoman1]

Might be able to use automation tools but I'm having a hard time finding the "Auto" flag in tesedit on any record. Only flags I see are "Edited" and "Local"...?

[markdf]

Removing the auto flag wouldn't help, because the autovar is already being referenced, particularly in subclasses. Much easier just to track the names of autovars.

[migal130]

My testing suggests aliases are only in the strings table if a property references them.  I have around 250 aliases in one of my mods, but my saves only show around 40 of them, all with "_var" appended, all of which are set as properties in scripts while the other aliases are not.  That means I am seeing the property, not the alias itself.

 

Just to make sure I understand correctly, it is concluded that a single property or var name will not show up inside the table more than once, yes?

[markdf]

First, the _var suffix only appears on variables. That's what it means -- the var is there to show that it's a variable.

Second, yes a string can only appear once. The strings aren't even used internally, data structures just include an index to the string table.

I've disassembled tons of scripts now, and experimented thoroughly. So trust me, anything with a _var suffix is a variable -- and almost all properties have a variable backing them. The auto flag just makes the compiler generate the variable automatically (with _var suffix and a :: prefix).

An auto property is literally just this:
 

Script ExampleScript extends Form

int ::ExampleProperty_var
; This variable stores the value of the property below.
; You cant actually assign names like this yourself, only the compiler can do it.

int Property ExampleProperty
    int Function get()
        return ::ExampleProperty_var
    EndFunction
    Function set(int val)
        ::ExampleProperty_var = val
    EndFunction
EndProperty

Function TestFunction()
    ; Because of the Papyus optimization, this assignment will directly access ::ExampleProperty_var, 
    ; rather than using the get() and set(int) methods.
    int myVar = ExampleProperty
EndFunction

The hard part is that functions in that script (as well as any scripts that extend it) access ::ExampleProperty_var directly. That's the optimization that Bethesda put into papyrus and it's why my patch didn't work. I'm implementing a workaround, but it will only work for mods that do not have any other mods extending their scripts. Like Interesting NPCs, Legacy of the Dragonborn, etc.

[migal130]

First, the _var suffix only appears on variables. That's what it means -- the var is there to show that it's a variable.

 

Second, yes a string can only appear once. The strings aren't even used internally, data structures just include an index to the string table.

 

I've disassembled tons of scripts now, and experimented thoroughly. So trust me, anything with a _var suffix is a variable -- and almost all properties have a variable backing them. The auto flag just makes the compiler generate the variable automatically (with _var suffix and a :: prefix).

 

An auto property is literally just this:

 

Script ExampleScript extends Form

int ::ExampleProperty_var
; This variable stores the value of the property below.
; You cant actually assign names like this yourself, only the compiler can do it.

int Property ExampleProperty
    int Function get()
        return ::ExampleProperty_var
    EndFunction
    Function set(int val)
        ::ExampleProperty_var = val
    EndFunction
EndProperty

Function TestFunction()
    ; Because of the Papyus optimization, this assignment will directly access ::ExampleProperty_var, 
    ; rather than using the get() and set(int) methods.
    int myVar = ExampleProperty
EndFunction

The hard part is that functions in that script (as well as any scripts that extend it) access ::ExampleProperty_var directly. That's the optimization that Bethesda put into papyrus and it's why my patch didn't work. I'm implementing a workaround, but it will only work for mods that do not have any other mods extending their scripts. Like Interesting NPCs, Legacy of the Dragonborn, etc.

 

Thanks, Mark.  You're saying that most properties become variables at compile time and so you're dealing with variables when trying to mitigate the number.  You're also saying that mod candidates for name compression are on a case by case basis, if I understand correctly.

 

I'm wondering if a modder could use the name compression tool a little more surgically, on only scripts the modder selected.

[markdf]

Thanks, Mark. You're saying that most properties become variables at compile time and so you're dealing with variables when trying to mitigate the number. You're also saying that mod candidates for name compression are on a case by case basis, if I understand correctly.

 

I'm wondering if a modder could use the name compression tool a little more surgically, on only scripts the modder selected.

It wont necessarily be about patching a mod at a time. You'll be able to select groups of mods and patch them all at once. For example, patching Campfire would be a mistake because it would probably kill Frostfall and Wearable Lanterns. But patching all three at once would be fine because the changes in Campfire would be propagated to the other two. And the patch goes in its own directory so nothing gets overwritten.

[sidfu]

well altest some mod authors are taking the string issue serously the guy who made the alternate start recently updated his mod and stated he lowered the string count. so some people see the work going on here and know its a issue to keep in mind.

[migal130]

 

It wont necessarily be about patching a mod at a time. You'll be able to select groups of mods and patch them all at once. For example, patching Campfire would be a mistake because it would probably kill Frostfall and Wearable Lanterns. But patching all three at once would be fine because the changes in Campfire would be propagated to the other two. And the patch goes in its own directory so nothing gets overwritten.

 

 

Ah, okay.  Totally makes sense.

[asd3636asd]

I can't seem to get the patch for sexlab to work on a new save game with Jcontainers 3.3 alpha. It just messes up my MCM menu and I notice some performance issues afterwards.

[sidfu]

I can't seem to get the patch for sexlab to work on a new save game with Jcontainers 3.3 alpha. It just messes up my MCM menu and I notice some performance issues afterwards.

 

make sure u usseing the sexlab patch and not the alpha string test he  posted ealier. they are 2 differnt things.

[asd3636asd]

 

I can't seem to get the patch for sexlab to work on a new save game with Jcontainers 3.3 alpha. It just messes up my MCM menu and I notice some performance issues afterwards.

 

make sure u usseing the sexlab patch and not the alpha string test he  posted ealier. they are 2 differnt things.

 

Yep, the one in the opening post.

[markdf]

 

 

I can't seem to get the patch for sexlab to work on a new save game with Jcontainers 3.3 alpha. It just messes up my MCM menu and I notice some performance issues afterwards.

 

make sure u usseing the sexlab patch and not the alpha string test he  posted ealier. they are 2 differnt things.

 

Yep, the one in the opening post.

 

 

Did you install it into a savegame that already has Sexlab installed? It only works for new saves, or existing saves that have never had Sexlab installed.

[markdf]

For those of you interested in the technical side, here's the status of the tool so far.

 

User interface: I set up a JSON file that controls the tool. I'll eventually add an actual GUI, but the JSON file will be good enough for alpha testing (I hope).

 

Function and property renaming: I've removed those features for now, since they're much more complicated than renaming variables and provide very little reduction in the string count.

 

Variable renaming: everything works except for conditional autovars. Conditional autovars can be accessed from the creation kit (getVMQuestVariable and getVMObjectVariable) which makes them a little trickier to rename. They don't seem to get used THAT much, so I may ignore them for now.

 

Here's the data for Legacy of the Dragonborn in my current tests.

• Before installing LoTD:  28400 strings
• After installing vanilla LoTD: 32800 strings (+4400)
• After installing patched LoTD: 29660 strings (+1260)
• After installing patched LoTD with renamed conditional autovars: 29660 (+1260)

 

[asd3636asd]

 

 

 

I can't seem to get the patch for sexlab to work on a new save game with Jcontainers 3.3 alpha. It just messes up my MCM menu and I notice some performance issues afterwards.

 

make sure u usseing the sexlab patch and not the alpha string test he  posted ealier. they are 2 differnt things.

 

Yep, the one in the opening post.

 

 

Did you install it into a savegame that already has Sexlab installed? It only works for new saves, or existing saves that have never had Sexlab installed.

 

Yea, brand new save inside the prison from Live another Life.

[markdf]

 

Yea, brand new save inside the prison from Live another Life.

 

 

Do you have the newest version of sexlab? Maybe post your load order, we can take a look and see if there's anything that might be an issue.

[Monoman1]

Reliability should obviously come first but that's still a ~ 70% reduction in string count. Impressive stuff mark.

[markdf]

Reliability should obviously come first but that's still a ~ 70% reduction in string count. Impressive stuff mark.

 

I agree. When I said "ignore them", I meant that I would just not change them at all. At least not until I have some free time and can implement it properly. My tool always errs on the side of caution.

[markdf]

Here is a pre-alpha prototype test version of "Restringer", for anyone feeling generous enough to test it out.

 

Notes:

• It is a java application, so you need Java installed.
• It is configured using the file "settings.json". It's really really simple, just a list of mod folders and an output folder for the patch.
• Try this with new saves only; if you must test it on an existing save, be sure to patch a mod that has NEVER been installed in the save ever before.
• If you patch a mod, you must patch every mod that depends on it at the same time.
  • Example 1: if you patch Campfire, you must patch Frostfall and Wearable Lanterns at the same time (assuming you have them installed).
  • Example 2: if you patch Sexlab, you must patch every single Sex mod that you have installed, all at once. BTW, I do not recommend trying that.
  • This only applies to mods with scripts. So if you patch Legacy of the Dragonborn, you don't have to patch the LoTD compatibility patches (since they don't have scripts).

If you encounter an error, try doing whatever you did but without the patch. That way you can verify whether or not the patch was actually the problem.

 

 

 

Restringer 2016-04-25.7z

[zippy57]

I was curious to see if I could run the Restringer through MO to get the entire install at once (intent being that I would simply run it once and override with the output, repeating the process every time a mod changed) and the answer is no. The only reason I'm bothering to report it, since I'm not really using it right, it because the program contracits itself: The log window says "Reading 21256 scripts" but then pops up a box that says "No scripts were found. At all."

[markdf]

I was curious to see if I could run the Restringer through MO to get the entire install at once (intent being that I would simply run it once and override with the output, repeating the process every time a mod changed) and the answer is no. The only reason I'm bothering to report it, since I'm not really using it right, it because the program contracits itself: The log window says "Reading 21256 scripts" but then pops up a box that says "No scripts were found. At all."

That's really interesting. Eventually the tool is meant to be run from MO, so that it can scan your entire setup looking for potential problems. But I haven't tested it from MO yet. I hope the JVM doesn't have some incompatibiloty with MO's virtual file system.

[Monoman1]

Shouldn't be a problem as long as it's 32bit java. Skyproc patchers are java applets aren't they? They've always ran fine for me. 

[dman]

Adding my 2 cents here. I have spent the past 3 months working on adding tons of mods. I have been using Merge plugins to merger various mods toghter. It uses the last first order to merge. Since I am merging would that solve the mod dependancy your restringer has?

Also between your patch and Merge plugins a person could in theory, make a modded skyrim that goes way beyond anything we have ever thought of doing.