Jump to content

Mini Case Study: ESL Flagging and Legacy of the Dragonborn



Welcome back, and hope you're all having a nice relaxing weekend. Instead of a normal modding diary today, we'll be doing a special mini case study focusing on one objective: cutting down the number of hard load order index reservations in SSE by compacting mods associated with Legacy of the Dragonborn


So why is this an issue? If you're familiar with ESL flagging, you know that often you have to change the form ID structure of a file to apply the ESL flag. That's because ESL-flagged files can only use three bytes for form IDs rather than the normal six. This was a big issue when we only had the CK for compacting files, because it resulted in any patches or other related mods getting their form IDs out of synch. xEdit fixed this by flowing the changes down to the dependent mods, but that only worked if you had them loaded. 


So the good news and the bad news about LOTD: Legacy hooks in to tons of mods. It provides patches, which themselves have ESL flags. The patcher also frequently gets updated, adding support for more mods. This has the unintended side effect of overwriting any patches where you flowed down compacted form IDs, meaning that you end up having to reindex mods you compacted over and over again.


But MO2 has very good asset load order management, so if we put our reindexed patches in a special override area, then they won't get wiped out when the main patcher pushes out something new. That's our project today:


  • Update LOTD to its latest version (and the Curator's Companion)
  • Install all the patches for the supported mods we currently have
  • Compact a bunch of the supported mods, especially the "collectibles" type ones
  • Put those patches in a special holding location so they're safe from accidental overwrites


As an aside, in the future we'll use the same methodology to ensure that Lux and Lux Via achieve their best level of support side-by-side with my various compacted mods, since their deployment strategy is similar (lots of small, targeted patches with a patcher that's frequently updated).


Understanding ESL Flagging


Before we start on that, I want to take this opportunity to talk about ESL flagging. This quote below is from a private conversation I had about this topic, and it's still accurate. Hopefully this will help readers fully understand what's involved with this feature.




So... the information out there is kind of scattered and has a lot of inaccurate or half-true "urban legends" in it. Let me try to break it down.


First, you need to understand the constraints that apply to an ESL flagged file:


  1. The file can't have more than 2048 records. In practice, the limit will be slightly less than this, since there are a number of "hidden" records such as file and section headers that count. In theory, this could be higher - math says there is room for 4096 - but the first 2048 records are reserved for Bethesda use and while you can hack them into normal records they make the CK go bonkers and try to delete your plugin if you load it in that program.
  2. The file has to be the sole source of top-level cell data for any new cells. If other mods also provide top-level cell data, they cell will glitch out.


That's it. So any mod with less than 2048 can be ESL flagged - you just have to watch out for the CELL records. But there are some other considerations.


  1. Voice files and facegen files are indexed by Form ID. This means that if your prospective ESL contains these kind of files, you need to reindex them after you compact the file's form IDs. There are tools that can automate this process, but if you don't do this you'll get blackface bug and/or silent dialogue. Be aware that you need to extract any FID-indexed files from BSA archives before you try to reindex them.
  2. The sequence (SEQ) file contains form ID indices within its structure. If your mod has an SEQ file, then you need to regenerate it after you compact it. You can do this in xEdit through the context menu.
  3. If the mod has patches or other downstream interfaces, you need to have them loaded in xEdit when you compact the main file. This will ensure that the form ID changes flow down into the other files. This is a key reason to never compact files in the CK because it does not flow down changes (among other issues).
  4. From the point of view of your saved game, a compacted plugin is effectively a whole different mod. Not only will its load order change, but all form ID references to it will be off. Making matters worse, SSE may not recognize it as a whole new mod and might not update the full form IDs - just their load order indices. This means that compacting mid-playthrough practically guarantees it'll brick your save. Compact files either when you first install them, or in-between playthroughs.
  5. Scripts that rely on getformfromfile calls likely contain hardcoded references to the canonical version of the mod. This can be complicated because these scripts are associated with 3rd mods, not the mod you are compacting. These scripts need to be recompiled in order to work properly, and you'll need to do an A-B comparison between the canonical file and your compacted version to identify the new form IDs. Unfortunately, I'm not aware of any automated tools to do this yet so this one is a manual operations. Fortunately, you can at least use an external script compiler to reduce your dependency on the CK's somewhat janky script editor.
  6. Some external utilities, like SPID, contain form ID indices in their manifest files. You'll need to update these manually (fortunately, you can do so in a text editor).
  7. Finally, some native code mods (mods with DLL plugins) contain hardcoded form IDs. Unfortunately, you're out of luck with these mods unless the author is willing to publish an ESL flagged version of their own. There's also no good way to analyze the DLL file, so you just have to kind of experiment with these mods if you want to compact them.


There are a couple of workarounds you can use to bypass the technical limitations I described at the top.


  • Cell Edit Rule: You can make a holding file, probably a master file, and change all the top level cell records for your prospective ESL mod into form IDs for that mod. This doesn't save you a load order slot the first time, since your cell holding file must be unflagged, but on the second and subsequent case, then you start getting back slots. Remember also that new cells that don't do anything, such as test cells, or cells that you're sure will never get touched by anything else including things like bashed patches or synthesis patches
  • Record Count Maximum: The only way to get around this is by brute force: you literally break the file into two or more parts. As long as you make sure that none of the upstream parts will depend on the downstream parts, this generally works fine. Of course, all the usual warnings about reindexing form IDs apply here, so it's often easiest to start with "safe" records such as temporary object references and ACHRs. This is how, for example, I got Dunpar Wall into the ESL space even through the mod had something like 3000 records in it.


I think that's everything from soup to nuts. Please let me know if I can drill down on anything or otherwise expand/explain.



With that said, let's get rolling with Legacy.



Updating to the Latest Version


First things first, we want to make sure we're running the latest version of Legacy. That means we need to install the following mods from the Nexus:



These can take a while to download as they are fairly large files, but thanks to the magic of the internet we can move right on to installation. These are well-established, high quality mods and we shouldn't run into any installation problems. However, let's approach them with due diligence and make sure they integrate properly based solely on the edits I've made for previous versions. If not, then we'll need to make the appropriate updates.


Fun fact, because of the compacting dilemma, I only have a small fraction of the LOTD content installed right now. So as we free up space, I expect we'll be back to this screen several times in the future!




Don't forget to manually check the mods on this page if you have them:




Now that we have them all installed, let's go through the files and make sure none of them are going to be problematic. Some of these definitely will - Quaint Raven Rock for example. We'll find these by running an error check and looking for null pointers. That comes back with the following plugins:


  • Immersive Sounds Compendium (parent mod was compacted)
  • Audio Overhaul Skyrim (compacted)
  • Reliquary of Myth (possible bad entry - needs further review)
  • Wyrmstooth (parent mod might be out of date - navmesh issue in patch)
  • Coins of Tamriel (compacted)
  • Books Books Books (compacted)


Now before we do anything wild, let's assess the errors in more detail to see if we actually even need the patch. If we don't, then we'll turn the patch file into an empty shell and put it in my patches override area to act as a blocker against this patch reappearing in the future.


  • ISC and AOS we genuinely need and we will accomplish those by temporarily activating an uncompacted version and then recompacting it with these patches loaded after. That'll reindex their form IDs and then we'll move them to the patch override location.
  • ROM and Wyrmstooth need more investigation. We'll look at those in a moment.
  • BBB is 100% covered by manual patching I already did when I installed that mod, so we will empty it out and turn it into a blocker
  • COT adds new and useful data (LOTD keywords) and so we do want this patch, however, we need to reindex some elements in it to match up with the underlying form IDs. It is a small enough mod that we will not use the ISC/AOS method and instead will just do it manually. We will delete the container edits as they duplicate my own private edits that I made during COT installation.




After doing that, you can see that now the keyword edits are properly overriding the relevant records in COT.




Now let's do AOS and ISC. We'll start by loading only the uncompacted AOS plugin and the DBM AOS plugin. We don't want to accidentally double-compact anything and propagate form IDs in a weird way.




With that done, we'll compact AOS the normal way (right click -> select "compact form IDs for ESL")




Note how afterwards, both AOS and the AOS Patch have edits. This is because xEdit automatically flowed down the changes to the patch.




Now we save only the patch, quit, out and deactivate the uncompacted AOS plugin (which is rigged up as an override for the compacted one in MO2). Then we repeat the process for ISC.





As I do these, we'll move the edited patches into a new "mod" in MO2 that has very high priority (i.e., it's near the bottom of the left pane) specifically for these override patches. This will prevent the official patcher from replacing them next time I run it.




That leaves ROM and Wyrmstooth. The first thing we want to do for both of these is rule out the possibility that the underlying mods have been updated with new content. Surprise, surprise, they have. So we'll update them first and then re-run the error checker (Note that ROM has its own huge patch stack, which we'll revisit at a different time).





The update fixed all the issues in ROM and half of the navmesh warnings for Wyrmstooth. The remaining warning is actually because of another, competing patch, so we'll fix that using normal conflict resolution.




Because neither ROM nor Wyrmstooth's DBM patches required edits, they will not go into the special override section.


Initial Patch Cleanup and Analysis


Now that we've finished up with installation, let's go through our normal cleaning and conflict resolution on these patch files. There are a lot of them, but hopefully the bulk of the conflict resolution is already complete - either as a result of my work to integration those mods or as a product of previous LOTD installations.


As we'd expect for a polished and professional mod like LOTD, the patches had no deletions and only had a couple of ITMs that may have been intentional (we'll still remove them and scrutinize the conflicts on our own rather than trusting an author who puts in an unmarked ITM). Only a single navmesh edge link was broken. Conflict checking reports many records of course, so we'll parse through them and make sure we are happy with what we are getting.


This step took quite a while, but I didn't run into any major problems. The only items I needed to leave unresolved were some navmesh conflicts that if I addressed right now, I'd have to go back and re-do in the future because they tied in with other unfinished work. Now that we know we're working from a clean starting place, let's analyze the LOTD-related files we have installed so far and see which ones are candidates for compacting and ESL flagging. We'll start by pulling the ESL report using the "find plugins which could be turned into ESL" script.




After parsing through the output in Excel, here's what we are looking at.




So right off, we have 8 mods that shouldn't cause us any trouble. For a couple of them we need to make sure we remove the vanilla refs from their DBM patches, since they've been converted into standalone armor mods, but otherwise we're in pretty good shape. Let's start with those.




That's the only one that required editing, though for Vanilla Armor Replacer I had to create a blocker plugin similar to what we did with BBB earlier. At the end of the day, here's what we edited, and here's what we're going to put in the patch override holder.





That squares away the easy pickings. Now let's look at the more complicated mods.



More Complex Targets


Let's look at what's left.


  • Tournament of the Ten Bloods recently received a substantive update to its contents, and it contains many Form ID-indexed assets, so future edits will result in a lot of re-work. We'll skip this one for now. 
  • Cheesemod for Everyone hasn't been updated in quite a while, but it has a bunch of native patches I need to assess before I compact it. We'll skip this mod, too.
  • Reliquary of Myth was updated recently and it has packed assets. We'll skip for now.
  • Fossil Mining, like Cheesemod, is a strong candidate but needs review of its patches first.
  • Skyrim's Unique Treasure has a packed SEQ file, but since there is nothing else to worry about, we'll just generate a new SEQ file and put it in the mod's folder, loose. This one is in.
  • Boethiah for Good Guys has packed assets, but it hasn't been updated in many years. We'll extract the BSA and proceed with flagging this mod.
  • Artifacts of Skyrim is "in" either way, as the mod is moribund due to the author hiding it. However, we do need to see if we can find the older LOTD patch for it so we can get the museum displays.


SUT is the most direct one to deal with. We'll just compact it, flag it, and generate the new SEQ file.




Boethiah for Good Guys will be very similar, but first we'll also run the scripts for Facegen ESLify. After converting the mod, we'll unpack its BSA and then reindex the Facegen. As it turned out, this was a waste of time because the single NPC in the mod didn't undergo Form ID change - his ID number was already in the three byte supported range.




Artifacts of Skyrim was removed from the LOTD patcher with release 2.7, because the author chose to hide the mod in protest of the Nexus exercising the license rights that the author had previously granted them. Fortunately, LOTD patcher 2.6 is available in the archived files, and so before we proceed we'll go in and grab that patch file.  





These files will go into our patch override section, and we'll scrutinize them briefly before we start compacting the underlying mod. Fortunately, if unsurprisingly, the patch was clean and only required some minor attention to a conflict with Lux. 


Since Artifacts does not contain any sensitive files or contested cells, we can proceed directly to compacting and flagging. We don't even need to relocate the patch, since it's already there. We won't be looking at Heir of the Septim's other three formerly-supported mods. They aren't integrated yet and they seem to be largely covered by the Creation Club already (though I might give them a chance later if I'm bored, so I didn't delete them). We will be adding the UCOW patch for this mod to the override folder, since we don't want to have to recompact it when we don't necessarily have a reliable source of the original file available.




And with that, we've completed this little project, regaining 10 load order slots in the process and establishing the infrastructure and procedures for processing future additions as well. 


A Quick Checkup


I do want to look at the Artifacts of Skyrim display and make sure it isn't transposed with some other display. This is always a risk with unsupported LOTD patches, and I'd rather know about it up front rather than get a surprise later. Let's head over to Solitude and visit the Hall of Oddities!


Oh wait, we're going to crash instead.




See the problem? One of the DBM patches has a null baseform in it. Let's investigate further! The issue seems to center on the Perfect Roe display and...








Let's try this again with a non-busted load order.





Much better - and as you can see, the room hasn't been double-booked. 



And there you have it, we've got Legacy of the Dragonborn fully up to date and touched on the technical approach for successfully ESL flagging its supporting mods while, at the same time, protecting against the altered patches getting overwritten by subsequent LOTD Patcher releases. With this squared away, now we're in a good position to start filling in the rest of the LOTD mod list while also shrinking our final load order index. When we started, the final load order index was EF (it grew by one compared to yesterday because of the Wraithguard vault cell patch, which has to be a full-sized plugin). Now, as we wrap up, our final load order index is E5 - a big improvement over what we had before!


Recommended Comments

Excellent writeup, need to save this for reference. I wasn't really aware of the SEQ thingy ... I darkly remember reading something like that, but I certainly would not have recalled if I had come upon such an issue.

Link to comment
18 minutes ago, Talesien said:

Excellent writeup, need to save this for reference. I wasn't really aware of the SEQ thingy ... I darkly remember reading something like that, but I certainly would not have recalled if I had come upon such an issue.


Yeah, failing to regenerate the SEQ file is the root of of the bugs that people sometimes report with NPCs not having their dialogue and/or journal entries for quests not populating until after loading a saved game. Basically, the SEQ file tells Skyrim to prepare the associated quests ahead of time, and encoded within the file are the form IDs for the quests.

Link to comment

Oh geez, that conversation was with me.  I saved it for reference. Helped me immensely a few months ago.  I have ESLified about a hundred mods since I was taught this. 



Edited by EnragedBard
Link to comment

  • Create New...