Author of mod Thief needs help regarding Objects ID

[Highborn]

So, Dogma, the author of the mod Thief, has made a huge post today asking for help or advice regarding Object IDs. If anyone knows a way to help him, please post here, and I'll send to him.

 

I'll post what he said, google translated:

 

_____________________

 

"I wrote to you that there is one thing that worries me. There is a lot of text, so you can not waste time on this, it is not required reading.

 

 

 

It's about ID. As you all know very well, in Skyrim, objects use IDs.

 

A bit of theory for those who are not familiar with this - ID is a unique number that is assigned by the program to any created object. It can be an actor, a dialogue line, a quest, and even a triangle on the navigation mesh - everything has its own ID. It consists of 6 characters, to which, during the game, two more characters are added from the plug-in loading serial number so that the mods do not conflict with each other and the main game (that's why you need to use the XX format for entering codes in the console --- ---, where xx is the sequence number of the plugin from which you want to perform some manipulations with a certain item). The ID uses a hexadecimal number system, that is, from 0 to F (0,1,2...9ABCDEF).

For example, - since only two characters are allocated for the plug-in loading serial number, then the maximum number of plug-ins that can be connected is FF - if translated into the decimal system we are used to, then this is 256 with zero (0 is skyrim.esm), that is, this is the maximum number of plugins that can be connected to the game (although I have never checked this personally, but the Internet confirms that this is the case, trust him).
In the editor, 6 characters are allocated for ID, that is, the maximum number of IDs, respectively, is FFFFFF, which is 16,777,216 in decimal. Seems to be more than enough, but not really... When you create a new plugin, the first object you create (a simple object like globalvalue, or keyword, which consists of only one element under which only one ID is allocated), then it will have ID - 000D62, next 000D63 and so on. (the fact that it starts immediately with 000D62 and not with 000001 is obviously due to the fact that when creating a plugin, a certain number of IDs are allocated in advance for some important elements, but this does not change the essence, all further objects will have +1, except for complex elements that take multiple IDs)

 

Why am I doing this - at the moment, the last object that I created has ID = C39445, so I have already used up 12,817,477 million IDs out of 16,777,216 possible. And even so, one could say that ~4 million should be enough with a margin, given that only the second part remains to be done, and the final version. Maybe, maybe not. They are spent very quickly, and it depends on many factors, for example, when I started working on the first part of Silentwood, I was at point 82 **** - which is about 8,500,000, that is, for the first part I spent about 4 million ID. Why so much? I also asked myself this question the first thing after I noticed this fact - mainly, of course, because of the location, and probably most of it because of the navmesh, which I deleted many times and sometimes remade 10 times. And if you added an item and then deleted it, then the ID was not restored, the ID counter for new objects only moves forward. I have no idea what will happen when all the IDs run out, and I don’t know if there is some kind of mechanism that will re-enumerate the free ones. Maybe you know something about this? Actually, for this reason, I decided to write this text. Maybe some of you know or can find information, because I personally could not dig anything sensible.

 

Based on all of the above, we can conclude that a very serious problem is approaching us. And as many of you have probably already guessed, the solution to which could be to create a separate ESP, for example 'Dogma - Thief (Final).esp' in which I would continue to work by simply connecting Dogma - Thief.esp kind of like we connect ESM, or by converting the original ESP to ESM. But, firstly, I have never done this, I don’t know what problems this can lead to, I already have a lot of problems and errors all the time and with one plugin, and here another additional factor is connected to this. Secondly, I don't think it's a good idea to do this when I have an unfinished part and everything is sorted around, I need to at least complete the second part, but at the same time I can't finish all the IDs (although in this case one could say, that I passed the Creation Kit :)), but who knows how the game will behave later, and that if I suddenly need to change something in the original plugin, I need to have a margin, at least stop at no more than F00000. Of course, I am no longer going to do such large-scale construction projects as in the first part, but I am not sure that I will meet this value.

 

There is another issue that worries me, it appeared about the middle of the work on the last update, or maybe earlier, and I think it is also related to the topic under discussion. When I'm working on dialogs, at some point the dialog box becomes inactive and I can't add new topics. This is solved by rebooting the CK. I'm worried that at some point it won't turn out at all that I won't be able to add new dialogs even after a reboot, because this problem is starting to manifest itself more and more often. If at the moment when it appeared, it could happen once a day, or even less often, with active work with dialogs, now it happens literally after adding 10-20 lines, and you need to restart the CK, and if you add here the fact that after loading the program in 50% of cases, when adding a script, the program crashes, then these are extra spokes in the wheels, which do not contribute to productive work in any way, and constantly slow down. In general, as you can see, the further - the more problems.

 

The above factors are probably the most important reasons why I want to finish work on The Thief as soon as possible and move on to a new project. Start from scratch, at least so that the program does not crash due to the large number of scripts in the folder, which it checks every time it starts, and so that everything works faster, because the more objects in the mod, the longer all windows inside the program open, and so that the plugin does not save for 20-30 seconds as it is now, which seems to be not much, but a lot of time is spent at a distance, this is especially felt when you need to find an error and you often have to re-save and restart the game. I also want Notepad++ to not open for 30+ minutes after restarting the computer (I'm not joking, it really opens for about half an hour, or even more, due to the number of scripts, but it's convenient for me to keep them all open at once, because I often have to look for functions and variables in all files, you can search there in a folder, but it searches for a very long time, and it is not possible to use it, in the same VS Code the search in the working folder is instantaneous, and there is no need to keep all files open, but I don’t I like to work there for various reasons, and I'm already used to Notepad ++, I set it up for myself for a long time, and the syntax and my functions with hints).

 

But, of course, I won't start a new project until I've added everything I want to The Thief and finished it completely, you don't have to worry about that. But when I do start a new project, I will take into account all my mistakes from the first one, of course I have no plans for another such huge project, but to be honest, even when I started this one I could not even think that I would be doing it more than 6- you years. So it’s better to play it safe, and at least I’ll do the same locations in a separate plugin, and then you won’t have to worry about the ID, and the plugin will save faster if there is no extra data about locations and navmeshes."

[MadMansGun]

1 hour ago, Highborn said:

For example, - since only two characters are allocated for the plug-in loading serial number, then the maximum number of plug-ins that can be connected is FF - if translated into the decimal system we are used to, then this is 256 with zero (0 is skyrim.esm), that is, this is the maximum number of plugins that can be connected to the game (although I have never checked this personally, but the Internet confirms that this is the case, trust him).

the max is 255, FF is reserved for in game generated objects.

 

1 hour ago, Highborn said:

(the fact that it starts immediately with 000D62 and not with 000001 is obviously due to the fact that when creating a plugin, a certain number of IDs are allocated in advance for some important elements, but this does not change the essence, all further objects will have +1, except for complex elements that take multiple IDs)

mine starts at 800

1 hour ago, Highborn said:

the ID counter for new objects only moves forward. I have no idea what will happen when all the IDs run out

who the hell would? that's a stupidly large mod to use up that many ids

[ghastley]

I believe the CK uses different ranges of IDs for different things, so there are multiple “next available” values being tracked. The high water mark for one type being in the x’C3nnnn’ range does not mean there are no large gaps lower down, so the total used may be nothing like that high.

 

Esl’s have the top two bytes as load order, and are restricted to a smaller number of IDs. Their IDs all start with FE, so that’s reserved one more of the 254 possible, but splits it into 255 sub-mods. 

[Tlam99]

I don't see the problem.

If one max out the ID's of an esp (as MMG mentioned, stupidly large)

then create  second esp which uses first as master.

 

However, I never would install a construct like this with so many references.

[Highborn]

28 minutes ago, Tlam99 said:

I don't see the problem.

If one max out the ID's of an esp (as MMG mentioned, stupidly large)

then create  second esp which uses first as master.

the author is concerned that using another .esp would cause issues, like he said: "I have never done this, I don’t know what problems this can lead to, I already have a lot of problems and errors all the time and with one plugin".

 

It's a quest mod, so he doesn't know if it will break things, having two .esps. But if it's not an issue, or if you have experience on this or know mods that did that with no issues, then I guess it's good news.

 

 

 

[Tlam99]

One can use as many esp until the the 254 limit is reached for a mod.

Following mods must use preceding mods as master(s).

 

I do this all the time for my mods, but not because of a limit of ID's.

More like it's much more comfortable creating addons instead messing with main mod.

One can choose tgen what she/he prefers.

 

Edited March 19, 2023 by Tlam99

[AndrewLRG]

5 hours ago, Highborn said:

I have no idea what will happen when all the IDs run out, and I don’t know if there is some kind of mechanism that will re-enumerate the free ones. Maybe you know something about this? Actually, for this reason, I decided to write this text. Maybe some of you know or can find information, because I personally could not dig anything sensible.

 

Tes5Edit has a script that renumbers FormIDs. It also tries to update affected references, but don't know how reliable it is.

[brewmasterhal]

6 hours ago, AndrewLRG said:

Tes5Edit has a script that renumbers FormIDs. It also tries to update affected references, but don't know how reliable it is.

 

Works fine, but it only updates formID references within the same mod file.  Using it will break any patches to the base mod, and critically will break the saved xTranslator XMLs that Dogma and I (and others) have been using to make the non-Russian versions.

[brewmasterhal]

10 hours ago, ghastley said:

I believe the CK uses different ranges of IDs for different things, so there are multiple “next available” values being tracked. The high water mark for one type being in the x’C3nnnn’ range does not mean there are no large gaps lower down, so the total used may be nothing like that high.

 

Esl’s have the top two bytes as load order, and are restricted to a smaller number of IDs. Their IDs all start with FE, so that’s reserved one more of the 254 possible, but splits it into 255 sub-mods. 

 

The actual total number of records is around 0x017500 or about 95,000, so there's tons of space left to fill.

[Highborn]

I talked with the author about this and apparently things are getting resolved, so Dogma asked me to thank you guys for the help.