Devious Devices Framework Development/Beta

[naaitsab]

53 minutes ago, HexBolt8 said:

@Scrab had recently posted here about working with a core set of DD functions in an SE environment, speaking from personal experience with a positive, can-do attitude.  We might look to that as the model to follow.

 

Let's see what Kimy thinks about it. Reading from the posts made here I think there are a few options to choose from. Which I will deliberately state quite broad. As there first needs to be taken a direction before we spend time on the details.

 

1. Leave it as is. So developed on and for LE. SE stays supported on a basis of users converting it and adding patches in the thread.

LE Positives: It's designed for LE

LE Negatives: None, bar any SE fixes that can't be implemented (a LE problem in general of course)

SE Positives: None

SE Negatives: Fan supported LE->SE ports, patches are posted in the thread can get lost and will get combability issues between patches. No official SE 'single-file' download.

 

2. Leave LE and mark the current version as the last build. Switch over to SE for future releases.

LE Positives: Current version will continue to work as frameworks that DD rely on are long abandoned. Fan made patches can always be made and contributed

LE Negatives: No new shiny things until patched in by fans.

SE Positives:  Make use of all the good things SE and it's mods bring to the table. Performance and stability being the primary ones.

SE Negatives: None

 

3. Split the mod into a LE and SE development cycle with official releases of both. Keeping a general core whilst each version get's their own change "under the hood" codewise.

LE Positives: New features. devices and tweaks are added

LE Negatives: Needs to be kept compatible with SE in the core. Needs more dev time and/or mod owners

SE Positives: New features, devices and tweaks are added. SE will get it's own official release. No patchfest

SE Negatives: Needs to be kept compatible with LE in the core. Needs more dev time and/or mod owners

 

 

If we look at it at a pure numbers game. So the amount of players and new mods made for SE then option 2 is the clear winner. But if we want to keep LE up to date, with all of it's EOL issues we need to work out 3. Which can be quite a hassle. Option 1 in my opinion is off the table. As Kimy said there are enough reasons now to shift our attention to another route. And if we keep running 1 I fear the ever dwindling popularity of LL will continue as SE players look elsewhere.

 

I propose we use this system to bring options to the table. If someone has a 4th or 5th option post it here.

[AndrewLRG]

12 minutes ago, naaitsab said:

1. Leave it as is. So developed on and for LE. SE stays supported on a basis of users converting it and adding patches in the thread.

SE Positives: None

 

2. Leave LE and mark the current version as the last build. Switch over to SE for future releases.

LE Positives: Current version will continue to work as frameworks that DD rely on are long abandoned. Fan made patches can always be made and contributed

 

That's funny.

 

[Hex Bolt]

1 hour ago, naaitsab said:

Let's see what Kimy thinks about it. Reading from the posts made here I think there are a few options to choose from.

Yes, it's already past the point where it feels weird talking about DD's future without Kimy's thoughts on the matter.  After Captain Kimy sets a course, it would be a good time to consider fruitful ideas for how to accomplish it.

 

Edit:  I know you were going for conciseness, which is appreciated, but points 1 & 3 above don't recognize the SE Player benefit of being able to continue to use LE-developed mods that use DD without extra conversion effort, possibly needing a third-party effort or patches.  That might include the newly re-released Captured Dreams, which targets DD 4.  It should work pretty well with DD 5.2, but maybe not so much with a divergent SE DD.

Edited June 24, 2023 by HexBolt8

[Scrab]

1 hour ago, naaitsab said:

Needs to be kept compatible with SE in the core. Needs more dev time and/or mod owners

This isnt really a negative but a rather neutral circumstance that is pretty common in a lot of ways. This "core" isnt really a big deal in the way you may think of it. Perhaps imagine this like a big room that is split in mutliple small ones:

 

Spoiler

 

Now, consider that in order for DD and other mods to interact, there needs be some extra layer that allows DD and these other mods to tell wtf is going on and this central layer is whats called an "API" and is also what HexBolt referred to as "Core" (I presume)

We use this API as a "middle man" to allow controlled and well defined data exchange between mods and DD itself:

 

 

The great thing here is that these mods are not actually interacting with DD but only with its API. This middle man acts essentially as an "interpreter". Similar to how you cant talk to someone that speaks a language that you cant through a middleman that speaks both languages, the API allows you to translate your requests to DD into some way that DD understands them

Now, what if DD were to change? Thats not a problem because if you tell the interpreter: Translate "I want cookies" into... french so that DD understands but DD then makes a change that causes it no longer speaks French, it now speaks German, then really all you need to do is change the interpreter. Then you got a different interpreter but you dont actually change anything on the dialogue, do you? The interpreter may be a different one, but you can still just tell that interpreter "I want cookies" and they will translate request for you into german so DD still gets it and give you cookies despite it no longer speaking french without you changing anything on your routine

 

This is why API are so powerful. They arent just there to give you some kind of easy access point into the framework but they also allow you to abstract the way you interact with a framework which gives the framework more ways to breathe

I hear some people cry over how SL+ is incompatible with a lot of things, but the reality is that SL+ made such a change and no longer speaks ... spanish. SL+ now speaks Japanese and if you try to make a request in spanish to SL+; avoiding or ignoring the interpreter/API it just wont understand you anymore, but mods that use the interpreter/API, can still make their request in spanish, and the interpreter will just translate it for you to Japanese, you will still get your expected result from that without actually noticing that SL no longer speaks spanish

 

Ok now, consider that we want to split DD into 2 parts, one that only speaks LE and another that only speaks SE, then really all we need to do is to make sure that the interpreter we use still speaks the language that the mods use to talk to DD and that that specific interpreter can speak the respective language of DD (either SE or LE) and that is all we need to care for. Essentially, going with that idea you basically turn this whole interaction model into something like this:

 

 

We have DD SE which only works on SE, we got DD LE which only works on LE, we got one and the same API for both and if you look at our mods room at the bottom, literally nothing changes. Theres still just 2 arrows, one in and one out and thats it. Both mods are indepdent from another, both mods could in theory have their own extra API if they wanted to but fundamentally, mods will that only use that one central API, will work fine with both DDs, no matter which version, no matter what either of them do as long as they dont mess with that central API everything will be fine

 

And what if one of them makes an important change that the other one should also get? You simply put a git between them and allow them to communicate through that:

 

 

And that is literally all

For mods using DD; nothing will change. They wont even notice this. Its essentially already like this except that Kimy is sitting in the DD LE chamber and the SE one is empty just getting an occasional visit to rid some dust because you dont use DD LE in your SE game and vice versa

 

These here are your 3 solutions:

 

                

 

 

Edited June 24, 2023 by Scrab

[naaitsab]

1 hour ago, HexBolt8 said:

Yes, it's already past the point where it feels weird talking about DD's future without Kimy's thoughts on the matter.  After Captain Kimy sets a course, it would be a good time to consider fruitful ideas for how to accomplish it.

 

Edit:  I know you were going for conciseness, which is appreciated, but points 1 & 3 above don't recognize the SE Player benefit of being able to continue to use LE-developed mods that use DD without extra conversion effort, possibly needing a third-party effort or patches.  That might include the newly re-released Captured Dreams, which targets DD 4.  It should work pretty well with DD 5.2, but maybe not so much with a divergent SE DD.

That's the exact reason I supposed to keep the calls the same between both versions. As mods are mostly API calls anyways. Otherwise you run into these kind of issues. Which are mostly easy fixes but one to mention non the less.

 

 But let's see what cap'n thinks of the options.

 

1 hour ago, Scrab said:

This isnt really a negative but a rather neutral circumstance that is pretty common in a lot of ways. This "core" isnt really a big deal in the way you may think of it. Perhaps imagine this like a big room that is split in mutliple small ones:

 

  Hide contents

 

Now, consider that in order for DD and other mods to interact, there needs be some extra layer that allows DD and these other mods to tell wtf is going on and this central layer is whats called an "API" and is also what HexBolt referred to as "Core" (I presume)

We use this API as a "middle man" to allow controlled and well defined data exchange between mods and DD itself:

 

 

The great thing here is that these mods are not actually interacting with DD but only with its API. This middle man acts essentially as an "interpreter". Similar to how you cant talk to someone that speaks a language that you cant through a middleman that speaks both languages, the API allows you to translate your requests to DD into some way that DD understands them

Now, what if DD were to change? Thats not a problem because if you tell the interpreter: Translate "I want cookies" into... french so that DD understands but DD then makes a change that causes it no longer speaks French, it now speaks German, then really all you need to do is change the interpreter. Then you got a different interpreter but you dont actually change anything on the dialogue, do you? The interpreter may be a different one, but you can still just tell that interpreter "I want cookies" and they will translate request for you into german so DD still gets it and give you cookies despite it no longer speaking french without you changing anything on your routine

 

This is why API are so powerful. They arent just there to give you some kind of easy access point into the framework but they also allow you to abstract the way you interact with a framework which gives the framework more ways to breathe

I hear some people cry over how SL+ is incompatible with a lot of things, but the reality is that SL+ made such a change and no longer speaks ... spanish. SL+ now speaks Japanese and if you try to make a request in spanish to SL+; avoiding or ignoring the interpreter/API it just wont understand you anymore, but mods that use the interpreter/API, can still make their request in spanish, and the interpreter will just translate it for you to Japanese, you will still get your expected result from that without actually noticing that SL no longer speaks spanish

 

Ok now, consider that we want to split DD into 2 parts, one that only speaks LE and another that only speaks SE, then really all we need to do is to make sure that the interpreter we use still speaks the language that the mods use to talk to DD and that that specific interpreter can speak the respective language of DD (either SE or LE) and that is all we need to care for. Essentially, going with that idea you basically turn this whole interaction model into something like this:

 

 

We have DD SE which only works on SE, we got DD LE which only works on LE, we got one and the same API for both and if you look at our mods room at the bottom, literally nothing changes. Theres still just 2 arrows, one in and one out and thats it. Both mods are indepdent from another, both mods could in theory have their own extra API if they wanted to but fundamentally, mods will that only use that one central API, will work fine with both DDs, no matter which version, no matter what either of them do as long as they dont mess with that central API everything will be fine

 

And what if one of them makes an important change that the other one should also get? You simply put a git between them and allow them to communicate through that:

 

 

And that is literally all

For mods using DD; nothing will change. They wont even notice this. Its essentially already like this except that Kimy is sitting in the DD LE chamber and the SE one is empty just getting an occasional visit to rid some dust because you dont use DD LE in your SE game and vice versa

 

These here are your 3 solutions:

 

                

 

 

 

In this case the left and right options still need patches if there are no official releases. In a "the needs of the many outweigh the needs of the few" usecase one could argument that LE should then go for the patch-support route. But let's see if we can get the middle one worked out. Or someone else has another angle to it all?

[thedarkone1234]

On 6/23/2023 at 8:54 PM, GreatCroco said:

Name a mod, any mod, that exists only on LE and, does not work with SE and has no replacement.

 

Not exactly something that "exists only in LE" per se, but BHUNP version 1.8 no longer exists as far as I searched on the internet. Current versions of it are incompatible with UUNP as too many sliders have been removed, added or changed. Which means: if I want to move to SE, I'd need to find myself good new armor replacers that work with the currently available versions of BHUNP (there aren't any good ones yet. Nothing like "Remodeled armor for UUNP"), or give up on BHUNP and just use UUNP, which has inferior breast physics and a much inferior vagina.

 

Not to mention that it appears HDT SMP is almost mandatory in most SE things (while on LE I can do with PE just fine), and I see a multitude of issues of people struggling to deal with havok objects and unworking physics, while mine work out of the box, on BHUNP 1.8, without me having to manually fix a bunch of files like all the solutions suggest.

 

If you can give me an actually simple-to-use one-fit-all body and armor replacers that fit it which also have compatible DD morphs, I might actually consider trying out a transfer. Else, if I can't even get my character to look right nude or clothed, with or without devices, without making me work manually through hundreds of bodyslide files, it's enough of a reason for me (and for most users of sex mods, I would guess), not to go for it.

[GreatCroco]

CBBE 3BA? That's what I'm using and I have BodySlide for every armour, with one or two exceptions iirc, but it's been a while. DD Sliders might be a bit wonky if memory serves correctly, but I don't have any problems. Unless I try to match ingame morphs to Bodyslide itself, but that's a completely different topic. CBBE 3BA also allows you to select between HDT-SMP and CBPC, or a mixture of both.

Edited June 24, 2023 by GreatCroco

[TrickyK]

7 hours ago, thedarkone1234 said:

Not to mention that it appears HDT SMP is almost mandatory in most SE things (while on LE I can do with PE just fine),

HDT SMP is HDT PE, just the SE version with a different name.

 

It also is outdated, because FSMP exists ( https://www.nexusmods.com/skyrimspecialedition/mods/57339 ). It's a vastly improved HDT version that can use modern CPU instruction sets and GPU acceleration.

 

7 hours ago, thedarkone1234 said:

I see a multitude of issues of people struggling to deal with havok objects and unworking physics, while mine work out of the box

 

Probably people running unlocked framerates above 60 causing the engine to spaz out. FSMP has a fix for this (or atleast some fix out there exists if I can remember it).

 

I've never had an issue with physics not working on SE when using FSMP, with one notable exception. The chains on some DD objects don't connect right. Zarantha has done some pretty heavy work getting them to look almost right though. That being said, the FSMP devs are actively working on a fix for this, and someone in the discord sent the dev a working bit of code for this specifically.

 

8 hours ago, thedarkone1234 said:

If you can give me an actually simple-to-use one-fit-all body and armor replacers that fit it which also have compatible DD morphs, I might actually consider trying out a transfer.

 

As croco said, CBBE 3BA is the best choice here. It is in no way inferior to BHUNP like UUNP vs CBBE was. 3BA can also have realistic jiggle if you choose the right options during install, if that's one of the reasons you went to BHUNP. You can also edit the config files to fine tune it.

 

As a bonus, it's also the most widely supported physics body, and has quite a few armor replacers. I'll just list a few.

 

Spoiler

https://www.nexusmods.com/skyrimspecialedition/mods/68235

 

https://www.nexusmods.com/skyrimspecialedition/mods/32518?tab=description

Note that this one wasn't updated to include the thigh gap slider, but is otherwise fine to use.

 

https://www.nexusmods.com/skyrimspecialedition/mods/81673

 

https://www.sunkeumjeong.com/post/the-amazing-world-of-bikini-armors-remastered-6-0

 

https://www.nexusmods.com/skyrimspecialedition/mods/70583

Replacers for the Anniversary edition DLC should you choose to buy it. Also includes zaps in the bodyslide.

 

https://www.nexusmods.com/skyrimspecialedition/mods/68238

Made by the same person as above, not a full replacer however.

 

 

[GreatCroco]

First a (small) bug report: If you have a keyholder and you loot a key from a container without looting the entire container (i.e. you right-click on the key to loot it) you might get the "cannot find the key" message in addition to "keyholder takes key". I have not checked if the keyholder actually gets the key or not. This does not happen if you loot the entire container.

Possible solutions:

1. The key always gets handed of to the keyholder no matter what

2. You first "loot" the key and it then gets handed to the keyholder. This also fixed the ""bug"" that you can just set the keyholder duration to one day, never return to them and not having to worry about dropping your keys ever again. If you need the keys you just waltz to the keyholder get the keys, remove devices and repeat the above. This could also be fixed by the keys magically getting returned to you when the duration is up, forcing the player to be more careful when exploring.

 

 

 

Second: I played around with CommonLibSSE-NG a bit and ported the DD SKSE plugin it it. Took me a whole of 30 minutes and now it has better logging, a dll for SE and AE, proper versioning, a single development setup with CMake that works with VSCode, VS, CLion, EMACS, etc. The plugin can now also be configured by a simple YAML fiel (e.g. for log levels) and supports hooking int SKSE and unit testing. It should also support easy ESP/ESM/ESL and FOMOD installer generation, although I haven't looked into that yet. It also integrates AdressLibrary and is therefore (almost) Skyrim version independent. SKSE is also dropped as a static dependency (users still need it though), making developing much less of a hassle.

 

Only downsides: Not compatible with the LE plugin in any case (No CommonLib), every change needs to be manually be ported back. I'd also like a better Conan integration instead of VCPKG, but that's still in beta.

 

 

20 hours ago, Scrab said:

This isnt really a negative but a rather neutral circumstance that is pretty common in a lot of ways. This "core" isnt really a big deal in the way you may think of it. Perhaps imagine this like a big room that is split in mutliple small ones:

Kimy already stated that they have no intention of making another braking change to the API as they did with V3->V4. 

Any future changes can then just add new API methods and deprecate the old ones with a simple log message, but otherwise keep them intact. This is similar to the way MathWorks and Microsoft do things and no matter how much I hate MATLAB, I give them, that I can run 15 year old programs with the latest version and maybe get a warning in the console.

 

But that was my idea anyway if a hard cut isn't done. Still wouldn't call that "common core" though.

[naaitsab]

1 hour ago, GreatCroco said:

But that was my idea anyway if a hard cut isn't done. Still wouldn't call that "common core" though.

Well if you look at it from a programming standpoint. The software core itself will be different. The calls (API "layer") to this core should not change. But one could also argue that the core could also be referred to as the core of the mod shared by both versions. So the meshes, animations, textures and functions of DD. (yes LE and SE in this also differs I know). This is used in more mods, mostly FOmod systems to define the mod without any patches, tweaks or addons.

 

Because we are dealing with different nationalisaties and backgrounds it might be best not to use pre-defined terminology before we make assumptions on each others outings.

[ChandraArgentis]

On 6/23/2023 at 12:54 PM, GreatCroco said:

Name a mod, any mod, that exists only on LE and, does not work with SE and has no replacement.

 

Devious Regulations.  Here's the compiled pack with the upgrades and patches for LE.  Doesn't work in SE.

Devious Regulations.zip

[GreatCroco]

The problem with Devious Regulations is not SE/AE, it's that it was built for DD version 2. See the topmost entry in the changelog:
 

Quote

1.7e
- Fixed camera getting stuck during Stormcloak scenes
- Improved behaviour with 3rd party collars and plugs
- Updated for DDi 2.8.1

Devious Devices had breaking API changes with version 4, which is what put Captured Dreams to its multi year hiatus.

Edited June 25, 2023 by GreatCroco

[naaitsab]

I've been running a customized version of that one on SE for a while now. Only thing that needs to be changed to make it work is some devices which one of the patches for LE also does. So yes, it's a DD "issue" not a LE-SE issue.

[ChandraArgentis]

2 hours ago, GreatCroco said:

The problem with Devious Regulations is not SE/AE, it's that it was built for DD version 2. See the topmost entry in the changelog:
 

Devious Devices had breaking API changes with version 4, which is what put Captured Dreams to its multi year hiatus.

1 hour ago, naaitsab said:

I've been running a customized version of that one on SE for a while now. Only thing that needs to be changed to make it work is some devices which one of the patches for LE also does. So yes, it's a DD "issue" not a LE-SE issue.

 

It works fine with LE though, so it does appear to be a LE-SE issue.  Tested with 5.1, I think.  I don't think I've tested with 5.2.

There are 3 variants of Regulars.  The initial mod, an update to it, and a patch to that one.  The file I provided is the most up-to-date I've found and works with LE and DD 5.x, but not SE.

Edited June 25, 2023 by ChandraArgentis

[TrickyK]

17 minutes ago, ChandraArgentis said:

I've found and works with LE and DD 5.x, but not SE.

 

I did a small amount of digging. It's because no one has bothered to post a converted version as most people just do it themselves with Cathedral Asset Optimizer. From the original thread >

On 1/13/2022 at 9:07 AM, Baltasarr80 said:

You can use the cathedral asset optimizer to make it work in se ... i tright it the the dev regulations plug version on se

 

This takes like 10 minutes to do. The plugin might also need ran through SSEdit for the bugfix patch, but probably works with no effort.

[DonQuiWho]

1 hour ago, TrickyK said:

 

I did a small amount of digging. It's because no one has bothered to post a converted version as most people just do it themselves with Cathedral Asset Optimizer. From the original thread >

 

This takes like 10 minutes to do. The plugin might also need ran through SSEdit for the bugfix patch, but probably works with no effort.

 

I hate to butt in on the technical discussions of my, maybe not elders, but almost certainly betters, but there some of us out here, maybe even quite a lot of us, who might well not have the faintest idea what you're talking about, let alone start to try to do what you suggest. 

 

Very grateful for the scraps from the masters' tables, but that's about the best some of us can hope for

 

DQW

[TrickyK]

11 minutes ago, DonQuiWho said:

there some of us out here, maybe even quite a lot of us, who might well not have the faintest idea what you're talking about, let alone start to try to do what you suggest. 

 

Alright, https://www.nexusmods.com/skyrimspecialedition/mods/23316 Cathedral Asset Optimizer is a mod made specifically for porting assets from LE mods and converting them to work with SE. Full use instructions are on the Nexus page ( https://gitlab.com/G_ka/Cathedral_Assets_Optimizer/-/wikis/Porting-a-mod ). As you can see, it's not a very long process.

 

As for SSEdit, it's the SE version of XEdit. It's not exactly needed to convert them however, but I use it to verify nothing went wrong.

 

The preferred way to convert plugins is to open the LE plugin in the SE creation kit, and save it. This converts it to form 44. This usually is not even needed though, but if you do want to, here's a tutorial. I just do it because mod organizer throws a warning if you don't.

Spoiler

 

 

As for why I'm bringing this kind of stuff up in the DD development page, there seems to be a sizable number of users here that seem to think going from LE to SE is a massive chore or will break everything, or will make them lose half of their mods.

 

Most of this is misinformation, and it's actually much easier than people think if they don't mind doing a bit of research. I was forced to move to SE because of the save string limit, a hard limit on the amount of data a save can contain before it just starts crashing your game. I kept getting into about 50 to 100 hours and then my saves all die to it.

 

This is not even remotely an issue on SE as long as you install SSE Engine fixes.

 

At no point did I ever regret switching over. The new mods only on SE more than made up for the ones I didn't bother converting on LE (which was like 4 mods, and nothing major).

 

I'm also of the opinion that sticking to LE is holding back the DD Framework. DD is already losing out on features and performance for the sake of backwards compatibility. I'd rather the mod authors on LE move forward with us than stick with LE, and show them that it really is not that hard to switch over. I don't like the idea of leaving anybody behind, but if we don't switch now, than when some other incredibly useful SE only mod comes along that DD could very much use, this discussion will end up coming back again.

[qawsedrftg765]

On 6/23/2023 at 4:13 AM, Kimy said:

In the end...I am not sure. Feel free to let me know what you guys think! 

I am not a major mod author on this site for SE/LE but I can say that in my experience as a heavy end user Skyrim SE has been a massive boon. The SE platform with things such as .NET scripting framework are just capable of offering more to the end user. On top of that outside our pervy corner of the internet ALMOST all MAJOR Skyrim content is SE first. I am personally running a save game with DCL, DD, and LotDB since March 2020 with hundreds of hours played and the save game is super stable with minimal issues. I could see an argument in 2019 to not swap over as there were still a lot of the major mods not ported over. But now It is really had to think of more than a handful of the 'big' mods that have not been outright ported or superseded in some way. The ones that I can think of tend to be the horny ones. 

the DD framework and mods that use it are probably one of the few that are LE first and SE second with Zarantha doing a hero's effort to make things like DCL and DD work on SE.

More users play on SE, SE is more powerful and has more options these days. Let us not forget SE came out in 2016. It is a matured game now... it has been the 'main' version of skyrim for longer than LE was the main version of skyrim. SE is obviously not the 'only' way to play skyrim and I OBVIOUSLY, hold nothing against people who play LE.

As a someone who has done a lot of dev work IRL (mostly for iOS and Android) I know the pain people like Kimy are in where you can look @ the user data and see that a small(er) percentage of people have not changed to the 'new thing' are simply happy with their windows XP/Vista machines (Seriously like 2% of users in 2023) But you want to do an update and you KNOW that those people can not have it no matter how much you would want to.

There are then two ways to frame the question

"Is it fair to not improve the end user experience as much as possible to accommodate legacy users? "
OR
"Is it fair to not accommodate legacy users to make the most complete version of the software?"

The easy option is to just say 'JuSt MaKE iT WoRK oN BoTh'. Which anyone with more than 2.3 minutes of dev experience will tell you is if not impossible, massively implausible.

There isn't a perfect answer but in my experience the overwhelming majority of people will swap. People will kick, scream, and hollar all the way upto and during release; whatever choice you make the unhappy group will ALWAYS be the loudest; I once got a furious email from someone upset my company was no longer supporting WINDOWS PHONE. But then they almost always swap and the project is better for it.

 

I would say go to SE, but I am on the outside looking in and the only one who can make the ultimate call is the dev and whatever Kimy decides should be respected.

As a note. Remember the Modernwarfare boycott where people where FURIOUS about MW2 and WOULD NEVER play it without dedicated servers, how forum users would make sure the game was a failure.


Edit: I have seen a lot of people mention Cathedral Assets Optimizer (CAO) and how it can be used to port mods over the SE. Which is true, but the more complex the mod the less likely it will work out of the box. Something like "Being a Cow, DCL or DD" do NOT work without specific effort being put in by people like zarantha. It is also not likely that every user will both use CAO and use it well.

It is also worth noting that if users will not swap for a new update, they still have the old stuff they like. I would assume Kimy would not just delete and LE content if SE became the primary development branch.

Edited June 26, 2023 by qawsedrftg765
CAO

[Hex Bolt]

Thank you, @qawsedrftg765.  Although I'm one of the people hoping for ongoing LE support for Devious Devices, that was a thoughtful and well-stated post.

[krzp]

Speaking of "easy to develop on LE and then convert to SE", there is a popular mod made for DD, it's being currently developed on LE and then converted by someone else to SE.

Has like 250 k downloads on SE (in fact, it's LE version is at 100k downloads, so SE is the more popular version)

 

It also has this:
 

Spoiler

 

 

 

 

 

 

.... ooops.

(I'm not missing any dependencies, it's a HITME thing)

 

P.S. Tbf, the mod works - trips up DynDOLOD and my OCD, but works.

Edited June 26, 2023 by krzp

[GreatCroco]

9 hours ago, TrickyK said:

The preferred way to convert plugins is to open the LE plugin in the SE creation kit, and save it. This converts it to form 44. This usually is not even needed though, but if you do want to, here's a tutorial. I just do it because mod organizer throws a warning if you don't.

Both Arthmood and Mator, (for those who don't know: two of the most well known and respected mod authors for Skyrim) recommend to resave the plugin because the structure is different. They say it most likely won't make a difference in small saves, but the longer you play the more likely a total save corruption becomes.

 

5 hours ago, qawsedrftg765 said:

As a someone who has done a lot of dev work IRL (mostly for iOS and Android) I know the pain people like Kimy are in where you can look @ the user data and see that a small(er) percentage of people have not changed to the 'new thing' are simply happy with their windows XP/Vista machines (Seriously like 2% of users in 2023) But you want to do an update and you KNOW that those people can not have it no matter how much you would want to.

The usual approach from companies like Microsoft, Apple, Google, and whatnot is either:
* Drop the support entirely (Google)

* Offer the people a year long upgrade path (Apple)

* Switch to the new version, and only port back the most serious of problems, but don't retire anything either (Microsoft and Intel)

 

 

 

 

 

 

 

A few more anecdotes from my "porting DD SKSE to SE" adventure. Porting the plugin itself was easy. But when I tried porting some heavier Papyrus functions to C++ it got wonky. Since you can't call Papyrus from C++, porting needs to start at the very core lib and since Papyrus and C++ are so different in how they approach Skyrim, it means you can keep the API surface the same, but not the underlying implementation. That means, that if the computationally intensive parts are ported to native code, it automatically means you can't back or forward port bugfixes without rewriting them entirely. I tried starting the with furniture library, as this is the area, which the least amount of mods depend on, but that calls the core lib, so that's a dead end for now.

 

 

There's also another problem I ran into: DD makes heavy use of StorageUtil to store states. While this is SKSE based, I can't find the sources, so I can't use it in C++. Whether I can leverage the SKSE co-save is tbd. 

 

 

 

 

@Scrab How did you interface with SexLab? That would be my next problem in porting some of the code

 

 

[naaitsab]

1 hour ago, GreatCroco said:

Both Arthmood and Mator, (for those who don't know: two of the most well known and respected mod authors for Skyrim) recommend to resave the plugin because the structure is different. They say it most likely won't make a difference in small saves, but the longer you play the more likely a total save corruption becomes.

 

It's not that difficult to convert a "plain" mod from LE to SE. 

 

-If it has a BSA, unpack it first

-Open the ESP/ESM in the CK and just click save to upgrade it to the new form and possible change some other things under the hood. Using Tes5Edit alone is not recommended.

-Convert the meshes and texture using Cathedral or manually for small mods using DD tools on Photoshop or GIMP and Bodyslide. Or NifSkope if you know what you are doing.

-Optional: Repack the unpacked files into the SE BSA format.

-Optional: use Tes5edit to convert the newly saved ESP to a ESL.

 

If it has animations these need to be converted as well. I'm not sure there are any batch processing tools for this and will probably require a import in blender and export to the new format. But I'm not skilled in animating so don't know the details.

 

If it relies on dll's like SKSE addons you start to run into problems quite fast. Those mods are not for new player to try and convert. Unless you are skilled in C++ and know how SKSE works under the hood.

 

Also anything physics based also needs a manual conversion. For example the poisoner chains in DD are a "fine" example of something that causes issues. Which seem to be quite a challenge to fix.

Edited June 26, 2023 by naaitsab

[GreatCroco]

I want to add that BSA's must be converted, they will crash the game if they are in the old format. It's also recommended to not have loose files because you might run into hard to debug conflicts, although the latter is somewhat mitigated by ModOrganizer.

And while converting into an ESL sounds fine, you should know when to convert it. All records are moved into the FE id-space, you can only have at most 2048 records in the plugin and ESL are loaded before ESPs, but after ESMs.

[Scrab]

2 hours ago, GreatCroco said:

How did you interface with SexLab? That would be my next problem in porting some of the code

You can compare my work on the front end to default SL on my github:
 

https://github.com/Scrabx3/SexLab/tree/animation

 

When Im done with some certain block of code I usually push it over asap. While my indev has a lot more changes, the scripts which I will focus on in the following text are pretty much completed:

• sslThreadSlots.psc
• sslThreadLibrary.psc
• sslAnimationSlots.psc
• SexLabFramework.psc

When you go over these scripts the sslThreadLibrary one will likely look the most intriguing to you as almost everything in this script has been re implemented as a native function but consider scripts like sslAnimationSlots where little native declarations are found, or even sslThreadSlots which has no native functions at all despite being a core script in SLs implementation, arguably even the most important one as it is fetching new threads for scenes to play on.. yet there isnt a single native declaration on it, why?

Also consider SexLabFramework.psc underwent some big changes  with large chunks of its code no longer being part of the API but if you take a closer look, you will find that all of its functions are still there, just made redundant and potentially having its content delted. I.e. you can still call them, they just dont do anything. These particular fucntions are breaking backwards compatibility as they make logic unavailable that is required for previous versions

Another important detail is that sslAnimationSlots is entirely redundant, this is the old registry that other versions run on, the registry that mods like SLAL and SL itself load animations into and there too, theres actually not a whole not native code there. I made 3 native functions and p much called it there

 

However, If youre trying to do what I think you are trying to do, then I will tell you right now that you are going to run into more walls than you can count and majority of these walls you will build yourself because of premature optimizations on the code base where they arent needed and may even be contra productive, may complicate things or create issues that cant be fixed in a way that will allow LE to continue using it - and this will also mean that you are going to split the community and likely result in your attempt being a failure because I will be supporting a SE exclusive DD (under these conditions) as much as HexBolt will

In these kind of projects, C++ is NOT the front end. Its the back end, and as the back end is has no actual control over what is happening on the front end, i.e. what papyrus does and to that end, papyrus needs to retain full control over what is happening. This is why sslThreadSlots is all in papyrus. I would have no issue porting everything into C++. in fact It would be an easy task to just move everything into C++ but doing so would mean that C++ is in absolute control over the framework when in it actually should just be there to control the data, offer access to it, offer some misc utility and otherwise be stateless

 

In order for this to work, you need to first understand what the actual issue in DD is and how you can approach them in a way that is compatible and necessary. Dont just run in there with a sledgehammer and hope for the best doing so. Quality is a matter of perception, putting "native" after every function isnt quality, its just a bunch of native functions

This is why I keep tellin ppl to hold their breath when they wish to compliment me on SLp+, because this first phase of this project (SL+) is literally only there for me to understand what I should be doing, what I can do, and to that end: If you look at what I did in p+ you will notice that I didnt really do anything in the dll at all. Like sure theres this cute little integer that looks kinda fancy and a dozen native utility functions but fundamentally? Nothing in there is really impressive. In fact all of the things in that dll could easily be done in Papyrus as well and you wouldnt notice any major  difference in how SLp+ performs

 

Understanding what DD is missing is literally what Kimy has been doing all along. If you read my first post which got this whole discussion started; This one here (avoiding the embed)

You will notice that in this post, in these 3 major points I made, I didnt actually mention anything about "fixing DD" or "addressing its issues" or anything like that. I did mention it once as a side effect, but thats a side effect resulting from doing something else (in this case overhauling the device hider). Performance, speed is literally not the point of why you would want to do anything SE exclusive. Of course its nice to be faster, operating in real time is the holy grail for mods like this but if you just blindly move everything into C++ and call it "optimization" without actually knowing what is wrong in first place, what you should address then all you really do is making the live of DD authors harder who now have to add a another 100ms delay between device equipping because DD trips up twice as fast as before

 

I know this all sounds incredibly harsh and I hope that doesnt completely kill your vibe to do anything modding, but I really want to stress that C++ and SKSE plugins arent some kind of wonder drug that fixes all issues just by being there. Youre probably super excited because you got your C++ setup running and can compile things and that is super cool, its great that it motivates and that youre trying to actually get things going but remember:

All of these things that you have access to in SE are just tools and as with any tool, you not only need to learn how to use them, but you also need to learn when to use them. So, if you wanna help getting DD forward with your SE Toolkit then you first need to understand what you should be doing and the best way to do this is to not actually use your SE toolkit at all. Help Kimy improve the Papyrus code base without using your dll understand why it has issues, address these issues and if you ever run into some kind of wall that Kimy has to walk around because LE you can look at your Papyrus toolkit and maybe youll have something in there that will help you either smash that wall or just go over it and that is when LE and SE go different paths: LE will go around that wall, SE goes straight through it. The important thing is that once you go through this wall, and you get to a path that LE is also walking, you dont strive from that path just because you can. Walk with LE again, and walk with LE until another one of these walls pop up. Rinse and repeat until both of you have a path that works and is optimal

 

Edited June 26, 2023 by Scrab

[GreatCroco]

1 hour ago, Scrab said:

You can compare my work on the front end to default SL on my github:

That sadly doesn't help me one bit, as your code is entirely in Papyrus. That was not what I was looking for.

 

1 hour ago, Scrab said:

Youre probably super excited because you got your C++ setup running and can compile things and that is super cool, its great that it motivates and that youre trying to actually get things going but remember:

 

Naww... I'm programming stuff for more than two decades now. Finding out what the customer needs (not wants) is literally my daily job. So, getting any C++ stuff running is more chore than pleasure and often not more than just a few shell commands. Sure, if I could, I'd take SKSE and port it to some sensible language with proper package and version management (I did already integrate Rust with UE, so nothing totally out of the ordinaty). I'm just glad it's not PASCAL or COBOL I'm dealing with?.

The thing is: Most of what's currently written in Papyrus is totally fine. For example: The device hider logic is totally fine, sure "UpdateSlotmask" runs in O(n), but that won't change in native code, nor does it run that often. The problem with the device hider is the MCM, that could use a total overhaul. I didn't even look at most of the scripts, because they have no purpose being native. Why should I be concerned with the 65 LoC in zadx_RopeHarnessScript? That's totally irrelevant.

 

zadclibs and zadlibs are the interesting bits, where most of the stuff happens

  

1 hour ago, Scrab said:

address then all you really do is making the live of DD authors harder who now have to add a another 100ms delay between device equipping because DD trips up twice as fast as before

Most of the tripping that DD does, at least in my (anecdotal) testing, was due to Papyrus throwing a hissy fit because the VM was overloaded. This has gotten better in SE because of... well SE. In LE I could go and make myself a tea while I waited for a complete unequip with every slot covered to complete. In SE this takes just a few seconds.

In the best of cases, moving stuff from the VM to native should help.

 

 

 

1 hour ago, Scrab said:

I know this all sounds incredibly harsh and I hope that doesnt completely kill your vibe to do anything modding, [...] 

Also on a side note, I may argue that going hands on with the "SE exclusive port" before Kimy actually has given any statement to any of this is a little bit rushed

I had never any drive to start it up (again) in the first place. The whole thing was just me messing around at a free weekend to see how easy it could be to move the most computationally intensive functions off Papyrus and into native code. Just move the plugin into CommonLibSSE-NG, improve some stuff, drop it here and leave it at that.

 

Even if you did "kill the vibe", it's not like I have a full disk of other open source project lying around. Maybe I really should finish that Papyrus Compiler finally?

 

  

1 hour ago, Scrab said:

offer some misc utility and otherwise be stateless

I come from a functional programming background, Haskell was the first language I learned in university. Having everything stateless is nothing new, or even remotely challenging.

Edited June 26, 2023 by GreatCroco