[SES] Slaverun Enforcer Standsalone

[Cleave]

On 11/15/2022 at 12:57 AM, kaxat said:

@Cleave That is not normal. Almost certainly a sign that the funky behavior continues but now has a different symptom. I would be concerned.

That message triggers when the quest SLV_FindSlaverPatrolQuest is at stage 0. But it should advance beyond stage 0 quite quickly. The message should only trigger once every few hours. You can use the console command sqv SLV_FindSlaverPatrolQuest to see if you notice anything funky. But again I think we are focusing on the symptom of a larger problem. Something is broken in this save and you should go back before the save is broken. Because more than likely other hidden things are broken too. And will reveal themselves in time.

 

 

Hi! So I tried sqv SLV_FindSlaverPatrol to see whats up and I got the image below, while in enslaved Riverwood... Everything works fine with the mod, except that script which doesnt fire up... It stays "stopped" for some reason, is there a way with the console to start it up? With a setstage maybe?

 

[alex77r4]

15 hours ago, kaxat said:

Does anyone have a strong aversion to me requiring powerofthree's Papyrus Extender in a future version? It would allow me to strip out a lot of code. It would also probably make things faster. Definitely make performance more configurable. Where you can dial back how many NPCs get checked each enforcer run. And possibly lead to new features. Such as outfit themes being able to dye an NPC's hair.

 

The Legendary version of Papyrus Extender does not have ALL the functions availables in its Special version.

So, if you want continue giving support to Legendary users, you must be a bit careful about what exact functions you use.

Specifically, I removed Papyrus Extender from Yamete REDUX because it was using two functions that don't exist in the legendary version of the DLL.

I replaced those two functions with my own code and now my mod works in Legendary, Special, Anniversary and VR.

Edited November 21, 2022 by alex77r4

[kaxat]

@alex77r4 Good point. I had already confirmed that the main function I want is included in LE. So this should not break LE compatibility. Assuming the LE version isn't buggy.

The function is GetActorsByProcessingLevel(). It allows me to get all actors that are currently being processed by the game engine. So much more efficient for my needs than the current setup. It's a function I had long wished existed. I stumbled upon it the other day. Currently I rely on a cached list of local SexLab valid actors to find people from neighboring cells. In order for somebody to get added to the list they have to first be in the same cell as you. I have always considered switching to a cloaking spell. But that seems to be the least performant option in my case.

 

With this I can poll for all local actors every 60-180 seconds. And on location changes. I already have a multithreaded actor validator built so I can just plug the new actorlist list into that. And store the filtered array of valid actors for future scenes.

 

On a separate timescale I will initiate new SexLab scenes. User configurable, likely default to every 5s. This will start scenes more regularly. And help spread out when script load is occurring.

In all this should notably increase performance. In part because all of the actor validation and stripping will be handled by the multithreaded part of the mod. And this function makes it a lot easier to multithread the remaining bits. But also in part because SexLab scenes will happen on their own interval. Which will remove that slightly strange behavior where the enforcer suddenly starts a bunch of SexLab scenes. Instead those scenes will regularly be starting if actors are available. More natural this way.

Edited November 21, 2022 by kaxat

[kaxat]

@Cleave I forgot to mention that is normal for this script to be stopped. What would worry me is if it is still running all of the time. If it was you could be getting those notifications you saw earlier.

 

You said you got the rest of the funky behavior to go away. Which is great. Sounds like you got it all to go away! Did you have to load back to a previous save? Start a new game?

[alex77r4]

40 minutes ago, kaxat said:

The function is GetActorsByProcessingLevel(). It allows me to get all actors that are currently being processed by the game engine.

 

I don't know why you need a specialized function to get that. IMO, the best way is to use a quest with say 50 aliases that the base game can fill instantly.

Also, using a quest, you can check many conditions when the alias is filled like IsDead(), IsActorTypeNPC, IsCommandedActor... and remove all those checks from your scripts because, if the actor is in the alias, it's because it matches the conditions.

[cbeyond8027]

Question about creating an outfit theme. If I have an item on the free women list does it need to be equipped or just in the inventory for them to be recognized as free? I was thinking of giving a group of women I want to remain free a specific ring or something using SPID to avoid having to add keywords to each of their outfits. Just concerned some of them might not equip it because of a conflict. It might even be a cool integration to SL survival if it works (i.e. amulet/ring designating freedom). 

[kaxat]

@alex77r4 Unfortunately that would not work for some of the functionality. And there is a decent chance it would cause the most script lag. While it might actually be the fastest way to get an actor list. It would do so at the expense of a lot more CPU time. Not benefiting from a lot of caching and deduplicaction of effort. Which is available to me with Papyrus. Each actor would need to do 20 or so checks and compare many of them against the MCM quest's properties. The current city actor list often has 60+ actors in it and I can pull from it in about 50 milliseconds, +1 frame delay per actor. The main area where a ref fill would speed things up is that I don't believe it will experience as many frame delays. But it would still end up doing a whole lot more duplicate work. I anticipate that once the cache is primed this new method will be equally fast. And a single frame delay is fine for me.

Personally I also like to keep things in Papyrus when possible because it's a lot easier to maintain than spreading logic over 50+ aliases.

 

@cbeyond8027 It needs to be equipped. And furthermore the body slot is weighted far more heavily than the other slots that get checked. Such as 46, 49, 52, and 56. So if she is wearing slave clothing on the body but gloves that are tagged for free women men will still think she is a slave. The point of the system is how she looks is how she gets treated.

[cbeyond8027]

35 minutes ago, kaxat said:

 

@cbeyond8027 It needs to be equipped. And furthermore the body slot is weighted far more heavily than the other slots that get checked. Such as 46, 49, 52, and 56. So if she is wearing slave clothing on the body but gloves that are tagged for free women men will still think she is a slave. The point of the system is how she looks is how she gets treated.

Got it, I’m trying to find a way to run Slaverun with SkyFem so apologies if my questions seem a bit strange. 

[alex77r4]

45 minutes ago, kaxat said:

there is a decent chance it would cause the most script lag. While it might actually be the fastest way to get an actor list. It would do so at the expense of a lot more CPU time.

 

Start a quest never can cause Script Lag and, of course, is the faster way because the game engine have their own list of actors in RAM. Exactly, the SKSE getNthRef and Extender GetActorsByProcessingLevel read the list of actors from the Game Engine. Fill a list of Alias is an operation that is integrated inside the Game Engine, because the game is entirelly made arround the Quest, and is very optimiced.

 

45 minutes ago, kaxat said:

Personally I also like to keep things in Papyrus when possible because it's a lot easier to maintain than spreading logic over 50+ aliases.

 

That is not the correct way to made a mod for Skyrim. The base concep of the Game is the Quest and everything that you can made in CK must be made in CK. You can use Quest, Alias, Spells, Keyworks, Formlist... a tremendous amount of posibilities... and, theoretically, you must use it because anything that you put inside CK is executed instantly, from one frame to another, by the Game Engine.

The Game has not been designed around Papyrus mainly because Papyrus is slow as hell. Make in Papyrus the same that you can make with a Quest mean hundreds of lines of code and an undetermined number of frames to execute it because nobody can know how many time need each delayed call and how many script are executing in the Script Engine.

 

For that all the mods have a lot of Quest, Alias, Spells... and, when necesary, Papyrus Code. That is how are made all the mods.

For example, SexLab have 15 Quest with 5 Alias in each Quest and each Alias is atached to their own script to take advantage of Papyrus Multiprocess.

 

But this is your mod and you can made it as you want. I only give recomendations.

[kaxat]

@alex77r4 I do appreciate the feedback and weighed it for quite a while. But I strongly feel it is not applicable here.

First of all I have done a decent bit of testing and found Papyrus to be nearly as fast as native code. It's actually quite impressive. I rebuilt some of the functionality that external DLLs add in Papyrus. Then tested both functions with repeat calls. In many cases there was no almost difference running native C as opposed to Papyrus. And while that is not my expectation globally it is a great indication that Papyrus can be fast. Its the frame delays and BudgetMS global settings that can make it appear slower. But those are by design and for the most part very beneficial delays.

 

As I understand GetActorsByProcessingLevel() is reading directly from the Skyrim actor list in memory and making a copy of it. Which should be stupidly fast and exactly what I want.

You absolutely can induce script lag by poorly designed conditions. One of the things they warn us about is having too many global conditions on packages. It can slow the game down globally even when the NPC isn't near. In this case having 20-40 conditions x 50+ actors seems like a great way to consume CPU cycles. Especially if the refs don't fully fill. I would have to measure to be sure. And it is difficult to measure performance outside your own code. But I bet it will be introducing a constant CPU load on the remaining unfilled refs. Why not just copy the list right from memory. Filter it down. Then refresh only when you need? As I said before I have done similar things and it is stupidly fast to do. Plus it gives me the benefit of easily caching certain bits and not ever having to check them again.

 

If designing mods such that they are more difficult to maintain and consume more CPU is the "right" way then I don't want to be right. ?

[kaxat]

@cbeyond8027 BTW I really liked your idea regarding factions and outfit themes. It won't be available soon enough to help you now. But I think it is a great idea to add in the future. Sorry I do not have a good solution in the interim.

[Corsec]

I released the SSE version of the Slaverun Comments submod, available in the link below-

Spoiler

 

 

This allows you to use the Slaverun Comments submod without needing to install Slaverun or Sexlab Sexual Fame Framework. NPCs will comment on nudity and any devices you or they are wearing. There is a third-party voicepack available.

[Mitrakara]

Die Mod Devious Devices SE letztes Update 2018 funktioniert nicht mehr mit der neusten Version von Skyrim SE AE daher kann man ihre Mod nicht  verwenden. 

[alex77r4]

11 hours ago, kaxat said:

...

 

That can be part of the problem. You have made your own tests ussing your own code and you get yours own conclusions but that not mean you can't be wrong.

 

Check a lot of conditions in CK can cause a hit in performance, specially when combine AND and OR conditions, as the CK Wiki say:

Spoiler

IMPORTANT: Keep in mind that when attaching conditions to "high-demand" items, like spells and magic effects, using too many conditions can result in a major hit to performance, and should be avoided. Checking whether one AND-pair is true in a set of more than three such pairs (beyond what is shown above) will quickly and exponentially increase the number of conditions that would have to be checked if distributed to account for OR-precedence. Try to find alternative methods if possible--like checking the necessary conditions in a script that can evaluate them more quickly and efficiently.

 

Sometimes can be better verify complex conditions in Papyrus but when we talk about normal conditions CK can made it 1000 times faster. I create a simply test for you.

Test_Papy.rar

 

Only have 4 Quest. One main to provide the spell and 3 Quest with 50, 100 and 800 Alias. All the Alias check the same conditions and i start and stop each quests 50 times.

I not notice any kind of stutering, frame drops, script lag or CPU ussage spike. So, start a quest plenty full of Alias is a trivial operation for the Game Engine.

I put a Papyrus code to make exactly the same as the Quest made and you can see how many time need Papyrus:

Spoiler

    1771    [11/22/2022 - 01:22:58PM] Test_Papy Time:4.125000 Average:0.082500
    1824    [11/22/2022 - 01:23:02PM] Test_Papy Time:3.312000 Average:0.066240
    1848    [11/22/2022 - 01:23:06PM] Test_Papy Time:3.382004 Average:0.067640
    1869    [11/22/2022 - 01:23:08PM] Test_Papy PAPYRUS Time:1.872002
    1931    [11/22/2022 - 01:23:21PM] Test_Papy Time:4.016006 Average:0.080320
    1987    [11/22/2022 - 01:23:25PM] Test_Papy Time:3.279999 Average:0.065600
    2004    [11/22/2022 - 01:23:28PM] Test_Papy Time:3.215996 Average:0.064320
    2021    [11/22/2022 - 01:23:30PM] Test_Papy PAPYRUS Time:1.615997
    2333    [11/22/2022 - 01:23:52PM] Test_Papy Time:4.015999 Average:0.080320
    2464    [11/22/2022 - 01:23:57PM] Test_Papy Time:3.694000 Average:0.073880
    2578    [11/22/2022 - 01:24:01PM] Test_Papy Time:3.424004 Average:0.068480
    2636    [11/22/2022 - 01:24:04PM] Test_Papy PAPYRUS Time:1.935997

 

So, a Quest only need 0.08 seconds while make the same in Papyrus need from 1.5 to 2 seconds.

You continue thinking is better have a script running for practically 2 seconds than start a quest in 0.08 seconds?

Edited November 22, 2022 by alex77r4

[kaxat]

@alex77r4  I will say at the outset I am impressed an 800 ref fill did not cause stuttering.

 

Your papyrus function will be heavily penalized by frame delays. It is also using getNthRef which is the slowest and least accurate way one can retrieve actors from the current cell.

 

For each actor that passes your condition there are 7 frame delayed functions. At 60 FPS each one of those function calls usually takes 16.67ms. It doesn't spend that time processing it spends that time waiting. Only a single function call gets executed in all that time because it waits until the next rendered frame. In total 7 * 16.67ms = 116.69ms.

 

From this we can deduce with a decent probability how many valid actors were in your cell when you did this test. There were 16 in the first test. Likely 14 in the next. This is why your test of 100 refs was notably faster than 50. And then there was likely 17 in the final one. Your script is spending 99.999% of its time waiting. Not using the CPU. If we bypass frame-delayed functions you will likely find that the execution time is near identical. If we throw in a bunch of checks for MCM properties and things we could procedurally preclude in a custom designed function then we start hitting the realm of where Papyrus might be faster.

 

If one wanted to do a proper test I would use GetActorsByProcessingLevel() to do a more faithful copy of the actor array. I would check all of the things you did but then I would cache the validity using SetIntValue(NPCActor, "ses_ActorValid" ...). And then on a subsequent check I would see if GetIntValue(NPCActor, "ses_ActorValid") == 1. Now retry that same loop and see if it doesn't complete in almost the same amount of time as your quest. That is what I am planning. It is more or less what I already do. And in my experience it is bloody fast. You can still get hit with a frame delay here or there simply due to Papyrus exceeding it's per-frame processing time. Which is usually 1.2ms for all scripts in your game. But again that is time spent waiting. Not time spent on the CPU.

 

But still I am impressed you did not notice stuttering from such a large ref fill. And it might be worth borrowing more CPU time to reduce frame delays. I wonder if there is not a way to get the best of both worlds. If the validity cache was a faction instead of an int stored in the co-save then one could still cache this data and prevent future reprocessing. But ideally there would be a way to clear this cache. At present I can easily use the StorageUtils function to clear all ses_ActorValid flags. Is there a good way to remove everyone in the game from a faction? I have never seen it if so.

 

There also might be a good way to reduce the number of refs needed. In particular starting a SexLab scene needs at most 5 refs. If I split the rest of the Enforcer off into its own quests and scripts then it might be possible to only gather 5 refs per scene. In your experience how random are ref fills? If I ran the same ref fill twice would it tend towards choosing the same actor if lots of actors are available?

[kaxat]

@Mitrakara Kimy is developing a new version with publicly viewable links on her Patreon pages.

Edited November 22, 2022 by kaxat

[kaxat]

@alex77r4 Thanks again for taking an interest and sharing a test script. Since you had already built it it made it easy to verify my proposed changes. Just did those proposed tweaks to the code + I merged my actual validation function in which includes more checks. Total processing time was on par with my predictions. I went to the most crowded city in my game which is Solitude. Waited for a busy time of day. With no cache 32 valid NPCs took 2.48s. With the cache it was very consistently 0.08s. +/- was .001s. Those frame delays will get ya.

 

That does validate my inclination to keep logic maintainable and inside of Papyrus. And I got to do a little testing with the GetActorsByProcessingLevel() to verify it works as expected. Which is great to do before diving down the rabbit hole of a major update. I do appreciate the script.

[alex77r4]

Of course all that are delayed calls and 99% of the time is wasted waiting the answer from the delayed calls.  The script is suspended in every delayed call until get the answer. But only the first call need a entire frame mainly because the object must be added to the serialization queue. The next calls are much more fast and not need one entire frame.

 

56 minutes ago, kaxat said:

If we bypass frame-delayed functions you will likely find that the execution time is near identical. If we throw in a bunch of checks for MCM properties and things we could procedurally preclude in a custom designed function then we start hitting the realm of where Papyrus might be faster.

 

Can you explain how you go to make that?

Make pre-processing not skip the delayed calls. Make a pre-compute and store the result in StorageUtil not mean that you can make the pre-process whitout delayed calls.

First, you must make a process with delayed calls and, later, you can access the stored results. But the first process is unavoidable.

Additionally, you must know that access a property in another script is a delayed call and any call to StorageUtil is a delayed call.

Access anything outside your script is a delayed call not mater if you call a function or access a property.

 

You insist in the use of GetActorsByProcessingLevel but let me ask two things:

That function need less than 0.08 seconds?

That function can filter the provided actors in any way?

In the best case GetActorsByProcessingLevel can need only 0.01 seconds and can save 0.07 secons but... is that enougth to justify their ussage? i think not...

Additionally not make any kind of filter. Simply give you an array of actor and you must manually verify each actor with ActorTypeNPC, IsDead, IsDisabled, Is3DLoaded...

How you go make that in less than 0.07 seconds?? Because all that verifications are delayed calls that are made instantly by the Quest.

 

1 hour ago, kaxat said:

But still I am impressed you did not notice stuttering from such a large ref fill.

 

Because the Game Engine is Multi-Thread and while some thread compute the Quest Objects another thread is computing the frame while another is processing scripts.

 

I strongly recomend not touch the factions. Is a very delicated area of the game and can cause a gigantic amount of problems. Yamete change factions and was one of their original problems. I must remade the entire faction process to ensure never fail. Because a fail in a system that change factions can break the entire game.

We not have a way to remove all actors from a faction other than delete the faction from the CK.

Inclusivelly, know what factions have one actor is a complex operation because, in Papyrus, we can't get a list with all the factions.

We only can know what factions have a specific actor. In others words, we can get a list of factions from one actor but we can't get the entire faction list from the game.

Additionally, any mod can add their own factions and identify the mod that own the faction is a bit complex.

 

I think we can't get a random list of references. Probably, every time we request a list of references we get a ordered list based in distance to PlayerRef. Simply because the game is made around the Player Character. The position of the camera is the position of the Player and the rendered cells are the cells around the Player and the rendered NPC's are the NPC's around the Player. A lot of things in the game are make around the Player. For that i think the list of actors is ordered by distance to Player. Because the Render Engine use that distance to compute the scale of the rendered NPC and the IA Engine use that distance to determine if must attack the Player of must attack other NPC.

Additionally, the NPC's have their own IA and move around the map and can open doors to enter their house or the inn. That mean, first, their can disapear, and second, their distance to the Player can change. Then request the list in a continous operation can show diferences caused by the own IA of the NPC's and show diferent results.

[alex77r4]

18 minutes ago, kaxat said:

With no cache 32 valid NPCs took 2.48s. With the cache it was very consistently 0.08s. +/- was .001s. Those frame delays will get ya.

 

With a Quest you can lower that time at least to half and probably can be below 1 second. That mean the cache can be unnecesary and you can remove hundreds of lines of code, simplify the process, use less memory, store less data and finally make the entire process much more consistent. Sometimes less is better.

[kaxat]

Good to know about the factions. That is unfortunate but what I suspected. Same with the ref fill randomness. If I had to guess I would wager its based on the same order GetNthRef() uses. I believe that's an internal function that SKSE is just exposing. I have seen other DLLs call it at the very least.

 

Quote

You insist in the use of GetActorsByProcessingLevel but let me ask two things:

That function need less than 0.08 seconds?

 

It usually takes significantly less. But that call was in the most crowded cell I could find. I could do a ton of grunt work building out all my conditions into a quest and then testing which is truly faster. But that would be exceedingly boring. And that is really the core point of all of this. I want something that's blinding fast and that is easy to maintain. Building conditions in the CK or xEdit is to me the worst parts of modding. So here I could spend all that time to likely prove my method is mere milliseconds faster. Or I could not and be happy. We are talking milliseconds either way. And it's a function that will run once on cell load and then occasionally refresh after that. Taking up to 0.08s is more than acceptable. ?That's what your own quest took at the outset with its simpler series of checks and probably fewer NPCs.

 

Quote

With a Quest you can lower that time at least to half and probably can be below 1 second. That mean the cache can be unnecesary and you can remove hundreds of lines of code, simplify the process, use less memory, store less data and finally make the entire process much more consistent.

 

I would disagree on nearly every point you made. Except that the caches uses slightly more memory. And SKSE co save space. Given the co save is compressed I'm guessing it uses around 3 bits per actor. Which again is totally acceptable to me. 3 bits is not worth severely complicating my mod. The only complication with the cache is when I need to invalidate it. And the invalidator is already built. It works. And it's not that complex TBH.

Edited November 22, 2022 by kaxat

[alex77r4]

1 hour ago, kaxat said:

I want something that's blinding fast

 

So, waste some time by creating a Quest in Ck and putting the basic conditions to fill the first alias. That's what I did this morning. Then duplicate the first Alias until you have 20 or 40 and use my code to access the Alias array.

The discussion here is not related to how long GetActorsByProcessingLevel takes because that function is practically instantaneous.

It's related to how many time need Papyrus to filter the array provided by GetActorsByProcessingLevel and, as you said, it needs around 2.5 seconds.

Because that array must be filtered by making delayed calls in Papyrus code. Of course some checks need to be done in Papyrus because we can't put ALL the checks in CK.

But by doing half of the checks in CK, with a Quest, you can cut the processing time in half or more. I think saving at least 50% of processing time is enough.

 

Of course, save that time need a bit of work in CK and transfer some of the logic from Papyrus to CK. But if really you want "something that's blinding fast" i not know why you not implement it.

[kaxat]

Without optimization it took 2.4s the first time. Realistically my mods validation routine will multi thread this and reduce much of the frame delay. Again already built. After that it takes at most 0.08s. That is fast enough.

I honestly don't want to waste my time. I don't have enough of it. Which is one of the main goals here. Making this easier to maintain. The blistering fastness is icing on the cake though. 0.08s is hard to beat.

[Leoosp]

@kaxat Has the modification SexLab Survival been taken into account with your Slaverun Enforcer - Standsalone? The reason being it's currently based off of the unmodified enforcer from Slaverun Reloaded, especially free women being naked. Your alterations throw this off, so may be worth alerting @Monoman1 to your alterations. Especially the ones where it's possible for free women to sometimes depending on them maybe be forced to wear specific clothes.

 

These alterations will affect the clothes license and freedom license of SexLab Survival, as they are different for your modification. Note though that SexLab Survival has an API utilising properties and mod events. So, these can be used when and/or if SexLab Survival is updated to take your modification into account.

 

@Monoman1 With your SexLab Survival modification and its licenses system is it possible for the API to enable the addition or removal of exclusionary items? In other words, so items from the outfit themes of Slaverun Enforcer - Standsalone can be excluded from needing a license, as otherwise a logic loop between the modifications would likely occur.

Edited November 23, 2022 by Leoosp

[gogostar01234]

Ive been trying to get this to work; The game loads fine, and slaverun works, but it isn't doing any checks. it was working fine before I decided to try to add some of the soft dependencies for slaverun, notably amputator beta and branding device of doom. I created a new save, but the message that showed up before doesnt and npcs arent getting stripped.

 

This is the loot setup i'm using

 

 

 

 

 

 

 

Edited November 23, 2022 by gogostar01234

[leakim]

Hey Kaxat, got a small question.

 

Took a look at the Changelog and I could not find anything on this.

 

Have you looked at the broken "slave reporting" functionality in the original Slaverun Reloaded? Or is this something you intend on looking at some point?

 

The idea of some sort of enforcement that tries to keep the slave in towns is really appealling.

 

Currently I am trying to "roleplay" this type of behaviour through use of POP and a few other mods to help me indicate whether or not I am "allowed" to try and escape.

Edited November 24, 2022 by leakim