Modding Skyrim in Linux

[MadGenuis]

Okay, so I recently switched CPU and motherboard in my PC and on that occasion - in no small part to avoid having to pay for another copy of Windows - I installed Ubuntu. Thus far I've had mixed results trying to get things to work (after two decades on Windows, switching OS is pretty jarring), but after some getting used to it, most things seem to cooperate once I figure out how to poke them just right, if poking is even necessary.

 

However, as the title probably gave away, trying to mod Skyrim has been... let's call it a "challenge" that made me wish I had reinstalled and fully modded Skyrim to my liking before installing Ubuntu. Skyrim itself works fine (in fact so well that it ran even before I realized it was supposed to be run through Wine, though it does hang every time I try to quit, forcing me to kill the process. I can live with that, though (at least until killing the process breaks something)). I even installed SKSE and some of the most fundamental mods for other mods to work, like SkyUI and such, and things just worked. Then came Fores New Idles.

 

I've gone through and solved several problems with FNIS already, including but not limited to it having a meltdown over disagreeing with the location of the Skyrim directory, an ERROR(5) and just general weirdness, but I'm somewhat at a loss as to how to fix the warnings it gives now:

 

 

 

Skyrim 32bit: 1.9.32.0 - C:\Program Files (x86)\Steam\steamapps\common\Skyrim\ (Steam)

Skeleton(hkx) female: XPMS2HDT (115 bones)   male: XPMS2HDT (115 bones)
Patch: "GENDER Specific Animations"
Patch: "SKELETON Arm Fix"

>>Warning: AnimationList found in  "Animations\FNISBase", but no corresponding Behavior file. Incomplete mod IGNORED.<<
>>Warning: AnimationList found in  "Animations\FNISCreatureVersion", but no corresponding Behavior file. Incomplete mod IGNORED.<<
Reading XPMSE V7.2  ( 0 furniture, 0 offset, 0 paired, 0 kill, 0 chair, 164 alternate animations) ...

All Anim Lists scanned. Generating Behavior Files...
No GENDER directory male
 10 GENDER modifications for Animations\female
Alternate Animation mods: 1 sets: 16 total groups: 30 added file slots: 170 alternate files: 164

Create Creature Behaviors ...

 164 animations for 1 mods successfully included (character).
 2 Warning(s).

 

(I've only installed XPMSE and Pretty Combat Animations that interact with FNIS thus far, just to test it and get things working before embarking on a more ambitious undertaking.)

I've tried looking into those warnings myself, obviously, but the general consensus seems to be that it's simply caused by someone either not installing things right or running FNIS wrong, which I'm relatively confident isn't the case anymore. Doing a search here came up with little besides confirmation that there are people on here who appear to be playing modded Skyrim on Linux successfully, so I decided to ask and see if someone encountered and solved this?

 

My own best guess as to what is causing it would be the fact that Ubuntu differentiates between upper and lower case letters in file- and folder-names, causing duplicates to appear with some mods (I have a "scripts" and a "Scripts" folder, a "source" and "Source", and an "actors" and "Actors"... and those are just the ones I've noticed at a glance). I don't know if that could cause the issue or not, and if it does then how to fix it, but I thought I'd mention it nonetheless.

 

Thank you in advance.

[xwhiteninjax]

linux of today is mostly a 64bit-OS, skyrim is 32bit... so installing tools for skyrim like FNIS suggest installing the 32bit-version...

wine is tricky insofar, that it is able to use several instances at the same time (so some tools for skyrim might be used in a completely different session, even you think you are using them in one other)

 

so look out for another tool in linux ( PlayOnLinux)... this is like a layer for wine, but more easy to use and it has the advantage to install tools in the same session you have skyrim, so you are sure to have all working in the same)... and you are able to "tweak" the settings in a much more obvious way as in wine with winetricks or console-settings

 

even MO (the 1.3x version) seems to work with the most functions... so maybe you have more success using PlayOnLinux and playing skyrim

[MadGenuis]

I am running the 32bit version of FNIS already, so I have that much covered at least. Not entirely sure I understand the bit about tools running in different sessions, but has such an issue caused this warning before?

 

I have gotten PlayOnLinux a little while back, actually, because it was one of the things recommended for making modded Skyrim "sort of" work (the advice I've tracked down all seem to revolve around getting it to "mostly" work), though I've been slightly hesitant about rearranging things to use the PlayOnLinux directory, mainly because it seems to insist that the directory must be placed on my boot drive, which doesn't have space for games and such. Is there a way to convince it to place the directory on a different drive, or would it perhaps work to simply symlink things into the PlayOnLinux directory?

[xwhiteninjax]

as always you are able to use links to put games on other directories/partitions as the original ones (like .playonlinux inside the /home-directory)...

 

with some experiences you will see, that it is easy to tweak to your preferences...

 

I stopped playing skyrim with linux myself running into some problems I could not solve... like there seems to be a RAM-limitation inside wine-sessions like the win10-cup, ENB was not really working and the speed in gaming was much slower as in original winOS...

 

another way would be installing a winOS only for gaming and using programs like VMware (not free, but similiar programs exist and are free to use) for having a linux session running on the same box (and linux is running very nice inside "windows")... the other way (using linux as host and running winOS inside VMware or other programs is mostly cupped with limited graphic-functions - no direct access to the card) is able, too.. but speed and graphic is less than original playing in winOS

 

for example... games like witcher will do (the 2-first versions... but version 3 is very very slow)... so "elder-games" will work (like the ones using directX9), but modern games are tricky

[Warlock3000]

Yeah, Skyrim works fine in Wine, but making utilities such as FNIS generator and SkyProc work is tricky...

Perhaps it can be accomplished by installing some very specific libraries in Wine.

Or have a virtual machine with some Windows XP just to run these utilities when you need to (provided it has access to your Skyrim files to read them...)

[MadGenuis]

Okay, so recommendations are to either start over in a PlayOnLinux directory to maybe solve the problem, installing winOS and running it through that, or just giving up because it doesn't really work on Linux no matter how well you do it? That's... discouraging. Oh well, I'll try PlayOnLinux first and see what that gets me, and if that doesn't work... eh, I'll cross that bridge when I get to it.

 

I do wish I knew why that warning was coming up, though... may try simply symlinking everything in duplicate folders so that they can all be reached in every folder before moving it to PlayOnLinux, just to see if that is the problem.

[xwhiteninjax]

if you don't mind you can try "dualboot" (having enough of hdd-size)... linux is able to read NTFS-partitions (so you don't really loose place)... a working linux with many many programs needs around 30 - 50 gb

 

install winOS first, after this install linux (lookout for some howtos for your brand of linux)... give your winOS enough of place/partitions to have a nice gaming gig and use your linux for save working and other stuff

[Warlock3000]

Heads-up: Got "Generate FNIS for Users" working with the latest Wine version (3.8). Animations actually processed and working in-game.

Installing something via Winetricks probably helped, too.

SkyProc patchers probably just require installing Java in Wine (which is tricky, but doable), so that they can make use of Wine's "Windows" file paths.

[Warlock3000]

Test results -> Workarounds here contains the working tips on installing and using Java in Wine, useful for SkyProc patchers.

 

As of now, all utilities that I need are working in Wine (FNIS, SkyProc, TES5Edit). ENB running fine too (but didn't test any presets other than the default due to my outdated hardware). Wrye Bash works too, although I don't use it or any other mod managers.

[MadGenuis]

That's the most encouraging thing I've read about Skyrim on Linux since I started looking into it. I'll be sure to revisit the game and see about getting it to work now that I have confirmation that it's doable.

[Warlock3000]

Heads-up: In Wine's current state (with DXVK and stuff), it actually runs Skyrim SE much better than Oldrim. The frame rate is much smoother.

 

The downside is that you need to compile Wine yourself with a specific patch in order for SKSE64 to work.

But really, that just requires to follow simple online instructions and getting compilation dependencies via Package Manager.

 

PS: Mods still bork the performance somewhat, but no worse than in Oldrim. At least the camera movement is definitely smoother.

[Larixx]

What might work for Oldrim (as mentioned; the 32-bit application vs (generally) 64 bit distro's) is making an 32 bit wineprefix.

As far as I understand; an wineprefix is so to say an local installation of "windows" , so you can have an "32 bit Windows" ( 32 bit wineprefix) and an " 64 bit Windows"  (normal wineprefixes). 

 

I haven't tried it before, but it should theoretically work.

The most  'difficult' step is linking skyrim in FNIS. 

 

Manual (copying the files to the Data/ dir) should always work, however Mod Managers (MO, NMM or Vortex) are just tools to more easily revert to an vanilla skyrim, so not necessary.

 

More info about 32 bit wineprefixes:

https://www.google.nl/search?q=wineprefix+32+bit&amp;oq=winepre&amp;aqs=chrome.1.69i57j35i39j0l4.2263j0j7&amp;client=ubuntu&amp;sourceid=chrome&amp;ie=UTF-8

[daveylighter]

I used Windows 10 installed on an extra hard drive, but not activated, to mod Skyrim LE and do a few other thing I needed to do in Windows.  All that happens after a few hours in Windows is an overlay that says 'go to settings to activate Windows.'

 

I play the fully modded game using Ubuntu with Steam Play, and the only real issue is having to end the process after quitting the game.  The one thing I haven't figured out is how to get ENB working.  I've tried both the wrapper and injector versions, but to no avail.  While mods like Dynavision and Imaginator do a great job of improving the look of old Skyrim, I would love to get a better depth of field bokeh working on the game.

 

My Radeon RX 580 can certainly handle the graphics load.  I am currently using the latest Proton version 4.2-1, but since I also have Gallium Nine for DX9 games, I can use a modded version of Proton that will run the game 'native' DX9.

 

When using the 'wrapper' ENB, nothing happens, and the game loads and runs as if there's no ENB components in the game's directory.  If I use the 'injector' by launching it with my system WINE (wine3.0 ppa by Sarnex) from the game's directory (wine ENBInjector.exe), the injector program launches, but then the game will be shown as 'running' in Steam, yet the TESV.exe process never starts.

 

The lack of ENB is not a show stopper for me, but it sure would be nice to have, and by all indications it should be possible.

[ziaccer]

On 4/1/2019 at 3:48 PM, daveylighter said:

I used Windows 10 installed on an extra hard drive, but not activated, to mod Skyrim LE and do a few other thing I needed to do in Windows.  All that happens after a few hours in Windows is an overlay that says 'go to settings to activate Windows.'

 

I play the fully modded game using Ubuntu with Steam Play, and the only real issue is having to end the process after quitting the game.  The one thing I haven't figured out is how to get ENB working.  I've tried both the wrapper and injector versions, but to no avail.  While mods like Dynavision and Imaginator do a great job of improving the look of old Skyrim, I would love to get a better depth of field bokeh working on the game.

 

My Radeon RX 580 can certainly handle the graphics load.  I am currently using the latest Proton version 4.2-1, but since I also have Gallium Nine for DX9 games, I can use a modded version of Proton that will run the game 'native' DX9.

 

When using the 'wrapper' ENB, nothing happens, and the game loads and runs as if there's no ENB components in the game's directory.  If I use the 'injector' by launching it with my system WINE (wine3.0 ppa by Sarnex) from the game's directory (wine ENBInjector.exe), the injector program launches, but then the game will be shown as 'running' in Steam, yet the TESV.exe process never starts.

 

The lack of ENB is not a show stopper for me, but it sure would be nice to have, and by all indications it should be possible.

Been trying to get modded skyrim to run via steamplay and failing. How exactly did you go about doing this? Also when you say fully modded you also got stuff like FNIS and Bodyslide to work?

[RW311]

I had the same problem on ubuntu with skyrim process not closing, may try a different distro and see how that goes.

 

I think to get bodyslide and finis working you will need to run them through wine. I've only been using ubuntu for a few months in a laptop that lost it's hdd so modding using native windows apps is still a ways out for me to even mess with yet.  You can create a dual boot or use any windows pc to run those tools then copy the changes over to the linux install which is how I setup skyirm on ubuntu to begin with.

[ziaccer]

Well I have ragequit twice trying to just get mods running. Like how the hell do I manually install the XP32 thing?

[supashang]

Hi guys. I wrote a small guide for Skyrim "Oldrim" on how to set up a modding environment on Ubuntu Linux. Maybe you will find it useful. It uses Linux Steam + Proton. Mod Organizer, LOOT, FNIS, Wrye Bash.

I'm still looking into a solution for ENB though.

[xwhiteninjax]

On 10/7/2019 at 4:31 AM, supashang said:

Hi guys. I wrote a small guide for Skyrim "Oldrim" on how to set up a modding environment on Ubuntu Linux. Maybe you will find it useful. It uses Linux Steam + Proton. Mod Organizer, LOOT, FNIS, Wrye Bash.

I'm still looking into a solution for ENB though.

look out for this mod on nexus (  https://www.nexusmods.com/skyrim/mods/13049  ). Sure, it is not ENB, but the results are very similiar and without FPS-Loss and this mod should work with Linux... and just for me... have you tried mint-linux (based on ubuntu, some better tools as the original) ?

 

And there are some interesting tools for skyrim, not mentioned inside your nice guide (good work). Have you tried "merge plugins - standalone", "DynDoLod", "Outfit-Studio", "creation kit" and script-compiling for skyrim with the python-plus-compiler/notepad++ ?

 

And my trials with linux/PlayOnLinux (pol) were limited to RAM 4 GB (maybe my faults). Would be nice if there would be full access to the complete RAM on the GPU and or the motherboard.

[supashang]

Hi xwhiteninjax, thanks for the feedback. I wanted to keep the guide small, as to not intimidate those just wanting to get a modding environment set up.               

The steps covered in the guide should be adequate to get most people going. Maybe I will cover an advanced modding guide in future, but for now, I just wanted to lay down a solution for some of the common problems I've seen Linux users having with the most common modding tools.

 

Yes, I've used Linux Mint and a whole bunch of other distros over the years. My guide should cover Linux Mint too, since that is based on Ubuntu. I've also tried "Merge Plugins", but recently been playing with "Mator Smash".

 

I've not tried "IMAGINATOR - Visual Control Device for Skyrim", but I will certainly check it out and maybe use it on another Mod Organizer profile.

Thanks for the tips.

[ziaccer]

Thanks for the guide. Did you manage to get bodyslide up and running in that setup? I had use some hacky workaround with a windows VM in the past for it.

[supashang]

Yes. I have installed "Body Slide" to Mod Organizer, just like I would any other mod - then added the "BodySlide.exe" as a tool in MO by going to Data tab > CalienteTools > BodySlide > BodySlide.exe and selecting "Add As Executable". I can launch the tool from MO and it seems to work. I can also use the UUNP MORPHS tab in RaceMenu too. I don't use this tool much aside from tweaking body part scales and stuff, so I have not tested anything beyond that. Also, I have no experience in using the Outfit Studio, so that may be for someone else to tell you if that works.

 

Spoiler

 

[xwhiteninjax]

what about the "usable RAM" inside a wine-session for skyrim? the speed comparing to the same setting inside a win-os? That was always the gamebreaker for me, that even a light-modded-skyrim without ENB was as slow or even slower as a heavy-modded-skyrim with ENB and windows on the same hardware....

[ziaccer]

On 10/27/2019 at 4:10 PM, supashang said:

Yes. I have installed "Body Slide" to Mod Organizer, just like I would any other mod - then added the "BodySlide.exe" as a tool in MO by going to Data tab > CalienteTools > BodySlide > BodySlide.exe and selecting "Add As Executable". I can launch the tool from MO and it seems to work. I can also use the UUNP MORPHS tab in RaceMenu too. I don't use this tool much aside from tweaking body part scales and stuff, so I have not tested anything beyond that. Also, I have no experience in using the Outfit Studio, so that may be for someone else to tell you if that works.

 

  Reveal hidden contents

 

Thanks, ill give it a try once I have some time and mood.

[ziaccer]

Gave it a try, and found it not worth the hassle. I got MO to run and install some mods but the entire thing collapsed in on itself after a while. Skyrim crashes on load. Uninstalled some mods. works again. Skyrim randomly starts in windowed mode. Reboot. Skyrim works again. Installed some mods, Skyrim runs at 2 fps. Uninstalled some mods, MO throws a random error when starting Skyrim.

 

I think if i ever go back to Skyrim, ill use my hacky manual install method.

[supashang]

12 hours ago, ziaccer said:

Gave it a try, and found it not worth the hassle. I got MO to run and install some mods but the entire thing collapsed in on itself after a while. Skyrim crashes on load. Uninstalled some mods. works again. Skyrim randomly starts in windowed mode. Reboot. Skyrim works again. Installed some mods, Skyrim runs at 2 fps. Uninstalled some mods, MO throws a random error when starting Skyrim.

 

I think if i ever go back to Skyrim, ill use my hacky manual install method.

Hey @ziaccer. 2FPS?! It sounds like something isn't right with your system if that's the case. What are your system specs and distro? Do you have the necessary graphics drivers and Vulkan libraries installed? I get a solid 60 FPS here on my old GTX 970 and Intel 4790k. Other users have tested my method and reported back with positive results.