Skyrim Modding Guide
NOTE: Some aspects of this guide is outdated, I'll go more carefully through it later but for now just ignore any mention of version numbers for SKSE, MO, Etc. Rest of it should be fine, there are new tools out like some crash fixes related to string count, but I have not tested these myself so can't fully recommend them until I do.
It's been a long way going, but after years of modding Skyrim I've finally reached a point where I not only can get a perfectly stable game with 200+ mods active at once, I can even help others get there. It's been a painful process of reinstalling the entire game at once, changing around on the computer's settings to make the game run better, and so on, and during that process I've broken every single vanilla quest in the game, from being unable to absorb Fus off the Bleak Falls wordwall, to just things in general failing, NPCs walking around naked or even headless, being stuck permanently bald, and so on. By now I've learned how to get rid of all of that, and so I decided to make this thread in other to help others possibly avoid some of the pitfalls I went through and still get a perfectly modded game.
First off a user guide to the guide! This thread has gotten bigger than I originally planned and may look daunting. Here's the only thing you need to do however, just read the FIRST PAGE of this thread, all the other pages are just general mod talk and bullshitting and troubleshooting people's problems. Read this page, then you're good!
Four important prefaces, first off while I have a lot of experience at this point I'm to no degree an expert, I still make mistakes, and if you spot me saying something bad in this thread please speak up and correct me, it'll only benefit all of us. Secondly, this guide is intended for someone looking for a complete makeover of Skyrim, not just someone looking to turn Skyrim into a pure sex game, if that's all you want you probably don't need much help and won't need many of the tips in this guide. Third, I'm going at this with only an idea on what works on my system, if you have a worse system these hints are probably not going to be good for you. On the other hand if you have a better system than me like a very new PC bought top of the line and overclocked, these tips are going to be highly necessary for you because Skyrim just isn't going to be able to handle your computer well enough without many of them. And fourth and most important, in this guide I'm not going to just tell you “do this and this and that and your game will work better, don't worry about it”, whnever I'm able to (as in whenever I'm not working off the assumption of having been told that by someone else!) I'm going to explain why you should be doing this, why you should use Mod Organizer, why those SKSE.ini settings should be that way, and why you should use an ENB. This way you'll be more informed when you decide to ignore me.
My computer is an Intel i7-6700k with 16 GB ram and with a nVidia GeForce 980 4 GB video card, and this guide will be written from the perspective of using that system. Note that an older i5, 8 GB RAM (or 6 if you somehow have that, try not going below that though), and a GTX 770, 1050, or equivalent lower card should work fine also, just make sure you don't overload your RAM, use * button on your Numpad with a ENB installed to check your FPS, keep it above 30 at all times and you should be good (that's the human eye's frequency after all, higher than that is mostly to smooth things out during rapid movement), and make sure to do CTRL-ALT-DEL before starting the game to bring up windows task manager, click "more details" and look at the RAM and CPU % use. Skyrim can never use more than 3 GB RAM at once itself (and unless you use 4k textures will rarely top 2GB even), while an ENB can potentially employ more, as well as properly use the VRAM in your graphics card which the game mostly won't use normally.
First off we're going to be installing Mod Organizer, and this is why; when manually installing a mod there's a good chance you're not going to be able to remember all of the changes you did when you six months later want to remove that mod, conversely when you install a mod using Nexus Mod Manager or Wrye Bash there's a good chance you have no idea what's actually being done and what's overwritten in the process. Even worse, the two problems can combine, Nexus Mod Manager is notorious for not rememebering what files it installed itself (this can happen if a file has been overwritten then reinstalled several times without the parent mod being reinstalled), which can lead to scripts and entire folders getting left behind in your game and causing problems for you even on a new save when you thought you'd gotten rid of a mod. Mod Organizer allows you a combination of the two, you get to install a mod through a manager that will remember what files were involved, you can do it the manual method and move files around during the install in the case of badly packaged mods, or you can use FOMOD automatic installers that some mods come packaged with. After you've installed a mod you can at any time go into it and manually disable and hide ESPs from the mod, or see what files it overwrites from other mods, or what files later have overwritten it, and easily move the priority of the entire mod (not just ESP) if you don't want that mesh to be overwritten after all without being forced to uninstall and reinstall mods.
The greatest benefit of Mod Organizer is however the Profiles ability, this allows you to have a full game Profile where you have all the stuff you use on your normal Main character, as well as perhaps a troubleshooting profile with only the bare necessity mods that you can swap to. Mod Organizer does this through the use of a virtual data folder, instead of dumping everything in your Skyrim/Data folder like all other methods do, it keeps each mod in a separate Mods directory, and when you start up the game it creates a virtual Data folder by combining mods together in the order you have specified, this means that you can have a fully modded Skyrim with everything replaced and aside from having put SKSE in your Skyrim folder your base Steam install of the game is completely virgin, including the Skyrim.ini and SkyrimPrefs.ini files which are kept on Mod Organizer, meaning at any time at all you can revert to an unmodded Skyrim and see if it works at all if you run into serious problems.
If I haven't convinced you by now I never will, so let's go on to installing it (or skip to the next post if you decide not to use MO after all); You have two options for installing Mod Organizer, the first is putting it in the base Skyrim folder (Steam/Steamapps/Common/Skyrim), the second is putting it on a different disc entirely. First off be aware that MO unlike NMM have to be installed separately for each game you want it to work on, so if you intend to use MO for both Skyrim and Fallout New Vegas for example you have to install it twice. If you install MO to the Skyrim folder then the benefit will be that it will immediately know what game it's been installed for and it'll be called Mod Organizer for Skyrim, otherwise you have to tell it what game you installed it for when you first start up. However if you happen to have low space on your main drive, like if it's an SSD drive for example, you may want to put Mod Organizer including the Mods directory on another disc entirely. This actually works fine and aside from a slow start up of the game as the virtual folder is being built you'll rarely really notice much difference, and you'll have a lot more space available for your mods. My Mods folder is now currently three times as big as I have room on my SSD after withdrawing the space needed for the operating system for example.
Download Mod Organizer from the Nexus here and the version you want is the 1.2.18, if the main page's version number updates to 1.3X something then change to that, for now the 1.3 versions while they add a lot of new functionality is just too buggy overall to deal with, and 1.2.18 works great still. Download it manually of course, and pick the Installer version, install it to either the Skyrim directory or a separate drive as said, and pick all the options form the install, including letting it handle Nexus links and put a shortcut, as you're going to for now use this program more than you will the game you're installing it for (that will change later on, using MO to install mods leads to much less work later instead of more). When you'll start it up you'll want to go through the tutorial it wants to show you, it'll make things easier.
Here's Gopher's tutorials on Mod Organizer, they're long but they give you a lot of basic information, including how to do things like SkyProc Patchers like those of Automatic Variants or ASIS, information that you can also use to make things like FNIS work for Mod Organizer. These videos also cover importing your mods from Nexus Mod Manager, I decided when I last installed Skyrim to start all over again with a fresh install and manually install all mods so I have not seen that and cannot provide help in that manner.