BSAs versus loose files

[Carabosse]

I've searched online and never found a satisfying answer one way or the other when it comes to the merits of either using loose files or archiving everything within a BSA. I personally prefer to use a BSA because I find it neater. One issue I understand is not having your files overriden by loose files from another mod but that's extremely unlikely to occur for my mod.

As my mod has grown, so has the number of scripts contained within it. At present there are over 400 script files, about 300 hundred are dialogue fragments the rest are tied to quests, aliases, objects or are utility scripts. The dialogue fragments are 99% made up of setstage calls as I don't trust the skyrim conversation system not to bug out at any given moment and would like the player to be able to replay those conversations when it does and have the quests carry on as normal. The other script files are of varying complexity.

Should I be worried from a performance perspective by having this ever increasing number of scripts in a BSA? I've never personally ran into an issue where a script hasn't run but both of my testing machines are relatively new and performant.

When I was searching for information about this, all I found was a lot of opinion and no real facts.

[pinky6225]

You may have already seen this but if you haven't http://forum.step-project.com/topic/5033-ramifications-of-bsa-extraction-in-mod-organizer/

 

personally as i don't use MO i prefer mods in BSA format instead of loose files as it makes them easier to remove so their is less chance you will mess up your install

[Carabosse]

Thanks for the link. I read it and learned a few new things from it. It didn't answer my real question which is whether there is a perfomance penalty or some kind of limit (manifest by undesired gameplay effects) to having scripts within a BSA.

 

When I mentioned neatness, that is as a primary concern regards the user. So that they may cleanly, and easily, remove the mod.

[Khe]

BSA is nicer for clean in/out, and is recommended for non-modders. however, you can't edit scripts or anything else in them, so for people who want to fix errors (if the source is included), a BSA isn't as desirable. mods should always be packed with them though, as it's easy to unpack them, but not as neat to install w/o them.

 

mostly it all boils down to filesize and convenience.

[Carabosse]

BSA is nicer for clean in/out, and is recommended for non-modders. however, you can't edit scripts or anything else in them, so for people who want to fix errors (if the source is included), a BSA isn't as desirable. mods should always be packed with them though, as it's easy to unpack them, but not as neat to install w/o them.

 

mostly it all boils down to filesize and convenience.

 

Preferences regarding ease of use for modders and non modders is noted, though it has less to do with my enquiry.

[Serithi]

BSA is nicer for clean in/out, and is recommended for non-modders. however, you can't edit scripts or anything else in them, so for people who want to fix errors (if the source is included), a BSA isn't as desirable. mods should always be packed with them though, as it's easy to unpack them, but not as neat to install w/o them.

 

mostly it all boils down to filesize and convenience.

If you have a .bsa extractor (and packager so you can remake them), then you can edit the scripts just fine, it's just a little more hassle.

 

I prefer .bsa's simply because they're neater, not to mention faster to load than loose files. Though obviously there's the issue of the 255 limit, i still try to keep as many mods in .bsa form as i can.

[Carabosse]

 

not to mention faster to load than loose files.

 

 

On what do you base this assertion?

[foreveraloneguy]

I've heard BSAs are faster as well, but have yet to see even one piece of objective data proving that.

[iggypop1]

Sacramenas modding page here on LL http://www.loverslab.com/topic/43043-skyrim-modding-guide/. He (as i remember) allways unpack  BSA . Performance thing.

 

mail him.....

[Carabosse]

Might you be kind enough to point me towards the specific part of that enormous wall of text which justifies unpacking the BSAs?

[yatol]

I've heard BSAs are faster as well, but have yet to see even one piece of objective data proving that.

 

why not test it and see for yourself?

 

 

in morrowind time, harddrives were much slower

loading 600 mo as loose files in your ram was longer, than loading 300 mo from a compressed bsa your cpu would decompress

harddrives are much faster now, wasting cpu power to decompress bsa into your ram is probably worse for your performance

that's why my daz.bsa is 1.5 go, and the folder is 1.8 go, that bsa is just a folder put into one file, it's uncompressed

 

 

uncompressed bsa are better than loose files, because that bsa is 1.5 go, and the folder is 1.8go

with loose files, if i equip angeloi, there's 72 mo of nif in my ram, with bsa, there's 31 mo

because my armor_1.nif are the same as my armor_0.nif

 

npc mods are nice to see that

loose files are 1.6 go, bsa is 650 mo

you wander in whiterun with lydia, you meet ysolda, and there's aela near

with loose file, that's maybe 300 mo in your ram

with bsa, that's maybe 150 mo in your ram (bsa check doubles, y is the same as x, y is a link to x, in game, you only load x)

 

when mod organiser load the game, it create a virtual data folder

if you have 3dnpc, open the folder and take a look at the sound folder, there's 40 000 files in it

put that into 2 bsa, now you are loading the game 1 second faster (maybe), because there's 39 998 less files to put in the virtual data folder

[pinky6225]

Thanks for the link. I read it and learned a few new things from it. It didn't answer my real question which is whether there is a perfomance penalty or some kind of limit (manifest by undesired gameplay effects) to having scripts within a BSA.

 

When I mentioned neatness, that is as a primary concern regards the user. So that they may cleanly, and easily, remove the mod.

 

from what i've read performance is better with stuff in BSA over loose file but the gains are very marginal so on a good PC with a SSD etc i doubt it you would notice much of a difference

 

Their might be a performance gain just from having a cleaner install (no dead scripts kicking around spamming papyrus for instance) but measuring that would be hard and it would be down to how thorough the user is since someone who manually installs loose files but makes sure to remove them all is going to be just as clean as someone using a BSA

 

[CGi]

BSA with good compression:

+ faster load time as less data needs to be read;

+ less stress for SSDs as less data needs to be read;

+ less stutter when loading new assets, as less data needs to be read;

- the file is read-in compressed and then uncompressed in RAM, which adds to the overhead;

- more RAM usage depending on the file size of the used assets;

- less stability as Skyrims engine has a problem with to many BSAs;

 

BSA with no compression or unpacked:

* The opposite of what's written above.

 

 

P.s.: Most mods use no or next-to-no compression for their BSAs, so most mods act like loose files performance wise.

Skyrim itself uses medium compression.

[foreveraloneguy]

 

I've heard BSAs are faster as well, but have yet to see even one piece of objective data proving that.

 

why not test it and see for yourself?

 

 

in morrowind time, harddrives were much slower

loading 600 mo as loose files in your ram was longer, than loading 300 mo from a compressed bsa your cpu would decompress

harddrives are much faster now, wasting cpu power to decompress bsa into your ram is probably worse for your performance

that's why my daz.bsa is 1.5 go, and the folder is 1.8 go, that bsa is just a folder put into one file, it's uncompressed

 

 

uncompressed bsa are better than loose files, because that bsa is 1.5 go, and the folder is 1.8go

with loose files, if i equip angeloi, there's 72 mo of nif in my ram, with bsa, there's 31 mo

because my armor_1.nif are the same as my armor_0.nif

 

npc mods are nice to see that

 

 

 

 

loose files are 1.6 go, bsa is 650 mo

you wander in whiterun with lydia, you meet ysolda, and there's aela near

with loose file, that's maybe 300 mo in your ram

with bsa, that's maybe 150 mo in your ram (bsa check doubles, y is the same as x, y is a link to x, in game, you only load x)

 

when mod organiser load the game, it create a virtual data folder

if you have 3dnpc, open the folder and take a look at the sound folder, there's 40 000 files in it

put that into 2 bsa, now you are loading the game 1 second faster (maybe), because there's 39 998 less files to put in the virtual data folder

 

 

Protip: spoiler tags are awesome.

 

I don't test because I don't care. And even if bsa's were slightly faster, I still wouldn't use them. In fact, I extract pretty much every bsa simply because I then go about further customizing the mod to suit my tastes, work better with my ENB, etc. And MO is a non-issue for me. After reading the docs and FAQ, I'm staying as far away from that thing as I can, so no copying to the magic virtual directory for me.

 

I'd still like to see objective benchmarks showing whether BSAs are, in fact, any faster than loading loose files on relatively modern hardware though.

[foreveraloneguy]

BSA with good compression:

+ less stress for SSDs as less data needs to be read;

 

Ah, this is blatantly incorrect. Reads on SSDs are free. It's the write/erase cycle that damage the cells. As far as the rest, again, unless some objective benchmarks can be posted, I'm going to write off all of this as "community wisdom", which also brought us the brilliantly bad idea that we should set papyrus to use 4 gig of ram.

[yatol]

 

I don't test because I don't care.

I'd still like to see objective benchmarks showing whether BSAs are, in fact, any faster than loading loose files on relatively modern hardware though.

 

 

skyrim bsa vs loose files benchmark in google... first response... and it's in this site...

[CGi]

Ah, this is blatantly incorrect. Reads on SSDs are free. It's the write/erase cycle that damage the cells. As far as the rest, again, unless some objective benchmarks can be posted, I'm going to write off all of this as "community wisdom", which also brought us the brilliantly bad idea that we should set papyrus to use 4 gig of ram.

i guess, i should have been more specific.

 

i was not refering to the lifetime, but to the hard limit of operations and transfered data an SSD is allowed in a specific amount of time (usually meassured per day), before it resets itself and may need a system restart. My old OCZ even needs a cooldown time, before it reports itself to the BiOS again after the limit is reached.

This ofc differs between SSD types, manufacturers and the used controller. An SSD on a real Raid controller (means one with own RAM and battery to ensure data is written) is hardly influenced by any of this, but people that have such an (exensive) setup are rare and in most cases not gamers.

[Sacremas]

Might you be kind enough to point me towards the specific part of that enormous wall of text which justifies unpacking the BSAs?

 

Sorry for showing up months late!

 

You unpack BSAs due to the amount of times they need to be read. This comes from uglykidcid who knows the technical limits of it more than me.

 

Say you have Skyrim - Textures.bsa, Unofficial Skyrim Patch.bsa, and Vivid Landscapes.bsa.

 

The game loads Skyrim, loading the textures first unpacking it. Then it loads the patch, unpacking the patch and unpacking the textures bsas again (both of them, nope it doesn't reference what it unpacked the first time). Then it loads Vivid Landscapes, unpacks it, views it, then loads skyrim textures, unpacks both of them and compares it to vivid landscapes, then it loads unofficial skyrim patch, unpacks is and and compares it to vivid landscapes, and then it unpacks again textures.bsa, unofficial patch.bsa and vivid landscapes.bsa and compares them all together. Turn a corner where there's a new texture... and the whole process happens over again because the game only loads whatever you're looking at at the time and unloads the rest when it comes to textures and meshes, hence why you get massive lag if you stand still in whiterun and spin around.

 

Now imagine you have 8-15 mods that edit the same thing, like details out of Dawnguard.bsa (and groan as you consider mods that edit both dawnguard, dragonborn and skyrim things that are the same as mods that only edit some of them... yeah the whole chain need to be loaded over agian, exponentially), and you start to see maybe where this is an issue.

 

If on the other hand you'd unpacked all of them and put them in the same directory (or have MO put them in a virtual directory), all the textures are read once, when you look at it, then some other textures are loaded, and your massive lag on spinning is gone mysteriously.

 

However if no other mod is referancing your mod and your mod is not referencing anything but loose files, then a correctly compressed BSA will work better. I look forward to seeing such a wonder, until then I will be unpacking all my BSAs..

 

Well the textures at least, unpack voice files and they drop in volume drastically, unpack animations in the base game and FNIS can't read them, unpack meshes and the game reference a mesh that's now gone awol, and you CTD, unpack textures and the game doesn't find it... and your game runs faster because there's less to load.

[Sacremas]

You unpack the following things;

 

Skyrim - Textures.BSa, then run a optimizer through it to optimize the textures and make them run better.

 

All the DLCs, then do the same on the textures, delete the rest that's unpacked, and unfortunately keep the rest of the BSA due to the voices, animations and meshes. As long as you don't have several other BSAs accessing it, Dawnguard BSA will run fine, when another mod changes graphics in Dawnguard the game will just compare Dawnguard.bsa with your loose files. Even if you have textures in both Dawnguard.bsa and your unpacked and optimized textures, loose files always overwrite BSAs.

 

All the mods, especially the unofficial patches. FNIS issues, missing meshes, etc doesn't apply to mods, if a mod mesh is missing Skyrim will just use what that mesh replaced. Possible exception to this is big New Lands mods like Falskaar, Wyrmstooth, etc, they make a minimum amount of references to and changes to the base game, and so you won't have much of an issue, as long as the BSA is well made (and assume as such with something like Falskaar) then it's safe to keep. Anything that modifies the main skyrim world however, unpack it.