Fomm - Custom Build - 0.14.11.13

[RoyBatterian]

I would be interested to see any articles or instances of the LAA flag being unsafe.

 

That's excellent news on updating the load order system. We won't be forced to use LOOT I assume? The tool is primarily designed for Skyrim and is not entirely friendly with the Fallout games or mods for them.

 

I plan to completely re-write FNV4GB sometime this year to give it a more friendly interface for the options, and to make it automatically know which SteamAppId to use based on hashing the executable. I will also remove the Skyrim part of it as it's no longer needed, and perhaps make it into a library that can be included in other tools like FOMM and MO.

 

[prideslayer]

I've been using these updated versions of FOMM for awhile and I'm wondering if it's an issue on my end or something that's fixable but when I have a lot of mods in FOMM it slows to a crawl when activating/deactivating or simply moving ESP's around in the load order. I'm not sure how to explain it better but I'll try. I'll click on a box to activate a mod and it will freeze up for a minute or so, giving me the "Not Responding" thing and eventually the mod will check.

 

It could be my PC which is an AMD Dual-Core 2ghz, 4GB RAM... but I'm just not sure what it is and am simply seeking clarification.

It's not your PC, mine does that too, and it's (basically) brand new. 8 core Xeon with 32G of memory.

 

I haven't seen it get slow just from having a bunch of mods though, only when dealing with large mods (like TTW or even the sexout data pack) does it freeze for a moment. I haven't investigated why it does this at all -- but it's something even the old FOMMs have done since forever. The code is... heh.

[prideslayer]

I would be interested to see any articles or instances of the LAA flag being unsafe.

I can't point you to specific "articles", but I'm a programmer, and it sounds like you are as well (or you're VERY ambitious with your FNV4GB plan!). Setting that flag without changing the code is like slapping a 4x4 sticker on a 4x2 truck. The driver (Windows, in this case) will happily start off-roading with it. It might do fine, but it might run into a situation it can't handle that a real 4x4 could.

 

LAA itself tells windows it's safe to give the program allocations above the 2GB boundary, it doesn't make them safe for large addresses, any more than the sticker actually made the truck a 4x4. If windows does this, and the value goes into a variable that's a signed 32bit integer (rather than unsigned), the program will crash and burn if it actually uses the value in the variable since it will be interpreted as negative.

 

If unsigned ints are used everywhere, then it should be safe to just turn it on, but I haven't checked to see if that's the case with all the FOMM code. Once I've done that, I'll turn it on in the compiler.

 

I'd love to just build it for 64bit only but my last poll here showed 10-15% of people still using a 32bit OS to game on.

 

That's excellent news on updating the load order system. We won't be forced to use LOOT I assume? The tool is primarily designed for Skyrim and is not entirely friendly with the Fallout games or mods for them.

You won't have to use it no, but it will be integrated with it much like BOSS was integrated in the past. The only way to iron the bugs out of LOOT for FNV is to get more people using it and reporting them.

 

I plan to completely re-write FNV4GB sometime this year to give it a more friendly interface for the options, and to make it automatically know which SteamAppId to use based on hashing the executable. I will also remove the Skyrim part of it as it's no longer needed, and perhaps make it into a library that can be included in other tools like FOMM and MO.

Hah, maybe when you're done it'll actually work right for me. I've tried using it several times and what I end up with is always these two things:

 

1. The FNV4GB image stays in memory after I've quit the game. If I run it 10 times in a row and quit, I end up with 10 FNV4GBs to manually kill in task manager.

 

2. I've never seen the game use more than 2GB, so it's not taking advantage of it even when it's "running".

[RoyBatterian]

Yes I am a programmer or rather was, I am a bit rusty still as have not done it seriosly for years. Not sure what signed or unsigned integer has to do with Windows XP's (and below) horrible memory manager contraints since physical address extension has been present in x86 processors since the first pentium. Anyways that's a discussion for another forum, hehe. I was hoping you had some technical paper you could point me to.

 

Ah well I never liked BOSS or LOOT, as long as I'm not forced to use it that's cool with me.

 

As for FNV4GB I've tried to fix that hang bug already since MonochromeWench did not setup a WinMain properly but it seems to hang waiting for the terminate message from the debugged process. In fact it should detach and terminate immediately after creating the child process. It probably hangs because it's still hooking GetTickCount which I think interferes with Steam and NVSE. Enabling Fast Exit in NVSR gets around the problem for some reason... I don't have the source for that so I couldn't say why. I just need to rewrite it rather than try to patch it any further. It's a pasted together mess from various sources anyways.

 

[prideslayer]

Yes I am a programmer or rather was, I am a bit rusty still as have not done it seriosly for years. Not sure what signed or unsigned integer has to do with Windows XP's (and below) horrible memory manager contraints since physical address extension has been present in x86 processors since the first pentium. Anyways that's a discussion for another forum, hehe. I was hoping you had some technical paper you could point me to.

This doesn't have anything to do with PAE or the memory manager.

 

When you do something like a malloc, you store the result in a 32bit value. A smart programmer uses a DWORD / unsigned int, since that's the only type that makes sense. An old-school or lazy (or plain ignorant) programmer uses a normal integer, which is signed by default.

 

If you turn on LAA, windows WILL give it values above 0x7FFFFFFF. In a signed integer, the MSB being on means the value is negative. 0xA0000000 is a normal positive number (and potential result of some windows memory function to an LAA app) in an unsigned 32bit integer. Stuff that into a signed integer though and it's a negative value.

 

If something in the code is saying "if iThisNumber >= 0 { ...... }" then turning on LAA is going to randomly cause problems when that code is executed. 0x80000000 is > 0 in an unsigned int, but it's < 0 in a signed int.

 

So this is why it's simply not "safe" to turn on LAA on an executable that wasn't built with it to begin with. It's a compile time option for a reason, and not something that you can guarantee is safe to fiddle with in an alread-compiled executable.

 

Ah well I never liked BOSS or LOOT, as long as I'm not forced to use it that's cool with me.

Nobody is forcing anyone to do anything around here. I will take my fork of FOMM in whatever direction I think is appropriate (or just chuck it out the window, it irritates me whenever I work on it) but I do not forsee ever making BOSS/LOOT calls a requirement. That's just un-american.

 

As for FNV4GB I've tried to fix that hang bug already since MonochromeWench did not setup a WinMain properly but it seems to hang waiting for the terminate message from the debugged process. In fact it should detach and terminate immediately after creating the child process. It probably hangs because it's still hooking GetTickCount which I think interferes with Steam and NVSE. Enabling Fast Exit in NVSR gets around the problem for some reason... I don't have the source for that so I couldn't say why. I just need to rewrite it rather than try to patch it any further. It's a pasted together mess from various sources anyways.

I'm interested in seeing where you go with it, but just beware, it's as "problematic" as LAA is. There is nothing "safe" about telling windows that a program compiled and built for signed 32bit values is, in fact, using all unsigned values -- but that's what's required to represent any values above 2GB (2^31).

[RitualClarity]

not sure if this has been stated before or if it is just my system.

 

There is a strange thing with the new version just downloaded. Not sure if previous ones had it since I have last used the custom FOMM.

 

 

Sunday, January 11, 2015 - 12:12:55 PM
Fomm 0.14.11.10
OS version: Microsoft Windows NT 6.1.7601 Service Pack 1

System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\ProgramData\Microsoft\Windows\Start Menu\Programs\I.R.I.S'.
at Fomm.BackgroundWorkerProgressDialog.System.IDisposable.Dispose()
at Fomm.Games.WorkingDirectorySelectionForm.butAutoDetect_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

 

 

It happens when I use the auto find option for Fallout 3 or Fallout NV. Works when manually finding the file. Just an FYI if it hasn't been mentioned before (or if it was my system.. )

[prideslayer]

Looks like it probably doesn't like the periods in that IRIS directory name it came across. Goddamn lazy developers.

 

I'll look at it in a bit, but unfortunately for you, I *just* finished building a new beta version and am uploading it now. Need some testing on that before I move on.

[prideslayer]

New version in OP: 0.14.11.11

New in this version:

• Built for "any cpu". 7z 64bit DLL included.

This should put an end to large memory/mod issues for 64bit users. If you're using this on a 32bit version of windows, you will probably still have trouble, as I have not yet investigated the safety of enabling the LAA flag.

 

If you're using a 64bit version of windows however, then FOMM will also run as a 64bit app, and use the included 64bit 7zip dll, making LAA pointless.

 

I've kept this as a separate download for now, just in case -- it's the one named 'anycpu'.

[RitualClarity]

New one in the last couple of minutes? Great.

 

I see it is a program I am using that is blocking the scan. Cool. It was my end that was causing that problem. Anyway good info. Can't expect you to fix all the issues with lazy developers.

[KainsChylde]

In initial testing it certainly loading a lot faster. I run Win7 64 bit.

[RitualClarity]

It does seem a bit more "snappy".

(I also use Win7 x64)

[prideslayer]

[LukeDuke]

It's finally working for me!! Thanks!

Or maybe I also should thank Windows 8.1 Update 3 x64 which I recently upgraded to, from Windows 7 SP1 x64. 

 

Anyway, thank you so much, I think that FOMM is superior to any mod manager so it is important for me 

[RoyBatterian]

Prideslayer: While it's true that you cannot allocate a SINGLE buffer with a size greater than 2GB with a standard malloc (you can by other means of course, CreateFileMapping for ex), that in no way affects the amount of ram that can be addressed. This is entirely handled by the kernel's memory manager which can in fact address a great deal of RAM indeed. Additionally calling malloc with with a negative value will return NULL, also the value returned by malloc is inherently an unsigned integer, but of course you know this. Any coder treating it as a signed integer fails to understand the API (and architecture in general) and has no business programming.

 

I'm not sure where you get such FUD, but it should be disregarded as completely untrue. There are plenty of examples (msdn, gcc documents), papers and blog posts on this very subject which are but a google search away.

 

Additionally I certainly don't need a lesson in binary or hexidecimal, I've been programming on various platforms for > 20 years so let us not be condescending to each other.

 

Now, let me thank you for the updates and improvements to FOMM, it's great that someone has picked up the torch and continues developing it. I look forward to future improvements.

 

 

[prideslayer]

What?

 

You've completely missed the point that I was explaining. Perhaps I explained it poorly, but I promise you, turning on the LAA flag on an executable is completely unsafe if the result of a call to malloc or similar is being stored in an int.

 

Perhaps you don't need a lesson on hex, but you certainly need one on manners.

 

You're welcome for FOMM. Now get lost.

[RoyBatterian]

Apologies if I offended you, it wasn't my intent. I did get your point, "also the value returned by malloc is inherhently an unsigned integer" this is clearly stated in any description of the API I have seen. If some programmer is dumb enough to put it in an int, that's neither my fault nor yours. Please remain calm.

 

[prideslayer]

I'm plenty calm, but when you admonish that you don't need a "lecture on hexadecimal" or whatever you said to that effect, when I was simply illustrating, got my hackles up -- because you didn't seem to be "getting it" and asking for white papers and talking about the VMM when really it's simple logic and math.

 

In short, LAA tells windows that it's safe to return addresses > (2^31 - 1) to an application in any of the memory related APIs. This is not safe if the authors code is using signed ints and doing any kind of math on them, or just checking that the value is > 0. As for "how dumb" do you have to be -- the Win32 API does it itself. GetCursorPos will fail spectacularly on Vista (they fixed this in Win7) in an LAA flagged application for example.

 

Other API calls have special notes for abnormal behavior in 32bit LAA applications, like ReadFileScatter.

 

Or see here: http://stackoverflow.com/questions/2288728/drawbacks-of-using-largeaddressaware-for-32-bit-windows-executables

 

Chen's blog link in one of the answers leads further down the rabbit hole.

 

tl;dr: Blindly turning LAA on, on an application you did not write yourself and know to be LAA safe, may produce errors. Think about why the flag exists in the first place and why it isn't automatically enabled on every application the compiler produces.

[RoyBatterian]

I realize it's simple logic and math, I just failed to see what memory allocation had to with it when it returns a pointer (void). However the stackflow article explains the pitfall when comparing two pointers more than 2GB away from each other. Still it's coder error if an application fails, but that's neither here nor there as you said. I'll test New Vegas with the top down memory allocation then and if it causes issues I'll take it under advisement and rethink/investigate what needs to be done.

 

I'm not surprised VISTA had issues with anything, I skipped that OS because it was hunk of garbage from day one.

 

Anyways, I also don't mean to hijack your thread, so I'll "get lost".

 

[RitualClarity]

I have a question..

 

The "fommsetup" version is still the 32 bit program. and the "anycpu" version is the 64bit capable version.

 

The reason I ask is I need the 32bit version even though I am running a 64bit OS.

[prideslayer]

Why do you "need" the 32bit version? I intend to remove it before long...

[RitualClarity]

MO won't use 64bit. I don't need much in fact the original does work for my use. I just need a package manager, BSA extractor, creator and that is about it.

 

I use it for FMODs when MO don't work properly.. . That is those FMODs that are so complex to work manually. Mostly I have just been extracting the FMOD and manually installing. Works well. I can create a TTW install in about 2 1/2 minutes.. (skip the FMOD creation process and manually install )

 

I just like how snappy yours is over the older version from Nexus. The nexus worked just fine. I just wanted to grab the very last stable version of the 32bit. So the one that I "need" is the download on the left side of the OP correct?

[prideslayer]

Yes, though I don't intend it to be there indefinitely. I kinda thought I'd already removed it.. What happens when you try to use the other one with MO? It just refuses to run?

[zippy57]

I think it tries to run 64-bit programs as 32-bit programs or something. All I know is that if you try to run the Skyrim Performance Monitor through MO, something somewhere is 64-bit so it throws a fit and the textures go wonky.

[RitualClarity]

Yes, though I don't intend it to be there indefinitely. I kinda thought I'd already removed it.. What happens when you try to use the other one with MO? It just refuses to run?

 

This is what I get from MO when I try the 64bit version. I uninstalled and manually deleted all the files r/t the Nexus one and did a clean install of your current "anycpu" version.

 

If I install the one on the left side which I assume is still 32 bit capable if not totally 32bit it works like a dream so far. At least I don't get that error. I don't expect you to support MO in any way and for what I need I believe the older version works just fine with MO so I would be good to go.

 

the Nexus one so far works fine but spits up an error sometimes when working with the records which I don't remember getting with your previous versions.No records seem to be harmed in the process but it is anointing none the less. That is the reason for me to seek out the last 32 bit version of FOMM that you made.

[prideslayer]

You might be able to create a shortcut with some emulation settings that tells the anycpu FOMM to run as 32bit, I'm not sure, but honestly... this is MOs problem, not FOMMs.

 

Why is it trying to inject DLLs into FOMM anyway? Can you tell it to not do that? I don't know what it's trying to do, or why, but I'm not going to continue to maintain the 32bit-only version. I'll leave it in the OP for the time being but if people start getting confused about which one they should download or features are added that make it incompatible with the new one (like the new archive manager, when it is done), I'm going to remove the download.

 

You should contact the author and beg for a 64bit version of MO if you can't turn off whatever it's trying to do to the FOMM executable.