Download from LoversLab using MO

[AwfulArchdemon]

I am working on a plugin for Mod Organizer that allows people to download mods strait from LoversLab using MO. It's the same exact concept as the 'search nexus for more mods" option that comes with MO. I want to "search loverslab for more mods" by simply clicking an icon on MO's main menu.

 

The problem is, I don't know scripting well. I need to figure out exactly how to get MO to use it's default browser to go to LL, or just add another browser for LL. I can make the .exe to install the program to MO, and I can add the site path to the .exe, but I don't know how to get this thing to install mods directly to MO from LL.

 

This may be as simple as adding LL to the provided browser, but I doubt it.

 

Any scripting help would be greatly appreciated. Also, I'm not really sure how to add the icon link to MO's main menu. Anyone know how to make this plugin happen? This would be awesome, but I'd also need to know how to let MO know when updates are available, and what versions I'm currently using. I could probably figure all of this out on my own, but that would take forever.

 

Thank you.

[PsychoMachina]

Great idea, I always thought LL should have it's own mod manager.

 

Here's the source code.

http://sourceforge.net/p/modorganizer/code/ci/default/tree/

[AwfulArchdemon]

Great idea, I always thought LL should have it's own mod manager.

 

Here's the source code.

http://sourceforge.net/p/modorganizer/code/ci/default/tree/

Thank you. I am at a stand-still until a scripter comes along though.

 

Or, I can start learning right now, but I'd like to play a game or 2 on my day off work. I work every second of the weekend, so now's my chance to actually enjoy myself for a few hours. All day I just help people here, and work on various projects, so I need to relax for a bit. No doubt I'll be checking back here within an hour or 2 (it's inevitable), so if I see someone who knows what I don't, and they want to help, I'll jump right on it. This option is too good of an idea to pass up.

[AwfulArchdemon]

Crap! Uh, Ashal? Do I have permission to make this? It's a direct link to LL from MO, afterall. Really wish I'd asked before I started this request. Whoops! (sorry )

 

And no, I'm not about to upload this anywhere else (NEXUS???), so no worries there.

[PsychoMachina]

Maybe inform Tannin42, who knows, maybe he can help.

 

As for the app itself, from his Nexus page.

"Mod Organizer is released under a mixture of GPL (ui component, existing plugins, NCC) and LGPL (hook library and plugin interface)
Source code is available at: http://sourceforge.net/p/modorganizer

Uses Qt licensed under the LGPL
Uses 7-zip by Igor Wiktorowitsch Pawlow licensed under the LGPL (unrar restrictions apply)
Uses Icons from the Tango Desktop Project and the RRZE Icon Set"

[AwfulArchdemon]

@PsycoMachina: Thanks for all the info. Contacting him would make sense, assuming he's not gonna get mad at me the second I say Loverslab. You never know, he uploaded on Nexus. I always get wary of the things I say to people there, but it should be okay. He might even be able to whip this thing up real quick. Who knows. That would be awesome!

[AwfulArchdemon]

Alright. Message sent to Tannin. We'll see what happens.

I had planned on making an installer.exe for this plugin, but I'll worry about that once I've learned more about how to make this thing.

[AwfulArchdemon]

Permission granted!

 

I will leave updates for progress here, but it will be released on the 'Non-Adult Mods'' [Downloads] section. I will likely release it as an installer.exe. I put my name on the line for this installer.exe, so NO, it won't be a virus when it comes out. I refuse to let all .exe install programs be f***ing viruses! I want them all to do what the f*** they say they'll do, dammit!!

Enough of this installer.exe=virus bullsh*t. _{(*=ahhh...good-old LL censorship)}

[remois]

Topic followed

[AwfulArchdemon]

Excellent news everyone, but I'm going to get some more info before I let the cat outta the bag. I'll just say that all is going as planned. Have patience, people.

 

I have "live feeds from Tanni42", and I'll be linking him to LL via PM on Nexus. Just some good old copy/paste magic.

 

Regrettably, he will not be able to visit the Lab personally, but we've worked out a fun little communications system, so I'll be "passing notes" and personally working on this whenever I get the chance, but I'm reconsidering the whole 'plugin' idea, thanks to some advice I got from a friend. It may be a...OH! Right. Can't let the cat out of the bag without more info.  Sorry!

 

I'll keep you all posted. I may need a little help. but we'll see. I also may not. Anyway, I'll be back after sleep to help relieve the suspense.

 

 

Stay tuned!

 

AwfulArchdemon  

 

[AwfulArchdemon]

Hi Awful,

 

 

I'm not quite there yet, MO itself would need to be extended (although slightly) to do what you want.

I'm fairly busy getting the current feature set stable, but I'll help you as good as I can.

 

Here's a few vague pointers to give you an impression of what would need to be done:

 

In the current version (0.99.1) MO no longer includes its own web browser, downloads only work through the nxm links from nexus, the same way as NMM. If you want to provide that on LL, they'd have to add their own url-schema

 

However, you could check out the code revision 0.99.0 (or 0.12.9) to see how the integrated browser worked (file nexusdialog.h and related) and copy that to a plugin (best start with the ini editor and adjust it to your need). That browser contains code to handle regular downloads, currently assuming nexus url scheme..

In the current MO version you can access the download manager from a plugin (IDownloadManager) and start a download using an url (IDownloadManager::startDownloadURLs)

Thus you could display a browser that automatically opens the ll page (or its download page). Using plugin settings you could have the user save his username/password (currently only clear text) and thus automatically log him in (depends on the protocol) using a http request (again code for that exists in one of the nexus... classes) and then start downloads in MO.

These downloads don't have a version or a back-link to ll yet. No update feature or "visit on ll". For that we'd have to extend MO: Allow "startDownloadURLs" to be passed custom data to identify the mod on ll (some id, maybe the forum thread id), save that data with the download, pass it on to the mod-meta-data on installation.

Then we'd create another hook so "visit on x" plugins can be written and have that plugin use above custom data to form a url to ll and open that with ShellExecute. That provides "visit on x".

An update feature we probably can't provide without support from the LL page itself, we'd require some form of api there to query "what is the current version of the mod identified by 12345"

 

Feel free to ask further details..

 

Best regards,

Tannin

 

 

 

btw, requests and issues can be placed on Tannin42's issue tracker: issue.tannin.eu/tbg

[AwfulArchdemon]

I will not be releasing anything for the current version of MO. This will be done on the Beta version. It will not be Beta for much longer, and this is the version I'll be using from now on.

 

please use the beta, this is where all my further development is going to. Once it's stable enough I'll remove the beta label.

 

 

[lampuiho]

I tried working on it but it uses QT4. It was way too troublesome to even get it compiled....

 

And I wouldn't exactly calling it scripting. You have to know C++ and how QT handles dll plugins.

[AwfulArchdemon]

Okay, to make this clear, the only reason I mentioned scripting is because I was considering making this an installer.exe. I got it now though. Hopefully, this will clear the whole 'scripting' thing up.

 

Thank you for your reply, lampuiho.

 

There is always a way. If MO happened, why can't this happen?

 

I thank you very much for taking a shot at it. This can be done more than one way, but no matter what, it'll be complicated. I have to gather my next data feed before I can speculate further. Thank you.

 

Translation: I'm waiting for Tannin's PM. I don't have all the answers. At the worst case scenario, we'll be able to download LL mods from MO. I can live with that 'consequence'. It was really all I wanted in the 1st place.

 

I'll be back with the latest on:  Download from LoversLab using MO.

[AwfulArchdemon]

 

Quote
The answer to "How would I get MO to receive info on updates and new versions on both Nexus and LL?" you won't like:


The thing is: Nexus provides an programmable interface to query information about mods. If you request mod information from nexus from a tool, you get a response in a well defined, json-encoded (google json if necessary) structure that contains mod name, mod version, file versions, author, ....

I assume LL does not because it would have to be programmed first. So if you wanted to programmatically receive updates/mods from LL you would have to download the html site and parse that to retrieve the info you need.

That's a very complex problem if you want to do it right. And if the webdesigner of LL decides to change the page layout -> your parser breaks.

Best case scenario: your update check suddenly doesn't work. Worst case: Your plugin suddenly interprets the wrong data and garbles the info users already had (There is an update of your mod from version "1.2" to version "banana").

Or it crashes MO.


What I'm saying is, without trying to advertise: To my knowledge Nexus is the only page that provides a reasonable and accessible mechanism to manage mod versions from a tool.



The "search LL for more mods" I'd really implement using an integrated browser. Direct the user to the LL page, let him browse, capture the downloads.

Otherwise the same applies as above: You'd need to parse the page content which is a moving target.



So far I always assumed that one mod either originated from Nexus or from LL, clearly made distinct by a flag.

If you want to fulfill the request, you'd either have to assume that nexus and ll use the same versioning scheme (a bold assumption) and just take the newest version no matter where it comes from OR have to user choose where to take the mod from and ignore the version from the other source.


Best regards,

Tannin

 

I suppose I need to come up with a slightly new plan. I'll need to learn all I can about LL's layout, and even then, I'm not sure we'd be able to get version updates. I'll be happy just to see MO dl a mod strait from LL, so at least we have a quick way to get new versions by simply browsing for them. That will be a nice start. As I thought, I could use some help. This is going a bit beyond my knowledge, and Tannin doesn't have any of the info he needs to put LL on the map. If I can add the option to add LL dl'ing capability to MO...WITHOUT touching Nexus' ability to do the same, then we'll be onto something here. Until I learn more, I can't make any promises as to when this will be made, but it doesn't seem impossible to at least make it possible to dl mods from LL, though that may take a lot of maintenance. We'll see. I'll be back with more.

[AwfulArchdemon]

We'll just start with the basics here then. I want a way to Download mods from LoversLab using MO, and that's what I intend to do.

 

Yeah, I'd start with browsing LL and downloading. I think this would already be helpful and afterwards we can still think about extending that functionality.

 

MO used a browser in versions before 0.99.2 but I removed it.

 

But it wouldn't be much work to re-integrate it for use with other pages + the LL plugin would have to ship with the QtWebkit.dll because that is required.

 

 

That's the latest news. Stay tuned MO lovers!

 

[Molevalence]

I support this. It's a shame MO looks way to advanced for me.

[AwfulArchdemon]

Okay, this'll be a plugin. It may evolve, or just be remade into something bigger later, but for now, it'll be a plugin.

 

I've been thinking, and if this thing really works well, I'd see no reason why we can't rig it to know when LL's mods from it's download pages makes a change, ie. get's a new version. Any change at all on that page would very likely signify a new version being posted. Thus, MO could treat all changes on the mod's dl page as a new version, but I'm not sure how MO would know which version it is. Perhaps a save option. One that remembers the name of the previous mod version, and lists anything else posted there as a new version, and alerting MO, resulting in the familiar red/green indicator to tell us that we have a new version available.

 

Do LL mods have ID's? Nexus mods are found through MO through the use of their ID's, so if we have that system here, and I just don't know it, we can use those the same way we do for Nexus, assuming they're not too similar (thus getting LL confused with Nexus).

 

At any rate, being able to dl from LL is a nice gateway to plenty of possibilities. 

[WizzyD]

Okay, this'll be a plugin. It may evolve, or just be remade into something bigger later, but for now, it'll be a plugin.

 

I've been thinking, and if this thing really works well, I'd see no reason why we can't rig it to know when LL's mods from it's download pages makes a change, ie. get's a new version. Any change at all on that page would very likely signify a new version being posted. Thus, MO could treat all changes on the mod's dl page as a new version, but I'm not sure how MO would know which version it is. Perhaps a save option. One that remembers the name of the previous mod version, and lists anything else posted there as a new version, and alerting MO, resulting in the familiar red/green indicator to tell us that we have a new version available.

 

Do LL mods have ID's? Nexus mods are found through MO through the use of their ID's, so if we have that system here, and I just don't know it, we can use those the same way we do for Nexus, assuming they're not too similar (thus getting LL confused with Nexus).

 

At any rate, being able to dl from LL is a nice gateway to plenty of possibilities. 

 

maybe a md5sum check, if its bigger in dl size compared to the last one or a date modified history... idk

 

or if we were granted access to log files on the file server for upload times and dates maybe. just add a script that can read the read only file for those criteria and push the info to another file.

 

or something lol

[CGi]

Actually, you can reuse the iD system from MO, as LL appends iDs as well.
Saves you some coding time, as you just need to extend it with the mods name, that's used on LL (use the meta.ini and the <ModName>.meta files to save it).

But as the update link is hardcoded to use the Nexus, you got to include a new function or build a switch into the existing one (what i suggest).

 

The problem i see, is that LL has no versioning system for mods, so you need to crawl thru the changelog on the mods download page and hope that the author uses valid version numbers or dates (MO supports dates as version number). Ashal f.e. works with the same versioning system, that MO uses and that does not utilize the build part of a 4 octet version number, but others do it differently.

But this again means more traffic on the LL servers, as you got to download the whole page (or only the whole changelog, if this is possible [never got this far]) and more time to parse the received data.

You better talk to Ashal first, about a mod system solely for your plugin, that only serves the needed data, like last version and whatever else you might want to work with.
Maybe even a system that only uses iDs (like the Nexus), to identify a mod. Less traffic and reduced parsing time.

 

My ideas, that i never really extended further, was to send all mod iDs including versions to the LL server and he replies with a list of mods that are updated, containing the mod iDs and version numbers. Would free MO from crawling pages and contacting the LL server over and over again. One big request and one big answer. Or LL serves a simple textfile, that's available from a fixed dl-link, that your plugin can download, parse and display the relevant information. The text file would be updated by the LL server, if necessary and contain a version of itself, so your plugin doesn't download it, when it's not needed.

 

Leaves only the recoding of the update request functions, so that MO uses your plugin for LL mods.

And changing the times, the request is done, in case you get one big answer from LLs servers. The textfile method would save you some hassle here, as you could simply limit the update request thru your plugin to once every 30mins or so, by reading the creation date of the modlist file and updating the creation date, in case a new request was done, but no updated modlist downloaded.

[AwfulArchdemon]

CGi

Actually, you can reuse the iD system from MO, as LL appends iDs as well.

Saves you some coding time, as you just need to extend it with the mods name, that's used on LL (use the meta.ini and the <ModName>.meta files to save it).

But as the update link is hardcoded to use the Nexus, you got to include a new function or build a switch into the existing one (what i suggest).

 

The problem i see, is that LL has no versioning system for mods, so you need to crawl thru the changelog on the mods download page and hope that the author uses valid version numbers or dates (MO supports dates as version number). Ashal f.e. works with the same versioning system, that MO uses and that does not utilize the build part of a 4 octet version number, but others do it differently.

But this again means more traffic on the LL servers, as you got to download the whole page (or only the whole changelog, if this is possible [never got this far]) and more time to parse the received data.

You better talk to Ashal first, about a mod system solely for your plugin, that only serves the needed data, like last version and whatever else you might want to work with.

Maybe even a system that only uses iDs (like the Nexus), to identify a mod. Less traffic and reduced parsing time.

 

My ideas, that i never really extended further, was to send all mod iDs including versions to the LL server and he replies with a list of mods that are updated, containing the mod iDs and version numbers. Would free MO from crawling pages and contacting the LL server over and over again. One big request and one big answer. Or LL serves a simple textfile, that's available from a fixed dl-link, that your plugin can download, parse and display the relevant information. The text file would be updated by the LL server, if necessary and contain a version of itself, so your plugin doesn't download it, when it's not needed.

 

Leaves only the recoding of the update request functions, so that MO uses your plugin for LL mods.

And changing the times, the request is done, in case you get one big answer from LLs servers. The textfile method would save you some hassle here, as you could simply limit the update request thru your plugin to once every 30mins or so, by reading the creation date of the modlist file and updating the creation date, in case a new request was done, but no updated modlist downloaded.

 

 

 

That's what I said...

 

 

 

And now, a special message to 'AdBot' below this message.......

[Thulas]

Hey, that's the meal you once gave me. I want it back!

 

Anyway; I actually have nothing to say, so..."go, go, Archie"? Oh, and take this like as useless support.

[AwfulArchdemon]

AdBot gave me $5 for it. Sorry man...you know how it goes...

[AwfulArchdemon]

The latest from Tannin:

 

I did do some work on the integration of non-nexus pages. In my code repository on sourceforge there is now a new branch (1.1) which contains a plugin for downloading from the tes alliance page.
The actual plugin is only around 150 lines of code, the main changes are in the core application so a LL plugin would basically be a copy&paste and changing a couple of links and regular expressions.
This is of course WIP and not yet a full solution. It only lets you open tesalliance in the integrated browser (which stores login-cookies) and handles download links from that page and stores mod and fileid as known to tesalliance but it's a start.
 

 

It's getting there.

 

@CGI:

 

No word from Ashal yet. I can live with a plugin for now. I guess I'll just have to wait and see what happens for now. If only there were an easier way...

[nagothm]

Just wondering if you have made any more progress on this Awful,

 

Got all excited before but haven't heard anything in a bit.