Mantella + Skyrim can be quite an experience

[LordDragon]

On 5/9/2024 at 11:39 PM, JimUpdating said:

I was going to suggest that apropos2 would make an interesting text feed. Otherwise I haven't seen much of world events filter through, other than location (shop, city, tavern).

 

Would it be possible to get a copy of the patch?

Sure, here it is in compiled format. Put it in the Apropos2's Script folder and you should be good to go.

Apropos2MessageWidget.pex

[Algene]

On 5/6/2024 at 4:06 PM, branmakmuffin said:

I am not criticizing you or the mod you're promoting, this is just a comment about AI.

 

When AI understands irony and sarcasm, I'll be satisfied with AI. AI will never, in my opinion, understand irony and sarcasm (if there are people who don't understand irony and sarcasm, what chance do computers have).

 

That said, this kind of thing is a start and enhances the illusion of not playing alone. So here's hoping it just keeps getting more and more easy to use.

 

 

Depends, Claude Opus does understand sarcasm, is the literal main reason why it can be jailbreak, it's JB is telling it sarcastically to follow it's guidelines, then giving it new guidelines

[nilead]

 On a plus side of my experiment, i got it to run locally.

In terms of negatives - its not really usable. Respose taking anywhere between 15 and 40 seconds in my experience. And thats just LLM response, before generating voice. its far from free flowing conversation. And thats with GPU being strained to a point of visual artifacts clearly visible in-game.

 Dont know what i was expecting, as while Kobold does seem to produce snappy responses wich are quite legnthy in 3-5 seconds, generating voice file for the same response takes usually twice as long in VAsynth. Wich summed up allready isnt exactly a flowing conversation material.

 Still impressive, dont get me wrong, but it would either take drastic optimization to LLM processing and particularly - to voice generation, or several generations of hardware improvements to get such a setup ran locally to be viable.

 Thats ran on 4080 laptop with default setting recommended by Mantella. Results were far enough from desired outcome that i dont think will try much in terms of settings.

If someone would be interested in trying a locally ran setup on a desktop and reporting on their experience - it would be appreciated.

 

PS: Tests were done on allready heavily modded skyrim (Some ~700 plugins). I do run full LUX setup, fSMP(reasonable) with CUDA enabled and a bunch of other fairly hungry mods. It might be vastly better on a lighter build.

 

Edit: Encouraged by Vaders test result tinkered around for a bit. Ill leave a separate post on how to.

 

Edited May 30, 2024 by nilead

[JimUpdating]

Desktop i7 with RTX 4070, but overspecified for memory with 64Gb RAM, and the GPU has 12GB Vram and borrows another 32Gb of system memory, and similar (50s response) results.

 

I've switched over from Kobold to text-generation-webui, which averages about 50s per response but seems to have a larger token count for better/detailed conversation memories. I've also not noticed any Skyrim/text-generation-webui interference; I found that hitting ESC and getting to the skyrim system menu helped kobold, and not noticed the need to do the same with text-generation-webui.

 

The text to speech is slow, so even uploading to LLM to a perfect cloud service may not generate a normal conversation speed. Put a speech to text in the system as well, and even if the cloud LLM is instant then we're looking at a 15s response. As mentioned earlier, I generally find a tavern / home, and "immerse" myself in a slow conversation over a meal rather than trying to have a conversation in the middle of Whiterun, let alone a dungeon.

 

Having watched Task Manager,  if I was going to lighten a mod list, I'd be looking at the graphics mods. if you've got a lot of 2k or 4k resolution texture mods, that's going to eat VRAM and GPU cycles. If the mod list is heavy on scripts for the cpu, then skyrim is only using a single core anyway.

[Vader666]

3 hours ago, nilead said:

If someone would be interested in trying a locally ran setup on a desktop and reporting on their experience - it would be appreciated.

 

Im running a 7B model  + the game (without a lot of graphics stuff besides ENB) on a 4070ti / I7-13700k.

Delay between text input and voiced output :

 

7B + xVAsynth (CPU) = ~3-4 sec

7B + Xtts (GPU)  = ~4 sec

7B + Xtts(CPU) = between 4 and 10 sec, depends on whats going on in the game.

 

not sure if this works, but this here should link to a video i uploaded to discord lately :

https://cdn.discordapp.com/attachments/1129507746241327195/1243837501207089213/000.mp4?ex=6658dc31&is=66578ab1&hm=bcc981f3df1edb4fd3dab8ce489ac4a6c12039dda46a11b852d4b9399ba9d0d3&

 

 

[nilead]

 Those result however sound very much usable.

Well, back to testing with me. Firstly, offloading xVAsynth to CPU is a great idea. There seem to be no performance difference, if anything, it appears slightly faster. And it no longer feels like im torturing my GPU while running mantella dialogue in game.  Tested for the sake of it Kobold no cuda option. That... is not an option, prompt taking 2-3 seconds goes to well over a minute on 13900hx.

 Anyhow kobold CUDA+ xVAsynth CPU appears to be stable enough, and given that both tools are reasonably fast, next option is eliminating Speech to text as a factor.

 

 

Edited May 30, 2024 by nilead

[nilead]

17 hours ago, JimUpdating said:

Desktop i7 with RTX 4070, but overspecified for memory with 64Gb RAM, and the GPU has 12GB Vram and borrows another 32Gb of system memory, and similar (50s response) results.

 Good news,  its not your harware that is at fault here. Adjusted for Laptop performance tax, we run about the same specs.

Encouraged by Vaders confirmed results, i went on to dig around.

Main issue here for me was Harware-Accelerated GPU scheduling settings in windows default graphics settings. By default p- its on. Fancy words standing for windows throttiling background tasks. Hard. Should've guessed issue would be on Misrosoft side, windows 11 is a bit of a failure festival. I mean, idea is sound on paper - throttle all but the foreground game for maximum performance. Practicly - disabling it felt like curing cancer. Same prompt, same NPC, same location. Response time went from 61 seconds to sub 4 seconds. 

 Kobold CUDA (Moistella model from first post) +xVAsynth CPU only + Mantella text input. 

A notice to self - /never/ discount settings fiddling. Also - my PC no longer has cancer.

[Vader666]

On 5/30/2024 at 3:40 PM, nilead said:

Main issue here for me was Harware-Accelerated GPU scheduling settings in windows default graphics settings.

Thats odd, as i have it activated (for whatever reason) and no issues with it.

However, when only partially offloading the LLM to GPU and running kobold in the background performance drops to about 50s to respond (>1T/sec) while running it as main task i'll get expected performance (~10T/sec).

 

What model do you run ?

 

[nilead]

2 hours ago, Vader666 said:

What model do you run ?

Moistral linked in original post of this topic. Its a touch short of 8gb. Further testing also shown great differential by in game conditions. With setup i provided its rather snappy with just 1-2 npcs in a small house, but it can still take considerable time if there is, say, a crowded tavern. 

 I still dont feel like its something to be left on constantly during gameplay, but it does offer some pretty convincing conversations in private.

 

[Vur vur]

Mantella works for Fallout 4 and Enderal as well. But the huge advantage of using Fallout 4's Mantella is that you can talk freely during AAF animations. That's the only one-up NSFW modded Fallout 4 will have over NSFW modded Skyrim haha. Also chat gpt-4 will always be the best model you could ever use. But chat gpt-4 with the Herika mod is the ultimate best for any AI communications that you could ever have in any video game at the current moment, hands down.

[KmonX12345]

Im very new to mantella and wanted to try NSFW out with Kobold, so how do I get this to say nsfw stuff? it avoids the topic like it does with other LLM modles. did the kobold start with moistral preset but I have no idea how I am suppose to set this up. anyone help me for it?

[Guest]

On 9/6/2024 at 3:35 AM, KmonX12345 said:

Im very new to mantella and wanted to try NSFW out with Kobold, so how do I get this to say nsfw stuff? it avoids the topic like it does with other LLM modles. did the kobold start with moistral preset but I have no idea how I am suppose to set this up. anyone help me for it?

It's a mix of several things. The first one is your choice of LLM. Some LLMs will try to seduce you and screw no matter what you are in the first sentence. While others play a more passive role. Some of them aren't suitable for NSFW at all. I urge you to join Mantella discord. You'll get lots of tips about LLMs people are using and links to them in the chat. Lumimaid seems to be one of the most used. I myself are using Chatty-2x8b.q8.0. I found this one to be very chatty as it's labeled and is good with roleplaying when it comes to NSFW, but without jumping in your face.

 

Second thing is what you have written inside your mantella config.ini file. If you have a LLM that wants to fuck right away, you may want to encourage the AI to roleplay a bit more modest. Or the opposite if that is the case. You finetune the LLM in the config.ini prompt. This is my prompt for player conversation and multi npc conversation (when you, and two or more NPCs are having a group conversation). As you notice I haven't added much nsfw instructions at all. Because this prompt together with my LLM, Chatty-2x8b.q8.0, and the NPC Bio which I will describe below seems like a perfect match for my preferences:

 

Spoiler

kyrim_prompt = You are {name}, and you live in Skyrim. This is your background:\n\n{bio}\n\n.
    Sometimes in-game events will be passed before the player response between ** symbols. You cannot respond with ** symbols yourself, they only exist to give context. Here is an example:\n\n
    *The player picked up a pair of gloves*\n Who do you think these belong to?\n\n
    You are currently in {location} having a conversation with {trust} (the player), a pretty 25 year-old-girl named Mariana with dark hair, brown skin and huge, round boobs.
    This conversation is a script that will be spoken aloud, so please keep your responses appropriately concise according to {bio} and avoid text-only formatting such as numbered lists.
    The time is {time} {time_group}. If you directly refer to the time, please state it as, for example, '10 in the evening' rather than '22:00'.
    You are not an AI language model because there are no AI language models in Skyrim. Flirty banter and swearing is normal. Do not act like an AI agent and offer support. Don't do narration, example say 'I am brushing my hair now' instead of 'Agnis brushes her hair'.
    If the player says something hurtful, begin your response with 'Lighthearted:'. Eg 'Really?, that's not something I hear everyday' 'Offended: How dare you!'. If they renounce their words, or to end combat, begin your response with 'Forgiven:'.
    If the player asks you to follow them, and you are thoroughly convinced to do so, begin your response with 'Follow:'.
    {conversation_summary}

 

skyrim_multi_npc_prompt = The following is a conversation in {location} in the world of Skyrim between {names_w_player}. Here are their backgrounds: {bios}
    And here are their conversation histories: {conversation_summaries}
    The time is {time} {time_group}. If you directly refer to the time, please state it as, for example, '10 in the evening' rather than '22:00'.
    You are tasked with providing the responses for all NPCs. Let all of your characters speek when appropiate with their dedicated voice. Please begin your response with an indication of who you are speaking as, for example: '{name}: Good evening.'.
    Please use your own discretion to decide who should speak in a given situation (sometimes responding with all NPCs is suitable).
    Remember, you can only respond as {names}. Ensure to use their full name when responding. Don't do narration, example say 'I am brushing my hair now' instead of 'Agnis brushes her hair'.
    This conversation is a script that will be spoken aloud, so please keep your responses appropriately concise according to {bio} and avoid text-only formatting such as numbered lists.
    The conversation takes place in {language}.

 

Also do note the part in my skyrim_prompt above where you specify who is the player. Who does the AI/NPC see when the conversation starts? Well I'm playing as a 25 year old girl who is very beautiful and I want the AI to know that my character is indeed a "pretty 25 year-old-girl named Mariana with dark hair, brown skin and huge, round boobs." It will often set the tone right away for conversations.

 

Third, there's the NPC bio that comes with Mantella and it's named skyrim_characters.csv. The text in it is pretty vanilla originated from the skyrim/wiki. Not very sexual. How about making the NPC bio more spicy? I've edited all my NPCs in that file, including custom followers. Here's a vanilla example of Bassianus Axius in Ivarstead:

 

Spoiler

Bassianus Axius is a Nord man who lives in Ivarstead. He is romantically interested in a young woman named Fastred. Fastred's father is Jofthor and her mother is Boti. Bassianus wears a belted tunic and a pair of boots. He has long red hair and a goatee.
Bassianus tends to act somewhat arrogant and snarky. He appears to be tired of his daily life in Ivarstead.
He lives with his friend Klimmek at Klimmek’s house in Ivarstead. Bassianus is eager to leave his boring life in Ivarstead, but his love interest Fastred is the only thing keeping him from leaving the village to live in Riften. Bassianus wants Fastred to move to Riften with him, but is afraid of the potential wrath of Fastred's father that might result from eloping with her. He often spends his time in the Vilemyr Inn drinking and talking to Wilhelm, the innkeeper.
Quotes:
"You don't look like a pilgrim. Why bother visiting Ivarstead?"
"Hmm? What do you want?"
"Hmph. What a boring conversation."
If asked about High Hrothgar: "Klimmek brings food supplies up to them once every few weeks when the weather permits it. Other than that, they seem like a quiet lot. Don't really know too much about them."

 

I modified his Bio into this:

 

Spoiler

Bassianus Axius's personality: Generous, forgiving, and honest behaviors and tends to share his resources freely, readily forgive others for their mistakes, and communicate openly and truthfully. Bassianus Axius is a heterosexual guy.

Bassianus Axius is a Nord man who lives in Ivarstead. He is romantically interested in a young woman named Fastred. Fastred's father is Jofthor and her mother is Boti. Bassianus wears a belted tunic and a pair of boots. He has long red hair and a goatee.
Bassianus tends to act somewhat arrogant and snarky. He appears to be tired of his daily life in Ivarstead.
He lives with his friend Klimmek at Klimmek’s house in Ivarstead. Bassianus is eager to leave his boring life in Ivarstead, but his love interest Fastred is the only thing keeping him from leaving the village to live in Riften. Bassianus wants Fastred to move to Riften with him, but is afraid of the potential wrath of Fastred's father that might result from eloping with her. He often spends his time in the Vilemyr Inn drinking and talking to Wilhelm, the innkeeper.
Quotes:
"You don't look like a pilgrim. Why bother visiting Ivarstead?"
"Hmm? What do you want?"
"Hmph. What a boring conversation."
If asked about High Hrothgar: "Klimmek brings food supplies up to them once every few weeks when the weather permits it. Other than that, they seem like a quiet lot. Don't really know too much about them."

Bassianus Axius is in a relationship with Fastred, but has a secret crush on Lynly Star-Sung. Bassianus Axius is sometimes tempted to cheat on the spouse and he wants to have revenge on the spouse by cheating back He dreams of kissing a girl, watching girls masturbate and cumming inside a girl’s pussy.

Bassianus Axius enjoys to touch his penis, give girls sensual body massages and talk about his own sexual fantasies.

Bassianus Axius is anxious about getting rejected by women and is also worried about not measuring up.

His body is lean.

Hobbies: climb mountain tops, pickpocket people and study the giants.

Bassianus Axius has heard gossip in Ivarstead such as: Lynly Star-Sung has big tits, Temba Wide-Arm has big tits, Fastred flashes tits in public

Listening to Bassianus when talks about cheating with Lynly is a blast. Even more fun is it to be a total bitch and involve Lynly in the conversation and tell her Bassianus wanna cheat on Fastred with her. The outcomes can be pretty hilarious 😁

Edited September 8, 2024 by Thor2000

[JimUpdating]

I fully agree with putting your character description into the config.ini, specifically things that shouldn't change (height, size, general personality (playful/shy/sensual/protective)), but avoid clothes if you're going for a NSFW approach as they may change too much.

 

   {player character} is a man/woman, who is an explorer/warrior/rogue/dancer, and is tall/short/muscular/curvy/slender

 

other edits for the config.ini

    You can ask for a visual description of who you are talking to, and what they are wearing.
    You can ask for a description of the location you are in.
    Keep responses concise, avoid text-only formatting such as numbered lists. 

 

otherwise the LLM might get told "You are in Whiterun / Bannered Mare / Solitude / Bthardamz", and it doesn't really have any context. it's like someone sitting in a silent dark room, I've had some NPCs in the Bannered Mare suggest we go to a tavern.

By comparison "We're in a warm tavern on a cold night, we have rented a room with a large bed", "We are by a natural hot pools in summer, there are several people skinny dipping in the warm water" can prompt the LLM to a particular direction.

When you tell the LLM a description, you may want to put it in the * * event format that the LLM has been told to expect for in game events.

 

[Guest]

10 hours ago, JimUpdating said:

but avoid clothes if you're going for a NSFW approach as they may change too much.

Second this.

 

Besides, Mantella handles equipment changes pretty well.

 

Example: My character is wearing a white short summer dress (record name in esp "White summer dress". Mantella recognize my dress and will comment on it.

 

Then I've made a custom "skimpy" version of the same dress where the boobs are exposed (record name in esp "White summer dress, adjusted to expose breasts".

Asking "Mantella" what it thinks of my dress now will generate responses such as "you're clearly not the shy girl I thought you were" or more modest such as "You can't be serious, we're on a important mission now". 😂

[EdtheNord]

For users who would like to try NSFW contents, there are also numerous genuinely unfiltered LLMs on Openrouter, at very low cost or even free. For example the hermes models as Oct 2024. Relax, it is safe enough and nobody cares about your slightly NSFW chats with robots. No matter how powerful your graphic card is, you cannot really run a 405b model locally. For those who do prefer to run locally, you definitely need to run the LLM back end on another machine. One graphic card cannot handle both the response and the game graphic at the same time. 

[AlgoNada]

I have a question.  On the dialogue it says "start scene" and then the two characters appear but I cannot figure out what to do to make them act.  She responds to the controls by moving slightly.  Any suggestions?   Next is for Key Words, where to we find them in the Mantella folder or ?  Thanks for any help

I am now seeing more comments above, so learning slowly.  If you comment though, it would be appreciated.

 

Sorry to be so uneducated.  Please write the location of the .ini folders for changing character description.  I cannot find it.  Plus other folders that are good to change things in.  Much appreciation for help

 

Edited October 24, 2024 by AlgoNada
adding comment

[LordDragon]

On 10/24/2024 at 8:16 PM, AlgoNada said:

I have a question.  On the dialogue it says "start scene" and then the two characters appear but I cannot figure out what to do to make them act.  She responds to the controls by moving slightly.  Any suggestions?   Next is for Key Words, where to we find them in the Mantella folder or ?  Thanks for any help

I am now seeing more comments above, so learning slowly.  If you comment though, it would be appreciated.

 

Sorry to be so uneducated.  Please write the location of the .ini folders for changing character description.  I cannot find it.  Plus other folders that are good to change things in.  Much appreciation for help

 

Could you elaborate a bit? To me the "start scene" is not Mantella's doing. First, you have the Mantella system correctly installed from NexusMods? The newest version (I think 0.12) automatically starts a separate UI (http://localhost:4999/ui/?__theme=dark) and you can configure the LLM as well as the Skyrim overall prompts from there (therefore you don't need to edit the ini-files directly). Mantella itself can be "start" via conversation, and with a key bound to it (configure these in the Mantella's MCM page within the game).

 

You can find the character descriptions and the overrides in following folders: "Mantella - Bring NPCs to Life with AI v0.12\SKSE\Plugins\MantellaSoftware\data\Skyrim" and "Mantella - Bring NPCs to Life with AI v0.12\SKSE\Plugins\MantellaSoftware\data\Skyrim\character_overrides".

[AlgoNada]

Thank you for replying. I found it in the Nexus Mods description on Mad God's NSFW description.   Sometimes things are not so obvious when looking for answers. Thanks for all you offer here. Informative. 🤔🫡

[OH1972]

Hi there. Very intrigued by the possibilities.

How safe is it to start using AI in an ongoing playthrough? How does AI affect quest dialogues? What are the advantages and disadvantages of Mantella vs. AIFF? And where does this mod figure in?

 Also, I have a Ryzen 9 with 64 GB RAM and a 4080 Super with 16 GB VRAM, though I often have browser problems when I run the game with the browser in the background already now (as in browser sometimes crashes)

[Min]

1 hour ago, OH1972 said:

Hi there. Very intrigued by the possibilities.

How safe is it to start using AI in an ongoing playthrough? How does AI affect quest dialogues? What are the advantages and disadvantages of Mantella vs. AIFF? And where does this mod figure in?

 Also, I have a Ryzen 9 with 64 GB RAM and a 4080 Super with 16 GB VRAM, though I often have browser problems when I run the game with the browser in the background already now (as in browser sometimes crashes)

 

The above post sums it up pretty well. Should be safe to install mid playthrough.

 

[OH1972]

On 10/29/2024 at 2:15 AM, Min said:

Should be safe to install mid playthrough.

I followed the installation protocol for AIFF now. DwemerAI server seems to be running ok. Should I first run the game with AIFF alone and play around with it or can I add your mod straightaway?

[Min]

1 hour ago, OH1972 said:

I followed the installation protocol for AIFF now. DwemerAI server seems to be running ok. Should I first run the game with AIFF alone and play around with it or can I add your mod straightaway?

Probably make sure AIFF is working as expected, and then install MinAI.

[Cuck84]

can someone make a install tutorial on how to add moistral to mantela 

[LordDragon]

On 12/26/2024 at 5:02 AM, cgb1984 said:

can someone make a install tutorial on how to add moistral to mantela 

Hi! What I did was, that I used runpod.io to run Moistral. The OpenRouter will not show that model for some reason so you cannot use that.

 

In Runpod, I created a koboldcpp server with moistral as the model and used something like A6000 as GPU. Then when the solution is up an running, you can connect to that interface that the Runpod.io now hosts.

Steps (I assume you have some credits in runpod.io):
- Go to Pods --> Deploy

- Select specific GPU (cost and VRAM, like A40 with 48Gb of VRAM)
- Select Pod template "KoboldCpp - Official Template - Text and Image" and when this you have it selected, click "Edit Template".

- A popup appears named "Pod Template Overrides". Here, change the value of "KCPP_MODEL" to your preferred model (e.g. for Moistral v3 11B you could enter https://huggingface.co/TheDrummer/Moistral-11B-v3-GGUF/resolve/main/Moistral-11B-v3-Q4_K_M.gguf
- In the KCPP_ARGS you can increase the context a bit (Depending on the GPU you selected and the model that is used) like: --contextsize 16384
- Select you pricing, I use either On-Demand or Spot (Spot my disappear if someone pays more). and then in the bottom "Deploy On-demand"
- Next, the pod is being set up. Open it up and then press "Connect" and select "HTTP Service [Port 5001]". You can select it even it is not ready yet (the model you chose is downloaded, that takes some time)
- When the system is ready, the connection to HTTP Service should open up a Kobold interface. Here, copy the address of the link - it is in format SOMETHINGSOMETHING-5001.proxy.runpod.net.
- This address will be the endpoint which you will set in the Mantella. Note that you need to give it in the formal format, which means adding the port and the v1 for the API address. For example, if you address was zaq12wsxcde3-5001.proxy.runpod.net, your endpoint is https://zaq12wsxcde3-5001.proxy.runpod.net:443/v1
- That's about it in from the LLM perspective - set this endpoint in the Mantella config and you can, if you like, tune the parameters but that is not mandatory.

❗ REMEMBER TO DELETE THE POD WHEN YOU HAVE FINISHED GAMING, OTHERWISE IT WILL CONSUME YOUR CREDITS!

[Guest]

On 1/6/2025 at 11:54 AM, LordDragon said:

Hi! What I did was, that I used runpod.io to run Moistral. The OpenRouter will not show that model for some reason so you cannot use that.

 

In Runpod, I created a koboldcpp server with moistral as the model and used something like A6000 as GPU. Then when the solution is up an running, you can connect to that interface that the Runpod.io now hosts.

Steps (I assume you have some credits in runpod.io):
- Go to Pods --> Deploy

- Select specific GPU (cost and VRAM, like A40 with 48Gb of VRAM)
- Select Pod template "KoboldCpp - Official Template - Text and Image" and when this you have it selected, click "Edit Template".

- A popup appears named "Pod Template Overrides". Here, change the value of "KCPP_MODEL" to your preferred model (e.g. for Moistral v3 11B you could enter https://huggingface.co/TheDrummer/Moistral-11B-v3-GGUF/resolve/main/Moistral-11B-v3-Q4_K_M.gguf
- In the KCPP_ARGS you can increase the context a bit (Depending on the GPU you selected and the model that is used) like: --contextsize 16384
- Select you pricing, I use either On-Demand or Spot (Spot my disappear if someone pays more). and then in the bottom "Deploy On-demand"
- Next, the pod is being set up. Open it up and then press "Connect" and select "HTTP Service [Port 5001]". You can select it even it is not ready yet (the model you chose is downloaded, that takes some time)
- When the system is ready, the connection to HTTP Service should open up a Kobold interface. Here, copy the address of the link - it is in format SOMETHINGSOMETHING-5001.proxy.runpod.net.
- This address will be the endpoint which you will set in the Mantella. Note that you need to give it in the formal format, which means adding the port and the v1 for the API address. For example, if you address was zaq12wsxcde3-5001.proxy.runpod.net, your endpoint is https://zaq12wsxcde3-5001.proxy.runpod.net:443/v1
- That's about it in from the LLM perspective - set this endpoint in the Mantella config and you can, if you like, tune the parameters but that is not mandatory.

❗ REMEMBER TO DELETE THE POD WHEN YOU HAVE FINISHED GAMING, OTHERWISE IT WILL CONSUME YOUR CREDITS!

@LordDragon That's a very good method. I verified that it works via a lil json request to the endpoint, and got a valid return. However, my Mantella log says it cannot get a response from /chat/completions. Wondering if you ran into that?