Skip to content

Install a local AI (free)

A local AI runs on your computer: it is free, nothing goes out to the Internet, and it powers Duke, the game assistant. This page explains everything, step by step, even if you have never installed an AI.

You need two things:

  1. a program that runs the AI: Ollama or LM Studio (both free, pick just one);
  2. a model, the AI itself, which this program downloads once (a few GB).

Then you tell ROMKADE Manager where to find that program.

Which one?

  • LM Studio: everything is done with the mouse, in a window. The easiest if you don't like the terminal. On a Mac, it needs Apple silicon (M1 or later).
  • Ollama: a single command to type to download the model, then it runs quietly in the background. Also works on Intel Macs.

Your computer

Memory (RAM) Suggested model Download
16 GB or more Qwen 2.5, 7B (qwen2.5:7b) about 4.7 GB
8 GB Qwen 2.5, 3B (qwen2.5:3b) about 1.9 GB

These models understand English, French and Spanish well. A local AI is slower and a little less accurate than a large online service; a recent graphics card or a Mac with Apple silicon makes answers much faster.

To check your memory: on a Mac, the Apple menu → About This Mac; on Windows, Settings → System → About; on Linux, free -h in a terminal.

Option 1: Ollama

1. Install Ollama

  1. Open ollama.com/download and download the macOS version.
  2. Open the downloaded file and drag Ollama into the Applications folder.
  3. Start Ollama from the Applications folder. Its icon (a small llama) appears in the menu bar, top right: it is running.
  1. Open ollama.com/download and download the Windows version.
  2. Run OllamaSetup.exe and follow the installer.
  3. Ollama starts on its own; its icon appears next to the clock, bottom right. It starts again with Windows.

In a terminal:

curl -fsSL https://ollama.com/install.sh | sh

Ollama is installed as a service that starts with the computer.

2. Download the model

Open a terminal: Terminal on a Mac (Applications → Utilities), PowerShell on Windows (Start menu, type "PowerShell"). Type, depending on your memory:

ollama pull qwen2.5:7b

(or ollama pull qwen2.5:3b with 8 GB of memory). The download takes a few minutes. To check:

ollama list

The model shows up in the list. You can close the terminal: Ollama keeps running.

3. Check that Ollama answers

Open http://localhost:11434 in your browser. The page says Ollama is running: you are ready. Go on to Connect the AI to ROMKADE Manager.

Option 2: LM Studio

1. Install LM Studio

  1. Open lmstudio.ai and download the version for your system (macOS with Apple silicon, Windows or Linux).
  2. Install it like any other application, then start it. You can skip the welcome screen.

2. Download the model

  1. Open the model search tab (Discover, the magnifying glass in the left bar).
  2. Search for Qwen2.5 7B Instruct (or Qwen2.5 3B Instruct with 8 GB of memory).
  3. Click Download and wait for it to finish.

3. Start the server

  1. Open the Developer tab (the >_ icon in the left bar).
  2. Pick the downloaded model at the top of the window to load it.
  3. Start the server (Status switch on Running). LM Studio shows the address http://localhost:1234.

Keep LM Studio open while you use the assistant: if you close it, Duke stops answering.

Connect the AI to ROMKADE Manager

  1. In ROMKADE Manager, open Settings → Assistant.
  2. Choose Local model or OpenAI-compatible service.

  3. Choose the service address. ROMKADE Manager suggests it; here it is if you need to type it:

    ROMKADE Manager runs… Ollama LM Studio
    with Docker (the usual case) http://host.docker.internal:11434/v1 http://host.docker.internal:1234/v1
    from the source code http://localhost:11434/v1 http://localhost:1234/v1

  4. Choose the model in the list (qwen2.5:7b, for example).

  5. The light turns green: Connected.

Try it: open the Duke bubble, bottom right, and ask "A two-player game in my library?". The answer says "local model: nothing leaves this computer".

Docker on Linux

By default, Ollama and LM Studio only accept connections from the computer itself, and ROMKADE Manager runs in a container. Open them to containers:

  • Ollama:

    sudo systemctl edit ollama      # add:  [Service]  Environment="OLLAMA_HOST=0.0.0.0:11434"
sudo systemctl restart ollama

  • LM Studio: in the server settings (Developer tab), turn on Serve on Local Network.

On Mac and Windows with Docker Desktop, nothing to do.

Troubleshooting

What you see What to do
"No service answers on …" Ollama or the LM Studio server is not started, or the address does not match: check http://localhost:11434 (Ollama) and the Developer tab (LM Studio). With Docker on Linux, see the box above.
The model list is empty No model is downloaded: go back to "Download the model", then Change → reload the list.
Answers are very slow Take the smaller model (qwen2.5:3b) and close heavy applications.
The model answers, but not in the expected format Take a larger model.

The assistant and its other uses (game descriptions, controls) are described in Assistant, knowledge and videos. For an online service instead, see Create an API key.