Local models (Ollama, LM Studio)

If you run Ollama or LM Studio on your computer, Granular finds it when you open the model menu and offers its models with one click. There's no key to paste and nothing to pay per message, and your messages go to the model on your computer rather than to an AI company.

A local model is an AI model that runs on your own computer instead of on a company's servers. Ollama and LM Studio are free apps that download and run them.

Before you start

• Install Ollama (ollama.com/download) or LM Studio (lmstudio.ai/download), and download at least one model in it.
• Make sure it's running:
  • Ollama: open the Ollama app, or run ollama serve in a terminal.
  • LM Studio: open LM Studio and start its local server, on its Developer tab or by running lms server start in a terminal.
• Local models need a fairly powerful computer. Bigger models answer better but need more memory.

Use your local models

1. In the row of controls under the message box, click the button that shows your current connection's name, such as Claude or G.hosted.
2. Look under the Accounts list for On this computer.
3. Click Use local models (Ollama) (or (LM Studio)). Under it, Granular shows how many models you have, how many can use tools, and "free, no key, stays on this computer".

That's it. Ollama (or LM Studio) is added to your Accounts list, and the current session switches to it. It also gets a card in Settings → AI accounts marked Local server, with the Local price tag. See Manage your AI accounts.

During first-run setup you can do the same from the Local models card under More ways to connect….

What you'll see

• The new Ollama row in Accounts works like any other provider. Pick it whenever you want this session to use your local models.
• The model button under the message box lists your installed models, up to 20. Models that can use tools come first, and ones that can't are marked No tools. Models that only make embeddings (used for search, not chat) are left out. You can search the list and pin favourites. See Pick and pin models.
• Each time you open the menu, Granular checks again, so a model you just downloaded shows up straight away.

Choose a model that can use tools

Granular's agents work by using tools: reading and writing files, running commands and using your apps. Many small local models can't use tools, so with them an agent can only chat.

When Granular can tell, it says so under the row:

• "None of your Ollama models can use tools, so agents can only chat with them…" means you should download a model that supports tools before doing agent work.
• "gemma3:1b can't use tools, so agents on it can only chat. Pick a model marked "tools" for agent work." names the models to avoid.
• "Local models vary…" appears when the app doesn't say which models can use tools. If an agent can't read files or run commands, try a bigger model.

If there's no model yet

If Ollama or LM Studio is running but has no models, the block shows how to get one:

• Ollama: "Download one in a terminal, for example: ollama pull qwen3"
• LM Studio: "Download one from its Discover tab, or in a terminal: lms get qwen3"

Download a model, then click the refresh button (↻) next to On this computer.

Change or undo it

• Look again: click the refresh button (↻) next to On this computer. Granular only looks when you open the menu or click this button; it never checks in the background.
• Use a different port: if your app runs at another address, point at the Ollama (or LM Studio) row in Accounts, click its ⋯ button (Options), then Edit provider, and change Base URL. Keep the /v1 at the end, for example http://127.0.0.1:11500/v1. Leave API key empty; a local app doesn't need one. After that, Granular looks for it at the new address.
• Remove it: click ⋯ on its row, then Remove. Nothing is deleted from Ollama or LM Studio.

If it stops while you're using it

If you close Ollama or LM Studio, or it stops, the chat tells you instead of blaming your internet connection:

• "Ollama isn't answering." Open it again, then send your message again.
• "LM Studio stopped in the middle of a reply." The answer above was cut short. Start the server again and resend.

In the menu, a local provider you've added that isn't running shows what to do, plus Change address and Check again.

Limits

• Only on this computer. Granular looks at 127.0.0.1:11434 for Ollama and 127.0.0.1:1234 for LM Studio, on Mac and Windows. Plain http:// addresses are only allowed for your own computer.
• Up to 20 models per local provider, the same as any custom provider.
• No test message when you add it. A large model can take a long time to load the first time, so your first message is the test.
• Tools depend on the model, as above. Quality and speed depend on the model and your computer.
• Local models use the same translation as other OpenAI-compatible providers. See Add a custom AI provider.

Troubleshooting

If "On this computer" doesn't appear
Neither app is running at its usual address. Start Ollama or LM Studio's server, then close and reopen the menu, or click ↻ if the block is showing.

If you see "Ollama didn't answer at 127.0.0.1:11434 in time"
It may be busy loading a model. Wait a moment, then click Check again or ↻.

If you see "Ollama answered … but its model list returned HTTP 404. Is this really Ollama?"
Something else is using that port. Change the address to where Ollama really runs, or stop the other program.

If you see "Ollama isn't running at 127.0.0.1:11434"
Open the Ollama app, or run ollama serve in a terminal. For LM Studio, open it and start its local server.

If an agent says it can't read or change files
The model probably can't use tools. In the model button, pick one that isn't marked No tools, or download one.

Related