Setting up Ollama for local chat and reasoning

Ollama lets CodeBot use local chat and reasoning models on your own computer. Once Ollama is installed and a model is downloaded, CodeBot can talk to it through a local Ollama account.

Use Ollama if you want CodeBot to use a local language model instead of a hosted provider such as OpenAI, Anthropic, Google Gemini or Grok.

Ollama support is currently best treated as experimental for full CodeBot orchestration. Local models can work well for direct chat and focused prompts, but CodeBot's full agent context, profile instructions and tool definitions can produce very large prompts. Large local prompts can take minutes to process before the model starts writing a response.

What You Need

Before you start, make sure you have:

  • A Mac, Windows PC or Linux PC that can run local language models.
  • Enough free disk space for models. A single model can be several gigabytes.
  • Time for the first model download.

Local models are useful for privacy, offline work and avoiding hosted AI credits. They are usually slower and less capable than the best hosted models, especially on small laptops.

Short Version

  1. Install Ollama from the official download page.
  2. Start Ollama.
  3. Download one model with ollama pull.
  4. Run the model once to confirm it works.
  5. Add an Ollama account in CodeBot with this URL http://127.0.0.1:11434

After that, choose the Ollama account in CodeBot when you want to try a local chat model.

Install Ollama

Download Ollama from the official Ollama download page: https://ollama.com/download

Install it like a normal app for your platform.

On macOS and Windows, Ollama usually runs in the background after you open it. On Linux, follow the install instructions shown on the Ollama download page.

Ollama usually runs at http://127.0.0.1:11434. You normally do not need to change this URL.

Add Your First Model

Ollama needs at least one model before CodeBot can use it.

Open Terminal on macOS or Linux, or PowerShell on Windows, and run:

ollama pull qwen3:30b

This downloads a strong general chat and reasoning model. The download can take a while.

When it finishes, run:

ollama run qwen3:30b

Type a short message, such as:

Hello

If the model answers, Ollama is working. You can close the chat by typing:

/bye

Choose a Model

Start with one small or medium model. Bigger models can be smarter, but they need more memory and run more slowly.

If you want Try Why
Strong local coding experiments qwen3-coder:30b Best first choice for coding-focused local tests on a machine with enough memory.
Strong general chat qwen3:30b Good default for interactive chat, planning and reasoning.
Fast/light coding fallback qwen2.5-coder:14b Useful for lower-latency internal CodeBot tasks and smaller coding requests.
Compact general fallback gemma3:12b Good smaller local chat model when you want something lighter.
Large general-model experiment llama3.3:70b Heavier model for comparison on higher-memory machines.

To download another model, use:

ollama pull <model-name>

For example:

ollama pull qwen3-coder:30b

Browse available models here: https://ollama.com/models

We suggest to not download many models at once. Start with one model, test it, then add more only if you need them.

Suggested Setup for a 64 GB Mac

For a Mac with 64 GB of unified memory, such as an M-series Pro or Max machine, this is a good local test set:

ollama pull qwen3-coder:30b
ollama pull qwen3:30b
ollama pull qwen2.5-coder:14b
ollama pull gemma3:12b
ollama pull llama3.3:70b

Use qwen3-coder:30b for coding experiments, qwen3:30b for general chat and reasoning, and qwen2.5-coder:14b as a faster fallback. llama3.3:70b is larger and slower, but useful for comparing general chat and deeper reasoning behavior on higher-memory Macs.

For quicker first tests, use the smaller qwen2.5-coder:7b instead of qwen2.5-coder:14b.

Add Ollama to CodeBot

Once Ollama is running and at least one model is installed, add it in the app:

  1. Open the app's Settings.
  2. Go to CodeBot.
  3. Click Manage & Add Accounts.
  4. Add a Local Ollama account.
  5. Set the URL to http://127.0.0.1:11434
  6. Save the account.

You do not need an API key for local Ollama.

Use It From CodeBot

Keep Ollama running in the background. Then choose the Ollama account in CodeBot.

CodeBot will ask Ollama which models are installed. If the model list is empty, install a model with ollama pull and try again.

Ollama is for local chat and reasoning models.

Current CodeBot Limitations

Full CodeBot sessions can send large prompts to the selected model. This includes system instructions, profile instructions, available tools, conversation history and workspace context. With local Ollama models, a single CodeBot request can contain tens of thousands of prompt tokens.

If Ollama logs show messages like this, the delay is prompt processing, not response generation:

prompt processing, n_tokens = 34816

At a few hundred prompt tokens per second, that can mean multiple minutes before a short answer appears. For this reason, Ollama is currently not recommended as the primary provider for full CodeBot orchestration. Hosted providers remain a better fit for normal CodeBot use.

Ollama is still useful for:

  • Direct local chat.
  • Focused local coding experiments.
  • Offline or privacy-sensitive prompts.
  • Future lightweight/local CodeBot modes with smaller prompts and fewer tools.

Running Ollama on Another Computer

You can run Ollama on another computer on your local network, such as a faster desktop or server. In that case, use that computer's local network address in the CodeBot account instead of 127.0.0.1, for example: http://192.168.1.25:11434

Note: Only do this on a trusted local network. Do not expose Ollama directly to the public internet.

On macOS, Ollama normally listens only on 127.0.0.1. To make it reachable from another computer, start it with OLLAMA_HOST set to 0.0.0.0:11434.

If you use a custom model folder, make sure the network-accessible Ollama process also has OLLAMA_MODELS set to that folder. Otherwise, 127.0.0.1 might show installed models while the LAN address shows an empty model list.

For a one-time command-line test:

OLLAMA_HOST="0.0.0.0:11434" \
OLLAMA_MODELS="/path/to/ollama/models" \
/Applications/Ollama.app/Contents/Resources/ollama serve

Then compare the model list from the Ollama computer itself:

curl http://127.0.0.1:11434/api/tags
curl http://<ollama-computer-lan-ip>:11434/api/tags

Both commands should return the same models.

Troubleshooting

CodeBot Cannot Connect

Check these things:

  • Ollama is running.
  • The CodeBot account URL is http://127.0.0.1:11434 if Ollama is on the same computer.
  • If Ollama is on another computer, the URL uses that computer's local network address.

CodeBot Shows No Models

Ollama probably does not have a model installed yet, or CodeBot is connecting to an Ollama process that uses a different model folder.

Run:

ollama list

If the list is empty, download a model:

ollama pull qwen3:30b

Then try CodeBot again.

If CodeBot connects to Ollama on another computer, compare these from that computer:

curl http://127.0.0.1:11434/api/tags
curl http://<ollama-computer-lan-ip>:11434/api/tags

If the results differ, stop all Ollama processes and restart one Ollama server with both OLLAMA_HOST and, if needed, OLLAMA_MODELS set.

The Model Is Too Slow

Try a smaller model. Large models need more memory and more time.

Local model speed depends on your computer, available memory and the model size. A small laptop may be fine for short tests but slow for large coding tasks.

For CodeBot, also check the Ollama server logs. If most of the time is spent in prompt processing with a large n_tokens value, the model is spending time reading the request context. Switching to a smaller model can help, but full CodeBot orchestration may still be slow because the prompt itself is large.

The Model Gives Weak Answers

Try a stronger model, or use a hosted provider for demanding work.

Local models can be very useful, but they are not all equally good at coding, planning, long context or tool use. If a local model struggles, try another model from the Ollama library.

Useful Commands

List installed models:

ollama list

Download a model:

ollama pull qwen3:30b

Run a model directly:

ollama run qwen3:30b

Remove a model:

ollama rm qwen3:30b

See Also