Ollama Architecture: How It Actually Works

Ollama is a local model runtime with an HTTP server in front of it: it resolves a model reference, loads the weights, schedules a runner, and streams tokens back, the same path vLLM walks with a continuous-batching scheduler and a paged KV cache. Symvanta's graph of the repo at 8f91241 groups it into 105 functional modules (modularity Q=0.90), and the four largest are the Go server with the naming layer under it, the shared API types with the per-model output parsers, the terminal chat UI, and the launcher that decides which installed model an editor starts with.

Two things about the shape stand out. The chat path concentrates in a single handler: ChatHandler runs from line 2440 to line 2941 of server/routes.go, and five of the eleven endpoints listed below end there, the OpenAI-compatible /v1/chat/completions and the Anthropic-compatible /v1/messages among them. And the Apple silicon runtime under x/ has grown into a second inference path in its own right: MLX model layers and quantization (678 symbols), the MLX KV cache (522), and the MLX runner loop itself (427) together outweigh the Go server cluster (1,261).

Module map

The diagram shows the 10 largest of the 105 detected modules, with edges weighted by how many calls cross between them. Every module name here was re-derived from the packages its members live in and checked symbol by symbol against the graph, because the clustering labels each module after its highest-PageRank symbol: three of those are helpers the test suite defines, and a fourth is a function from the Go standard library. The symbol counts, hubs, and edge weights are what the graph computed. The heaviest arrow runs from the launcher config cluster into the launcher itself (94 calls), and the launcher sends 33 back.

Where to start reading

These are the most depended-upon symbols by PageRank over the call graph, blended with the hub of each module on the diagram, followed by the HTTP surface registered in server/routes.go. Six entries are left out below. New resolves to Go's own crypto/md5.New, and Models is a bare Go name that many packages define independently: nearly every editor adapter in cmd/launch has one, and so do envconfig, cmd/config, api, and app/store. Arrays is three different methods in x/mlxrunner that the graph groups by name. setTestHome, setLaunchTestHome, and Setup are test-harness helpers that rank high because every test in their package calls them.

• clamp
• ParseRef
• keyValue
• NewKVCache
• Var
• Trace
• POST /api/chat
• POST /api/generate
• GET /api/version
• POST /v1/chat/completions
• POST /v1/completions
• POST /v1/embeddings
• GET /v1/models
• GET /v1/models/:model
• POST /v1/responses
• POST /v1/audio/transcriptions
• POST /v1/messages

The endpoint list is worth a second look. Alongside its own /api/* surface, Ollama serves an OpenAI-compatible /v1/chat/completions and an Anthropic-compatible /v1/messages, and both land in the same ChatHandler as /api/chat. The compatibility work happens in gin middleware (middleware.ChatMiddleware, middleware.AnthropicMessagesMiddleware) that rewrites the request before the handler sees it, which is why one handler can serve three wire formats.

Key subsystems

HTTP server and model resolution

The gin server and the naming layer it resolves against: types/model validates a reference, manifest.BlobsPath turns a digest into a path on disk, and server caches what it read. At 1,261 symbols it is the largest cluster on the map. Its two heaviest edges weigh 44 calls each, one into the API types cluster and one into the model reference cluster, and 41 come back from the latter.

API types and model parsers

The request and response types every surface shares, in api/types.go (ThinkValue, ToolCallFunction, NewToolPropertiesMap), together with the per-model output parsers that model/parsers.ParserForName dispatches to. At 1,173 symbols it is second on the map, and its PageRank hub is an external symbol: Go's own crypto/md5.New.

Terminal chat UI

The chat pane ollama run draws: cmd/tui/chat lays out the transcript, wraps text to the pane width, and repaints as tokens stream in. 1,155 symbols, third on the map, and its heaviest edge is 59 calls into the agent cluster that runs tools on the user's behalf.

Launcher and model inventory

cmd/launch decides which installed model an editor or coding agent starts with, through fallbackLaunchModel, launchModelsFromNames, and the cloud-limit rules in model_inventory.go. It calls the launcher config cluster 33 times and takes 94 calls back, the heaviest single edge on the diagram.

MLX model layers and quantization

The Apple silicon model stack: x/models/nn builds the linear and attention layers, and x/mlxrunner/model/quant.go resolves group size, bit width, and mode per quantized tensor. 678 symbols, with 31 calls into the MLX KV cache and 12 coming back.

MLX KV cache and attention

x/mlxrunner/cache holds the key-value cache the MLX path decodes against, along with the batching and causal-mask helpers around it. 522 symbols, ninth on the map, taking 31 calls from the layer stack and 9 from the runner loop.

Canonical request flow

A chat request enters at the gin route and stays inside one handler for most of its life. Every step below is a call out of ChatHandler, in the order its call sites appear.

1. r.POST./api/chat (server/routes.go:1909) registers the route, wrapped in withInferenceRequestLogging.
2. server.Server.ChatHandler (server/routes.go:2440) validates the request body and keeps the request through to line 2941.
3. server.parseAndValidateModelRef (server/model_resolver.go:36) splits the model string, and a cloud source sends the request straight to the proxy path.
4. server.getExistingName (server/routes.go:1222) searches the models directory for the longest prefix match and fills in the parts of the name that already exist there.
5. server.Server.getModel (server/model_inference_cache.go:116) serves the model out of the inference cache, falling back to GetModel in server/images.go; a miss on disk becomes the handler's 404.
6. server.Model.Capabilities (server/images.go:112) reports what the model supports, which is where a thinking request on a non-thinking model gets rejected.
7. server.Server.scheduleRunner (server/routes.go:202) checks the model against those capabilities, then blocks on s.sched.getRunner until a loaded runner comes back.
8. server.filterThinkTags (server/routes.go:3122) strips <think> blocks out of the assistant turns before the last user message, for qwen3 and deepseek-r1 only.
9. server.chatModeForModel (server/routes.go:2357) picks the execution mode, and the native path leaves this handler entirely.
10. server.chatPrompt (server/prompt.go:23) renders the messages through the model template, dropping them from the front until the result fits the context window.

Health signals

Symvanta detected 3 dependency cycles across 105 modules (modularity Q=0.90). The largest cycle spans 2 files in the components area. 12 sets of mutually recursive symbols were also detected, the largest being convert (3 symbols).

All three cycles sit in the desktop UI, under app/ui/app/src: two between React components and one between the useChats and useSelectedModel hooks. The Go server has none. Seven of the twelve recursion groups are recursive-descent walks over a nested value: two in the model output parsers (parseArray / parseObject / parseValue, and the Olmo3 equivalents) and five in model/renderers, the schema and tool-argument formatters.