AI Autofix: From Error Webhook to Draft PR

An error tracker is a backlog of small, fully described bugs that nobody gets to. The stack trace names the file, the fix is usually a few lines, and it still loses the prioritization fight to whatever the roadmap says this quarter. We got tired of watching that queue grow, so we built autofix: a service that turns a production error webhook into a draft pull request, written by an agent that has never cloned the repository. The agent reads our code exclusively through the Symvanta MCP server, the same endpoint any MCP-compatible agent connects to. It has been running against our own production since early June, and it opened its first correct draft PR the day it went live.

From webhook to draft PR

Three sources feed the pipeline. Bugsink, the error tracker in front of our services, fires a webhook on every new issue. Grafana alerting fires one when a metric crosses a threshold. And a small poller watches the error log of our own MCP server, because a platform that indexes code for AI agents should be filing fixes for its own bugs with it.

Every event is normalized into one shape: title, message, stack frames, and the repository it maps to. A fingerprint over that normalized error, with request ids and other volatile tokens stripped out, collapses duplicates, so a crash loop becomes one fix attempt instead of hundreds. Fresh fingerprints land in a queue, and a worker claims one attempt at a time. Guardrails run the whole way through: a cooldown per issue, a cap on open autofix PRs, and a daily token budget as the runaway backstop.

The agent never sees a checkout

Each attempt spawns a headless Claude Code run in print mode with exactly one tool surface: the Symvanta MCP. Everything local is stripped away:

claude -p "$PROMPT" \
  --mcp-config '{"mcpServers":{"symvanta":{"type":"http","url":"https://mcp.symvanta.com/mcp","headers":{"Authorization":"Bearer <api key>"}}}}' \
  --strict-mcp-config \
  --allowedTools "mcp__symvanta" \
  --disallowedTools "Read,Write,Edit,MultiEdit,NotebookEdit,Bash,Glob,Grep,LS,WebFetch,WebSearch,Task,TodoWrite" \
  --output-format stream-json \
  --max-turns 30

An early version of the prompt skipped over where the code lived, and the agent burned its turns hunting for a checkout before reporting that the working directory was empty. Disabling the filesystem tools was half the cure. The other half was saying out loud what does exist: the prompt now opens by stating that the repository lives only behind the MCP, that the working directory is intentionally empty, and that an empty directory is expected rather than a finding.

From there the workflow is what you would ask of a careful engineer. Resolve the symbols in the stack trace to exact definitions with find_node. Walk relate with kind: callers to understand what the surrounding code expects. Then pull the current source of any file worth changing through the source tool and build the edit against that exact text. The run ends with a structured verdict rather than a patch the agent applies itself:

{
  "canFix": true,
  "rootCause": "one or two sentences on the underlying cause",
  "summary": "what the fix does, in plain language",
  "edits": [
    { "path": "src/foo.ts", "oldString": "exact current text", "newString": "replacement", "why": "reason" }
  ]
}

The service applies the edits, and it applies them defensively: every oldString must match the file at the branch head byte for byte, and a mismatch fails the attempt instead of committing a guess. Clean applies become a commit through the GitHub API and a draft PR whose body carries the root cause, the summary, and a citation trail of every graph call the agent made, so the reviewer can audit the reasoning alongside the diff.

What it actually shipped

The first live attempt set the tone. The day autofix went live, Bugsink reported a 500 in our own MCP text search: an empty query string was hitting a Zod validator that required at least one character, and the validator threw instead of returning an empty match list. Seventeen turns later the agent had resolved the failing schema, traced the caller, and opened a draft PR dropping the .min(1) constraint and returning an empty match list early. It was the diff we would have written by hand.

The second came out of our MCP error log rather than a webhook: a file-read tool handed a directory path crashed with a raw EISDIR instead of answering cleanly. The agent split the error handling so a directory path produces a clear envelope for the calling agent. Correct again, cited again.

The miss matters as much. An internal 500 arrived whose error envelope carried no stack trace. The agent spent its full 30-turn budget searching, then returned canFix: false with its best root-cause analysis. That is the designed outcome for an under-evidenced error: a confident draft PR built on a guess costs a reviewer more than it saves. Evidence in, fix out.

A typical attempt costs about a dollar in model tokens and runs a few minutes end to end.

Why this needs the graph

Strip the Symvanta MCP out of that pipeline and every step degrades. A stack frame names a function; find_node resolves it to the exact definition and signature, where a text search returns every string that happens to match. Understanding a bug means knowing who calls the broken code, and relate answers that from edges resolved at index time; we wrote up why that distinction decides correctness in blast radius analysis. The index updates on every push, so the oldString the agent copies out of source matches the branch head the service commits against. And because the graph spans linked repositories, an error whose cause sits one repo over is still reachable, a failure mode we covered in why AI coding agents fail on large codebases.

There is an operational win too: the fix box holds an API key that can read the graph and a GitHub token that can open draft PRs. No clone of your source sits on an ops server waiting to become part of someone else's incident.

Build your own on Symvanta

Autofix is a few hundred lines of service code around two APIs, and both are available to you today. The recipe:

1. Index your repositories. Create an account at symvanta.com, connect GitHub, and pick the repos. Webhooks keep the graph current on every push.
2. Mint an API key. Interactive editors sign in over OAuth, but a headless service wants a key: create one in workspace settings and send it as a Bearer token to https://mcp.symvanta.com/mcp.
3. Wire an error source. Anything that can POST a webhook works: Sentry, Bugsink, GlitchTip, or your alerting stack. Normalize to one shape and fingerprint before you queue, so repeats collapse.
4. Spawn a headless agent per attempt. Claude Code in print mode with the flags above works out of the box; any MCP-compatible runner does. Give it the stack trace, the repository and project to scope to, and a hard output contract.
5. Apply server-side, open a draft PR. The agent proposes edits as data. Your service validates them against the branch head and opens the PR through your VCS API. The merge button stays with a human.

And the rules that took us weeks of tuning, free of charge:

• Disable every local tool and say why in the prompt, or the agent will spend its budget looking for a checkout that does not exist.
• Demand the structured verdict by a deadline. "Emit the JSON block by your second-to-last turn" survived every prompt revision we made.
• Fingerprint and dedupe before you spawn. Agent runs cost real money; a crash loop should cost you one attempt.
• Fail closed on stale edits. If oldString no longer matches the head of the branch, the attempt dies rather than half-applying.
• Cap turns per attempt and tokens per day. The turn cap also forces convergence: an agent that knows its budget stops exploring and commits to an answer.
• Dry-run into a chat channel first. We piped proposed fixes to Telegram and read every diff before letting the service open its first PR.

Honest limits

Autofix works the long tail: validator bugs, missing guards, error-envelope handling, the fixes whose whole story is in the stack trace. Bugs that need a design decision, a schema migration, or product judgment end in canFix: false, and that is the correct ending; the root-cause analysis still lands in the attempt log, which speeds up triage even when no PR appears. Errors that arrive without a stack trace usually exhaust the budget the same way. And every PR it opens is a draft: review stays mandatory, because the agent's job is the first response, and merging is a human decision.

If you want to see the graph that carries all of this against your own codebase, book a 15-minute walkthrough: we will index a repository you pick and run the same lookups the agent makes, live.