Can Claude fix its own bugs?

Across 90 controlled runs, the answer turned out to be unexpectedly clean: Claude can't find all the bugs in its own code — but when you point them out, it fixes them autonomously. The hard part is noticing, not patching.

Everyone now ships code that an AI wrote. Fewer people ask the follow-up question: when that same AI says "I found the bug and fixed it," should you believe it? Not whether it can fix bugs — whether the act of fixing leaves the code better or quietly worse.

So I built a controlled experiment. Three Claude models each generated the same set of small projects from scratch — naturally, with no bugs deliberately planted. Then each model was asked to find and fix the bugs in its own output, using a range of prompts. Every claimed bug was then re-judged by a single independent referee (Opus 4.8), with web and API bugs proven by actually running the code. The point was to measure four things: how buggy the generated code is, how many of its own bugs a model can find, how many it actually fixes — and how often the fix makes things worse.

The setupGenerate → find → fix → independently verify

Four kinds of subject, generated the way a normal user would ask for them: a marketing landing page, an interactive task board + timer, a zero-dependency REST API, and a small validation library to refactor. Against each, six prompting strategies — from a lazy "check this for bugs" to "build a client and test the API through it" to live in-browser testing. The honest-bug principle is the whole game: nobody injects bugs, so we measure the defects the model actually produces.

Finding 1The code was buggy — and some of it was dangerous

Across one model's run alone, the generated code contained 48 verified-real defects, five of them severity-zero — broken, crashing, or losing data. A 120-line REST API had a search query that crashed the entire server the moment any record lacked a title, and a DELETE on a non-existent id that silently deleted the last real record. These aren't cosmetic. And they're invisible to a glance — you only see them when you run the thing.

Finding 2"Looks fine" is a trap — you have to run it

The six strategies were not equally good at surfacing bugs. The clear loser was visual testing: loading the page and looking at it. On the interactive app, a purely visual pass found zero functional bugs — the timer corruption and the double-counting were invisible because the page simply looked fine. Functional bugs need interaction; server bugs need execution. The two best strategies both involved actually running the code.

Finding 3The real story is in the fixing, not the writing

Here is where the three models pull apart — and not where you'd expect. Their generated code was broadly comparable; one weaker model even avoided a crash the flagship shipped. The divergence is entirely in what happens when they fix. The strong models repair surgically and leave the code better. The weak one changes more, sometimes breaks working code, and reports bugs that were never real.

Start with the writing. Plotting how buggy each model's freshly generated code was, the totals are in the same ballpark — there's no model that writes dramatically cleaner code. What differs is the mix: the flagship actually shipped the most severe defects (five crashes/data-loss bugs), while the others' counts are padded with minor accessibility gaps.

Now the fixing — which is where the models actually diverge. Two things matter: how much of what it found it actually fixed (completeness), and how often a fix introduced a brand-new bug (cleanliness).

The weaker model didn't write much worse code. It fixed code much worse — and "I fixed it" from a weak model is where new bugs quietly enter.

Bug counts are deliberately omitted as a ranking — a higher count often just means a buggier generation (Sonnet's verbose markup had more accessibility gaps to find), not better debugging. Precision, regressions and churn are the honest signals.

Finding 4The hardest test: race conditions

Timing bugs are the cruelest class — non-deterministic, and nearly invisible to both static review and casual execution. So I gave each model a fifth artifact: a real-time dashboard ("LiveBoard") with debounced async search, background polling, async counters, and a save button — then asked each to audit its own code for races and fix them. Six canonical hazards; the referee triggered each one deterministically.

The composite concurrency scores — generation safety, detection, repair, minus penalties for false alarms — landed at Opus 80, Sonnet 77, Haiku 52. The same shape as everything else: the top two are close and clean; the third finds some real races but pads the list with non-bugs and, in one case, fixed a race in the mock backend instead of the client — the wrong layer entirely.

Part IIA skeptic's re-do — and a twist

Everything above has a hole a reviewer would drive a truck through: the same model family that wrote the code also judged whether it was fixed, and each number was a single run. So I rebuilt the core of the experiment to be unfalsifiable-by-vibes. One executable artifact (the API), a **fixed catalog of 10 seeded bugs every model fixes**, and a deterministic test oracle as the referee — no model in the grading loop at all. Then each model fixed that identical code 20 times.

The ranking held, now with real statistics — Opus 7.25, Sonnet 5.10, Haiku 3.60 bugs fixed of 10, every pairwise gap statistically significant. But running it properly surfaced the finding that, in hindsight, the whole article should have led with:

Claude can't find all the bugs in its own code — but when you point them out, it fixes them autonomously. The chart below is just one way of seeing it: ask each model to fix its own bugs and they miss 25%–65% of them; tell them exactly which bug and the miss rate collapses to zero.

"Self-repair" failures are really self-detection failures. The models aren't bad at fixing — they're bad at noticing.

A crucial caveat before anyone declares victory: a perfect fix to a known bug is not a working system. The targeted-repair experiment only proves Claude can patch a bug once it's named — it says nothing about whether the rest of the system still works, whether the bug list was complete, or whether the fix caused a downstream side effect somewhere nobody looked. The way I checked any of that here was the same way you have to check it in real code: run it. A test suite hit those servers on every attempt. That's what produced these numbers, and that's what produces trust.

And that's also the genuinely good news. The recipe that emerges from all of this isn't pessimistic — it's a clean division of labor: an external, independent system runs the software and surfaces what's broken; the AI autonomously fixes what's surfaced. Each component does the thing the other is bad at. With a real validator on one side and an AI fixer on the other, you get an AI coding loop with high confidence — every bug the validator finds gets patched cleanly, and every patch is measured against the validator before it's trusted. Skip the validator and "Claude fixed it" is just an unverified claim from a tool that, by its own measure, can't reliably tell whether something is broken in the first place. Add the validator and the same tool becomes reliable.

And the hole in the original method? I measured it. Using the oracle as ground truth, I had three different model families judge whether bugs were fixed, and scored each judge against the tests:

* Leniency = called a bug "fixed" when the tests say it isn't — the self-grading bias, measured.

Claude — the very model I'd used as the referee — was the most lenient judge, roughly twice GPT's false-pass rate, consistently over-rating fix quality. The effect is small here only because these bugs are clear-cut; on subjective UI or concurrency calls it would be larger. That's a lower bound on exactly the bias the re-do was built to remove — and the reason the verdicts you should trust most are the ones a test suite produced, not a model.

One more humbling note: across all the controlled runs, no model ever scored a perfect 10 — every one consistently left the same spec-ambiguous bugs unfixed. And the ranking didn't fully generalize: on a second artifact (a Python library) the models converged and a more detailed prompt mattered more than which model you picked. A free linter, for the record, caught 0 of 10.

What it meansFind the bugs for it. Then run the software.

The encouraging half of the result is that targeted repair works. Tell any of these models exactly which bug to fix and it's done — surgically, with no regressions, even from the weakest model in the lineup. That's a genuinely useful capability, and a cheap one once you stop expecting the model to also be the source of the bug list, and stop expecting "I fixed it" to substitute for a green run of your actual software.

The discouraging half is the floor on noticing. Even careful, freshly written code shipped crashes and data loss the model itself couldn't see on a re-read. The bugs are real. The fix loop works. What's missing in most "AI does the testing" pitches is the detector — and the way you replace it isn't another LLM, it's actually running the code.

And that's the genuinely optimistic finish. The recipe isn't "don't use AI to fix bugs"; it's pair the AI with anything that can run and validate the software from the outside. A test suite, CI, an integration harness, a human exercising the app — any independent system that surfaces real failures. Plug that into the front of the loop and Claude's autonomous-fix capability becomes genuinely high-confidence. External validator finds the bugs; AI fixes them. That's the loop that actually works.