My AI Code Reviewer Beats Your Human One

Somewhere on your feed right now, someone is explaining that AI-generated code is a legacy time bomb: no foundations, no understanding, and the only safe way to ship it is a human signing off on every line.

For twenty years, I was the person companies called after the time bomb went off. System architect on the full rebuild of a twenty-million-ticket platform. Six years as CTO turning a struggling SaaS into something enterprises would sign. Most recently, rebuilding a 40-person engineering organisation around delivery discipline. Different companies, same wreckage: spaghetti nobody dared refactor, modules documented in folklore, deploys that took courage, firefighting that ate weeks.

Every line of it was written by humans. Plenty of it by senior engineers. All of it had passed human review.

The debate has it backwards. The vibe-coder half of the critique is real: generate code you can’t read and you’re stacking floors on a building you’ve never entered. But the other half, the part where a human eyeball is the quality gate that keeps codebases healthy, is a story we tell ourselves. I watched what that gate actually did on the teams I was hired to rescue: it skimmed, it nodded, it approved before lunch. The shortcuts it waved through became the architecture I was later paid to untangle.

With AI, used properly, that era can end. Not because the model is smarter than your senior engineer, but because for the first time the harness around the code can be stronger than the habits of the people in it. I built that harness: an AI reviews every pull request I ship, and a machine makes ignoring it impossible.

Here’s the harness, layer by layer, and what it still catches.

Six layers between a prompt and a merge

Praxeum, the product this pipeline guards, is a multi-tenant “company brain”: Postgres behind row-level security, data ingestion, background workers. AI agents write most of it, at a pace no human review round keeps up with. That works because the writing happens inside a harness that starts before the first line of code and doesn’t let go until merge:

1. Before any code. Written plans, a domain glossary, repo rules. Architecture invariants live as ast-grep rules a machine can check; hooks block the commands an agent should never run; every feature starts from a failing test.
2. While writing. A local hook runs typecheck, lint and the invariant rules on every file an agent touches.
3. Before the PR. A reviewer agent prompted as a paranoid staff engineer goes over the branch; a security reviewer follows on anything sensitive.
4. On push. CI runs the gauntlet: build, tests, secret scan, static analysis, invariants. And Claude reviews the full diff: Sonnet by default, Opus with a security threat model whenever it touches database, auth, or migration paths.
5. After the review. A merge gate holds the PR red until every finding has a written, human answer.
6. On every fix. A new push means a fresh review, until a round comes back clean. Only then do I merge.

Now the part that surprised me, and the reason this post exists: layer four still finds real things. Code that was planned, written test-first, checked on every edit, and reviewed twice before the PR ever opened still shows up with problems the upstream layers missed. Three weeks in, that has not stopped. If you believe the model plus its own local review is enough, my triage log disagrees, 22 rows deep. That’s not redundancy. That’s the foundation everyone keeps saying AI code doesn’t have.

Wiring up the PR reviewer took an afternoon. Making it trustworthy took three weeks and six silent failures.

The dangerous failure mode of code review was never “nobody looked.” It was always “someone looked, nodded, and moved on.”

Review is advisory. Triage is the law.

Layers four through six run on GitHub and look like this:

A push lands, a filter strips generated files out of the diff (naming every file it dropped), and Claude reviews what’s left, posting one sticky comment tagged <!-- claude-review -->.

The part that makes it work: a commit status called triaged stays red until a comment containing <!-- triage --> exists that is newer than the review. Every finding gets a disposition: fixed (with the commit), consciously deferred (with the reason), or refuted (with evidence). No disposition, no merge; branch protection enforces it.

A disposition is the end of a loop, not a reply: I read the findings, fix what’s real, push, and the push triggers a fresh review. Only a clean round gets its written answer and a green gate.

Two details carry the design: a re-review doesn’t post a new comment, it edits the sticky comment in place, so its timestamp moves and my previous triage instantly stops counting; one answer can never quietly cover two reviews. And the gate never judges what I answered (“re-reviewed, no new findings” is valid); it only knows an answer exists. More on why later.

And the merge button stays human: the coding agents that write most of this codebase are hook-blocked from ever running gh pr merge. The AI reviews; I answer; I merge.

Of the six layers, this is the only one that leaves a paper trail a merge can be blocked on. What follows is why it’s built so distrustful: the layer that guards the code turned out to need guarding itself.

A green check is not a review

Six checks, all green. PR #29 was ready to merge.

Then I opened the review comment. The entire review was 217 characters long, and it ended mid-word: “There’s no ten”.

The job log explained it: stop_reason=max_tokens, content blocks ["thinking"]. The model spent nearly the whole token budget thinking about my diff and got cut off writing it down. The API returned HTTP 200, my pipeline posted the fragment as a finished review, and the check went green.

It wasn’t even the first silent failure. Version one used the official GitHub Action, which ran the model successfully and silently posted nothing. I seeded a blatant return true tenant-isolation hole into a diff: run success, zero comments. Whatever the model found, nobody ever saw it. So the pipeline owns its own last mile now: fetch the diff, call the API, post the comment, first-party.

PR #24 taught me completions can be empty. #29 taught the subtler lesson: the guard at the time was literally [ -n "$REVIEW" ], and a fragment cut off mid-word passes that test. The fix, straight from the script:

# Non-empty is not the same as complete. A completion cut off by the token
# cap ends mid-sentence but still reads as a review, so the old
# `[ -n "$REVIEW" ]` guard posted a fragment as if it were finished.
# Treat max_tokens as a failed attempt: retry, and if it persists, say so.
if [ -n "$REVIEW" ] && [ "$stop_reason" != "max_tokens" ]; then break; fi

The vicious part: the budget covers thinking and response together, so truncation fires hardest on the diffs that provoke the most reasoning: exactly the ones most worth reviewing. Your reviewer dies preferentially where you need it most, and the checkmark stays green.

#29 also produced a triage script that classifies every PR into FINDINGS (my diff’s problem) or TOOLING (the reviewer broke) and appends a row to a committed log: a reviewer that intermittently returns nothing shows up as a pattern, not a one-off you merge past.

My reviewer flagged code it never saw

Three weeks in, the reviewer rated two findings HIGH on a connector PR: a missing unique index, and a missing tenant-resolution key. Alarming. Also wrong: both already existed, past the diff-truncation line. The model never saw them, so it did what language models do with a gap: filled it, confidently, from the plan document sitting next to the code in the diff.

Two days later, the same failure, this time measured:

209 KB of a 279 KB patch was a generated database snapshot, docs, and an issue-tracker export. The reviewer’s budget was gone before it reached the code it was supposed to review.

The fix sounds boring and is anything but: filter generated files out before the budget, cut only at whole-file boundaries (the model can’t tell a truncated hunk from a complete one), and name every dropped file in the posted comment. An omission the reader can see is a known gap. On the fix’s own PR, the reviewer named the two files it wasn’t shown and declined to vouch for them. Same gap, no fiction.

The pipeline reviewed its own paperwork. Twice.

My favorite failure, because nobody broke it but the system itself.

Remember the triage log that gets committed with the PR? Committing it is a push. A push triggers a re-review. The re-review edits the sticky comment, which invalidates the triage I just wrote, which demands a new triage, which gets committed, which is a push…

On PR #59 I went around that loop twice before staring at the workflow file.

The obvious fix, paths-ignore, doesn't work: for pull_request events GitHub evaluates paths against the PR's cumulative diff, not the individual push, and the PR as a whole always contains real code (actions/runner#2324). The fix asks the compare API what this push touched; bookkeeping-only pushes skip the re-review, and the standing triage survives.

The script fails open: a wrongly skipped review is silent (nothing turns red), while a wrongly run review costs one review, so force-push, API error and every other ambiguous case resolve to “review anyway.” When you’re building the thing that guards the thing, the failure costs are never symmetric, and the code should know which side it’s on.

The gate exists because I caught myself cheating

The triage rule started as a convention: always read the review before merging. I broke it within two days, twice: saw green, merged, moved on. The convention didn’t survive contact with me, its author.

That’s why the enforcement is mechanical, and why it gates the answer instead of the findings. Hard-gate on findings and a reviewer that cries wolf gets routed around, the fate of every noisy linter you’ve ever muted. “A disposition newer than the review exists” is a fact, not a judgment; its false-positive rate is zero. Judgment stays with me, the gate just makes exercising it non-optional.

The same week, two of one review’s three suggested fixes were wrong; one would have quietly made a CI gate pass forever if I’d applied it verbatim. The AI detects. I decide.

The objections, and what twenty years of cleanup says about them

The debate around this is loud, so let me take the three strongest objections head-on.

“AI reviewers hallucinate.” They do. Mine flagged two HIGH-severity problems that did not exist; Cursor’s own launch numbers said only half of Bugbot’s flags led to a fix. But put the failure modes side by side: a hallucinated finding costs me a two-minute “refuted, here’s why” line. A skimmed human review costs silently, weeks later, in production. Noisy and checkable beats quiet and wrong.

“AI lacks context, and a human must stay accountable.” Both true, and neither means what people think. The AI doesn’t know the roadmap; neither did the reviewer who approved a 900-line PR in nine minutes after a day of meetings. The classic SmartBear study found real defect-finding takes 60 to 90 minutes per 400 lines. Ask your team when that last happened. And the 1979 IBM slide is right, a computer must never make the call: which is why nothing merges without a named human’s written answer to every finding. More accountability than an approve stamp ever encoded.

“AI code is the next legacy wave.” GitClear’s data is real: duplicated blocks up eightfold, refactoring collapsing. But read what that measures: AI authoring filtered through today’s human review. The stated defense did not defend. Structural rot is precisely what a tireless reviewer plus a merge gate is good at catching, and what a bored human at line 1,400 of a diff is not.

Two numbers say I’m not just talking my own book. Uber measured its production AI reviewer against its own engineers: 51% of human review comments turned out to be real, addressed bugs; the AI’s ran above 65%, with three quarters rated useful by the receiving engineers. And the most honest stat in the field cuts against my side: developers who see low hallucination rates become 2.5 times more likely to merge with no human look at all. Good AI review makes humans stop reading. That is exactly why the gate has to be mechanical.

Foundations were never about who typed the code. They come from the harness around the typing: review that provably happened, findings that can’t be skimmed past, invariants enforced by machines, omissions named. Most teams I walked into had none of that. They had trust, tenure, and a wiki. Give AI that harness and the firefighting era is one I don’t expect to relive.

What it costs, and where your human eyes still win

The honest ledger. Cost: one model call per substantive push, the expensive model reserved for risky paths. Cheapest insurance I buy. The real cost is discipline: a written disposition for every finding is work, even when it’s one line. What humans still win: whether the change should exist at all, product judgment, taste. My reviewer has never once told me a feature was a bad idea. And it’s one repo, 61 PRs, three weeks: receipts, not a randomized trial.

But here’s the part I’ll defend against any team of human reviewers: since the gate went in, zero findings have merged unanswered: by construction, not by discipline. Answered, in writing, attached to the PR, enforced by the same machinery that runs the tests. Every “resolved” thread your team ever collapsed without a decision says your process can’t claim that.

Steal the rules, not the stack

My scripts are bash, my reviewer is Claude, my gate is a commit status. Yours don’t have to be. The rules are the system:

1. Own the last mile. If a third-party action posts your reviews, you don’t know your reviews get posted. Mine provably didn’t.
2. Non-empty is not complete. Check why generation stopped, not whether something came back.
3. Name what the reviewer didn’t see. A review is only trustworthy relative to its inputs, so make the inputs part of the output.
4. Gate the disposition, not the findings. Let the AI be wrong sometimes. Never let it be ignored silently.

The short version, small enough to remember:

If you’re shipping with AI faster than anyone around you can read it: you can still get reviewed. The models have been good enough for a while. The pipeline around them is where the trust comes from, and that part, nobody ships for you.

Open your last merged PR and check one thing: can you prove anyone, human or machine, actually answered what the reviewer said? If you can’t tell, your green checks are decoration.

I’m Bram van Gestel: twenty years of platform rescues and engineering transformations, currently building Praxeum, a company brain for teams. I write up the infrastructure as I build it. If you want the scripts from this post as gists, or want to argue that your human eyes would have caught PR #29, comments are open.

Sources from the objections section

• Cursor Bugbot ~52% resolution at launch: “Building a better Bugbot” (Jan 2026): https://cursor.com/blog/building-bugbot · the “climbed to 76%” claim: “Bugbot Autofix” (Feb 2026): https://cursor.com/blog/bugbot-autofix
• GitClear “The Maintainability Gap” (Jul 2026; +81% duplication since 2023, refactoring under 4%): https://www.gitclear.com/the_ai_code_quality_maintainability_gap
• SmartBear/Cisco review study (the classic; 60–90 min per ≤400 lines): https://smartbear.com/learn/code-review/best-practices-for-peer-code-review/
• IBM 1979 accountability slide (via Simon Willison, Feb 2025): https://simonwillison.net/2025/Feb/3/a-computer-can-never-be-held-accountable/
• Uber uReview (2025; the 51% human baseline and 75%-useful numbers, production-scale): https://www.uber.com/ug/en/blog/ureview
• Qodo State of AI Code Quality (Jun 2025 — still the latest edition, no 2026 yet; the 2.5x merge-unreviewed stat, exact wording “24% vs. 9%”): https://www.qodo.ai/reports/state-of-ai-code-quality/