Open Code Review: Alibaba open-sourced the code reviewer it validated on a million real tasks

Same model, same two hundred pull requests. Run through Claude Code, it scores 11.6. Run through Open Code Review, it scores 25.1, on a fifteenth of the tokens. This is Alibaba's code review tool, the one that reviewed code for tens of thousands of developers inside Alibaba for two years before they open sourced it in May. It is written in Go, Apache licensed, and it runs on every desktop.

By the end of this, you'll know how a pipeline with hard-coded gates gets that score out of the same model, why Claude Code still finds more of the real defects, and whether you should put this in your pull request pipeline this week. I'll also say where I would not use it.

Here's what it does. You point ocr at a diff: everything you have changed, one commit, or a branch against main. It reads the changes, sends them to whichever model you configured, and hands back comments. Each one is pinned to a file and a line range, and carries the existing code and a suggestion. Ask for JSON and it writes a file your own tooling can read.

If you've asked a general coding agent to review a pull request, you know the three ways it goes wrong. On a large change it cuts corners and skips files. The line numbers drift, so a real finding points at the wrong place. And a small change to the prompt swings the whole result. Alibaba's diagnosis is that a purely language-driven reviewer has no hard constraints, so they built the constraints in code and left the model two jobs.

The whole thing is a pipeline, and the model only gets the wheel in two places. Go code loads the diff from git, decides which files are reviewable, and picks a rulebook per file. Then one model call groups related files. Each group gets its own sub-agent with its own context, up to eight running in parallel. And Go code, not the model, pins every comment back to its lines before anything is written out.

Every changed file passes five gates before a token is spent. Binary files drop first. Your own exclude list wins next. A file on your include list is kept no matter what follows. Then an extension allowlist, then a built-in list of test file patterns. And you can run the preview and see the verdict for every file, with the reason, without calling a model at all. I like that a lot.

Then the files get grouped, so a handler, its service and its test are reviewed in one conversation. That grouping is one model call, and it only sees file paths and line counts, never the code. Three guards keep it sane. Ten files a group at most. A group that would blow the token budget splits back to single files. And if the model's answer is garbage, every file gets its own group. The grouping can fail. The review cannot.

Each group's sub-agent runs a loop. If any file changed fifty lines or more, or the group changed a hundred, it plans first with read-only tools, and that plan becomes a checklist. Small changes skip the plan. Then the main loop runs, up to a hundred tool calls: search the code, read a file, file a comment. And it goes around again with the confirmed findings pinned, one to three rounds depending on the effort you set, stopping early when a round finds nothing new.

This is the part that fixes position drift. A comment quotes the existing code, and Go code slides that quote across the diff to find the exact lines. If it can't match, one more model call re-anchors it. Then a filter pass reads every comment against the diff and throws out the ones it can prove wrong, and a second line pass runs over the whole set. The position is computed, so it does not drift.

Each file also arrives with a rulebook for its type, and there are fifty-two of them, from Java and Go to Terraform and Solidity. The Java one tells the model when to flag a thread safety issue: a check-then-act race, an unsafe lazy singleton, a HashMap written from several threads. And it says when to stay quiet: local variables, read-only access, code that is already synchronized. The do-not-report list is half the rule. That is where the precision comes from.

Now the number from the title. Alibaba built a benchmark from fifty open source repos, two hundred real pull requests in ten languages, with 1,505 defects annotated by more than eighty senior engineers. The dataset is public on Hugging Face. The top row of their table is Open Code Review driving Claude Opus 4.6: an F1 of 25.1, on 385 thousand tokens a review.

Same model, through Claude Code: 11.6. So what does the 25 hide? Recall. Open Code Review finds one in five of the real defects. Claude Code finds 28.9 percent, more of them. It also files nearly six thousand comments to get there, and one in fourteen is right. Open Code Review files 889, and one in three is right, on a fifteenth of the tokens. The score measures signal, and this tool buys signal by staying quiet.

The obvious pushback: Claude Code already has a review command, and CodeRabbit will comment on every pull request for a subscription. True. What those don't give you is a filter you can preview for free, a guard that skips a monster diff before the request is sent, and a JSON envelope you own end to end. One caveat cuts the other way. That benchmark ran a June build of Alibaba's tool against a June build of Claude Code. Both have shipped dozens of releases since, and the table has not been rerun.

Here's the catch, and it is in the FAQ. The model has to make real tool calls. A model that writes them out as text, the way some reasoning models do, loops and returns zero comments. You need Git 2.41 or newer. When the model paraphrases a quote instead of copying it, the comment lands at line zero and you find the spot yourself. And a finding in a file outside the group is off limits, so a bug that spans two groups can go unreported. That is the recall number, explained.

So how do you read one in five? Not as a scanner. A security scanner is judged on recall. This is judged on your attention: every third comment is a real defect, which means you read them, instead of scrolling past forty false alarms to find one. And at 385 thousand tokens, that is a review you can afford on every pull request, on your own key, with whichever model you already pay for.

Where does it sit? Qodo's PR-Agent is the open source one that lives in the pull request. CodeRabbit is the hosted one. Claude Code and Copilot each ship a review of their own. Open Code Review is the one that is a deterministic pipeline you run yourself, with a rulebook per file type, and a track record: twenty thousand active users inside Alibaba, by their own count.

Should you install it? Depends which of these is you. If you're on a team with a pull request pipeline and a model key, yes, this week. The GitHub Action posts the findings on the reviewed commit. If you're solo on Claude Code or Cursor, install the skill and use delegation mode. Your agent runs the review with its own model, and you need no second key. If what you need is recall, an audit, a security pass, I'd run it beside your scanner and not instead of it.

One line installs it from npm, and the command is ocr. A new release shipped today, and that makes 125 since May 21, about one a day. The licence is Apache 2.0, so it is fine at work. Plugins for Claude Code, Codex, Cursor and OpenCode ship in the repo.

So: a pipeline with hard gates, a model that only gets the wheel twice, and a score that comes from staying quiet. Are you running this on your own key, or handing the review to your coding agent? Tell me which, and what it caught. That is alibaba slash open dash code dash review, on GitHub. New repo tomorrow.