Trail of Bits' regression-hunt skill only reports a bug a fresh agent can reproduce

Trail of Bits added a plugin called regression-hunt to its skills marketplace on October 20. It confirms a bug in a pull request only when the same proof of concept passes on the old code, fails on the new code, and a fresh agent reruns it. If you merge agent-written changes, the bug report now comes with a failure you can run.

A regression here is a new defect from the change that breaks supported behavior that used to work, on a path a user or attacker can reach. It hunts security flaws, broken features, slowdowns, and builds or tests that now fail.

Separate fresh agents reproduce each candidate, replay it and judge it, on real builds of both versions through a real entry point. The proof has to show correct behavior before the change and the failure after it. Synthetic tests, failures inside one helper, and simulated dependencies can't establish a regression.

Only a reproduced regression counts toward the total. A fix that missed a caller, an old bug the change exposed, and a finding still missing proof each get their own label. And a complete run still doesn't mean the code is bug-free.

The second plugin, patching-bugs, takes one bug you name. One agent reproduces it, and a different agent reruns that proof without seeing the first one's transcript. The fix lands as a single local commit, and a fresh agent checks three trees. The original fails. The patched tree passes. Fully reverting the patch makes it fail again.

It gets up to three repair attempts. Each proven patch then goes through regression-hunt, and any reproduced regression marks it Not ready. Only a ready patch gets a maintainer report, and the plugin never pushes or opens a pull request.

Both plugins need an agent host that can start fresh agents and run your code. Without that, candidates stay unresolved, and confirmations still need a human review. Both load in Claude Code and Codex. Which agent would you run regression-hunt in?