Trail of Bits' security skills stop calling unscanned code clean

20 hours ago

Trail of Bits' skill pack for Claude Code and Codex merged four fixes between August 27 and September 28 for scans that reported clean on code they never examined: files over 1 MB, undetected languages, dropped third-party rulesets, and a Burp Suite search with its extension missing. If a clean result came from static-analysis before 1.5.0 or burpsuite-project-parser before 1.1.0, re-run it.

Ask

Ask about this presentation

Answers are generated from this presentation.

Chapters

  1. 0:00Trail of Bits' security skills stop calling unscanned code clean
  2. 0:26trailofbits/skills is 44 security plugins for coding agents
  3. 0:53A cross-site scripting sink in a 1 MB bundle came back as zero findings
  4. 1:18The limit is now 20 MB, and files still over it are named in the report
  5. 1:38Part 2 · Three earlier scans that read clean
  6. 1:40The skill looked for 14 file types, and its scanner handles 41
  7. 2:08Three bugs dropped whole third-party rulesets, and the run still said complete
  8. 2:33The Burp Suite parser no longer reports an unverified empty search as clean
  9. 3:07The repository's contributor guide now names this bug
  10. 3:29Part 3 · If you use these skills
  11. 3:31Re-run any clean scan that came from these versions
  12. 3:55Aside: this pack is share-alike licensed, unlike most skill repositories
  13. 4:20Update the plugin, then re-run the last clean scan
  14. 4:40Close
Show transcript

Trail of Bits' security skills stop calling unscanned code clean

github.com/trailofbits/skills
findings: 0 · clean
src/big.js · 1.05 MB · never opened
static-analysis 1.5.0 · merged Sep 28, 2026 · PR #343

Trail of Bits publishes a pack of security skills for Claude Code and Codex. On September 28 it merged a fix to its Semgrep scanning skill. Any source file over one megabyte had been skipped, and the scan still reported zero findings. It's the fourth fix since late August to close the same gap, a scan that reports clean on code it never looked at. If one of these skills gave you a clean result recently, this one is for you.

trailofbits/skills is 44 security plugins for coding agents

github.com/trailofbits/skills
44plugins
Claude Code + Codexone plugin marketplace
≈7,300GitHub stars since Jan 2026
C and Rust review · smart contracts · supply chain · Semgrep and CodeQL · Burp Suite · YARA
README · .claude-plugin/marketplace.json

The repository is trailofbits slash skills. It holds 44 plugins. You add it to Claude Code as a plugin marketplace, and Codex reads the same marketplace. It started in January and has about 7,300 stars. The plugins cover code review for C and Rust, smart contract scanners, supply chain audits, Burp Suite traffic, and a static analysis plugin that drives Semgrep and CodeQL. That last one is where most of these fixes landed.

A cross-site scripting sink in a 1 MB bundle came back as zero findings

static-analysis · semgrep skill
Aug 27Aug 31Sep 1Sep 28
$ semgrep over a 1,050,060-byte bundle + a 36-byte file
before {"findings":0,"filesScanned":1,"oversized":"absent"}
after  {"findings":1,"filesScanned":2,"maxTargetBytes":20000000}
PR #343, test output on semgrep 1.178.0 · issue #298

Here is that September fix, in the contributor's own test. A JavaScript bundle just over one megabyte holds a cross-site scripting sink, sitting next to one tiny file. Semgrep skips any file above its size limit and doesn't mention it in the results, and the skill never raised that limit. So the scan opened one file and reported zero findings. With the fix, it opens both files and finds the bug.

The limit is now 20 MB, and files still over it are named in the report

static-analysis 1.5.0
Aug 27Aug 31Sep 1Sep 28
limit before
1 MB
semgrep's default, never raised
limit now
20 MB
files still over it listed as oversized
PR #343 · semgrep/SKILL.md, success criteria

The fix raises the default limit to twenty megabytes. It also admits the limit still exists. Any file over it is now listed as oversized in the scan results, and the skill's checklist says the report has to show that list. Bundles, generated code and old monoliths are the files most likely to cross it.

Part 2 · Three earlier scans that read clean

Part 2

Three earlier scans that read clean

Three earlier fixes close the same gap.

The skill looked for 14 file types, and its scanner handles 41

static-analysis 1.4.2
Aug 27Aug 31Sep 1Sep 28
report: clean
rulesets never selected
YAML: Kubernetes, GitHub Actions, CloudFormationC#KotlinScalaSwiftElixirApexSolidityAWS policy JSON
PR #281, merged Aug 27, 2026

August 27th. Before it scans, the skill checks which languages a repository contains and picks rulesets from that. It looked for 14 file extensions, while its scanner handles 41. So those rulesets were never selected, and the report read clean instead of incomplete. That covered C sharp, Kotlin, Swift and Solidity, plus Kubernetes and GitHub Actions files. Monorepos also lost their framework rulesets, because the markers only matched the top folder.

Three bugs dropped whole third-party rulesets, and the run still said complete

static-analysis 1.4.3
Aug 27Aug 31Sep 1Sep 28
a third-party malicious-code ruleset, its own log
✅ Scan completed successfully • Findings: 51
merged report: those 51 missing, ruleset filed as failed
PR #250, merged Aug 31, 2026

August 31st. Three bugs in the scan script could each drop an entire third-party ruleset while the run still reported complete. A single CI config file sitting next to the rules was enough to abort one. In another case a malicious-code ruleset logged a successful scan with 51 findings. Those 51 findings were missing from the merged report. Runs like that are now kept, and marked partial.

The Burp Suite parser no longer reports an unverified empty search as clean

burpsuite-project-parser 1.1.0
Aug 27Aug 31Sep 1Sep 28
empty output · “no matching traffic”
extension may not have loaded
exit 3empty output: can't tell a real empty result from a missing extension
exit 4output isn't the parser's JSON: the search flags were dropped
PR #288, merged Sep 1, 2026

September 1st, and this one is outside Semgrep. The Burp Suite project parser searches captured web traffic, and it depends on a Burp extension. The script passed along whatever Burp returned. If that extension was missing, Burp drops the search flags, and an empty answer reads to an agent as no matching traffic. Now an empty result gets its own exit code, and output that isn't the parser's JSON gets another. The author tested this against a stand-in for Burp, and says how real Burp behaves without the extension is still unverified.

The repository's contributor guide now names this bug

trailofbits/skills · AGENTS.md
A checker that inspects zero items must fail, not pass.

“the single most expensive class of bug in a repo like this one, because it is invisible on every read and in every review”

AGENTS.md, “Scripts a plugin ships”

The repository's guide for contributors names this pattern directly. A checker that inspects zero items must fail, not pass. It calls this the most expensive class of bug in the repo, because it's invisible in every read and every review, and it lists earlier examples that stayed green for months. The four fixes are that rule, applied to the scanners.

Part 3 · If you use these skills

Part 3

If you use these skills

So what should you do if you use them?

Re-run any clean scan that came from these versions

plugin.json versions at each merge
static-analysis < 1.5.0files over 1 MB unscanned
static-analysis < 1.4.3rulesets dropped, languages undetected
burpsuite-project-parser < 1.1.0empty search read as clean
plugin.json at 82fe822, ea5327d, 14e5a10

Here's who this reaches. If you ran the static analysis plugin's Semgrep scan before version 1.5.0, files over a megabyte went unscanned. Before 1.4.3, whole rulesets could drop out and several languages went undetected. The Burp parser fix is version 1.1.0. Every one of these fixes adds coverage, so the results to revisit are the clean ones.

Aside: this pack is share-alike licensed, unlike most skill repositories

LICENSE
trailofbits/skills
CC BY-SA 4.0
share-alike: a published modified copy keeps the licence and credits Trail of Bits
cloudflare/skills, huggingface/skills
Apache 2.0
microsoft/skills and supabase/agent-skills: MIT
LICENSE files and GitHub licence metadata, Sep 28, 2026

An aside for teams that copy skills into their own repositories. This pack is licensed CC BY-SA 4.0, a Creative Commons share-alike licence. If you publish a modified copy of one of these skills, it has to carry the same licence and credit Trail of Bits. Cloudflare's and Hugging Face's skill repositories use Apache 2.0, and Microsoft's and Supabase's use MIT.

Update the plugin, then re-run the last clean scan

README · Installation
# Claude Code (restart after)
claude plugin update static-analysis@trailofbits
# Codex: README, "Codex" section
github.com/trailofbits/skills#installationgithub.com/trailofbits/skills/pull/343
README · claude plugin update --help

The exact steps are in the repository's readme. In Claude Code, you update the static analysis plugin from the Trail of Bits marketplace and restart. The readme has a section for Codex. The size fix itself is pull request 343. Then run the scan again on anything that came back clean.

Close

github.com/trailofbits/skills

A file over 20 MB is still skipped, now by name. The Burp fix was tested against a stand-in.

findings: 0 · cleannot scanned, and now it says so

Trail of Bits' security skills stop calling unscanned code clean

One limit on all of this. The size fix moves the cliff to twenty megabytes, and the Burp fix was checked against a stand-in. What changed is that the skills now say when they didn't look. Trail of Bits' security skills stop calling unscanned code clean. The repository is trailofbits slash skills.