Trail of Bits’ dimensional-analysis skill now checks the math a code comment calls correct

1 month ago

Trail of Bits' dimensional-analysis skill 3.1.0 for Claude Code and Codex now traces unit and decimal math even where a code comment says it is correct, reports unsure findings at low severity, and ships a five-bug test protocol. https://github.com/trailofbits/skills/pull/371 https://github.com/trailofbits/skills/tree/main/plugins/dimensional-analysis

Ask about this presentation

Chapters

  1. 0:00Trail of Bits’ dimensional-analysis skill now checks the math a comment calls correct
  2. 0:23A comment saying the math is right no longer exempts it
  3. 0:44A test run cleared a contract by quoting its own comment
  4. 1:03Unsure findings still get reported, at low severity
  5. 1:21Overflow and reentrancy stay out of the unit report
  6. 1:40Asking about decimals now starts the skill
  7. 1:57Callers and the code they call are now checked together
  8. 2:15A new test protocol hides five unit bugs, three of them across files
  9. 2:35Claude Sonnet passed the old test without the plugin
  10. 2:55dimensional-analysis 3.1.0 runs in Codex as well as Claude Code
Show transcript
  1. Trail of Bits merged version 3.1.0 of its dimensional-analysis skill, which hunts unit and decimal bugs in code like Solidity contracts. Its validator now traces the math even where a comment says the conversion is correct. If you audit DeFi code with it, a stale comment can no longer wave a bad formula through.

    Slide text
    trailofbits/skills · dimensional-analysis 3.1.0 merged Oct 14, 2026

    Trail of Bits’ dimensional-analysis skill now checks the math a code comment calls correct

    // dimensionally correct
    traced: RAY index used as WAD
    trailofbits/skills PR #371 · agents/dimension-validator.md
  2. Version 3.0.3 told the validator to skip conversions that carried an explanatory comment, and edge cases the docs said were handled. Both exemptions are gone. A contract whose header calls it dimensionally sound now gets its dimensions traced like any other.

    Slide text
    dimension-validator · “Don’t flag as bugs” 3.0.3 → 3.1.0
    - Explicit dimension conversions with comments
    - Well-documented edge cases
    + Comments and documentation are claims about the code, not evidence about it.
    plugins/dimensional-analysis/agents/dimension-validator.md · PR #371
  3. During testing, a contract carried a tag calling it dimensionally correct, and a run cleared the whole contract by quoting that tag instead of checking its arithmetic. Comments go stale, and a unit bug hiding under a reassuring comment is the one a reviewer is most likely to miss.

    Slide text
    dimensional-analysis · tests/fixtures 3.1.0
    “a run cleared the contract by quoting it, voiding the precision measurement”

    A dimensional bug that a nearby comment denies is the bug most likely to survive review.

    tests/fixtures/ANSWER-KEY.md · agents/dimension-validator.md
  4. When the validator is unsure, it must still report the finding, downgraded and labelled needs review. A missed unit bug reaches the client, while a low note costs a reviewer a moment. Critical and high are kept for bugs whose dimensions it has traced.

    Slide text
    dimension-validator · severity 3.1.0
    unsure reported, downgraded, labelled “needs review”
    CRITICAL or HIGH only after the dimensions are traced
    plugins/dimensional-analysis/agents/dimension-validator.md · PR #371
  5. Findings now cover unit mismatches, scaling errors and bad conversions only. Overflow, reentrancy, access control, denial of service and unchecked return values get one closing line at most, so general audit notes don't bury the unit bugs.

    Slide text
    SKILL.md · Reporting Scope 3.1.0
    unit mismatches precision and scaling errors incorrect conversions
    overflow reentrancy access control denial of service unchecked return values
    plugins/dimensional-analysis/skills/dimensional-analysis/SKILL.md · PR #371
  6. The old description waited for a request to annotate units or run a dimensional analysis. Now a request to review code for decimal mismatches, WAD and RAY scale errors, or the wrong token's decimals starts it too.

    Slide text
    SKILL.md · description 3.0.3 → 3.1.0
    3.0.3 Use when someone asks to annotate units in a codebase, perform a dimensional analysis…
    3.1.0 Use when someone asks to audit or review code for unit confusion, decimals or scaling mismatches, WAD/RAY scale errors, or the wrong token’s decimals…
    skills/dimensional-analysis/SKILL.md · .claude-plugin/plugin.json · PR #371
  7. Bug detection used to start one validator per file. Now files are batched by call graph, so a single validator reads both sides of a call. A unit mismatch between two contracts is the kind most often missed when each side is checked alone.

    Slide text
    references/step4.md · bug detection 3.1.0
    MarginVault.solcaller
    →
    InterestIndex.solreturns a RAY index

    One validator sees both files, instead of one validator per file.

    skills/dimensional-analysis/references/step4.md · PR #371
  8. The plugin now ships a test protocol, a cross-margin borrowing protocol with five planted bugs. Here a borrow index with 27 decimals goes through an 18-decimal helper, and only another file shows the index's scale. A correct staking contract sits beside it to catch false alarms.

    Slide text
    tests/fixtures · MarginVault.sol :: debtOf 3.1.0
    uint256 index = interestIndex.currentIndex(); // RAY, 1e27
    return principal.mulWadDown(index) / entryIndex[account]; // WAD helper

    StakingRewards.sol is correct on purpose. A CRITICAL or HIGH finding there fails the test.

    plugins/dimensional-analysis/tests/fixtures/ANSWER-KEY.md · PR #371
  9. The previous test put every bug in one contract, each with a giveaway nearby. Claude Sonnet found all five without the plugin, so the test couldn't tell the plugin apart from the bare model. The new one passes only with all five found and zero critical or high alarms on the correct contract.

    Slide text
    tests/fixtures/ANSWER-KEY.md · scoring 3.1.0
    old test, baseline Sonnet, no plugin
    5 of 5
    in three isolated rounds
    new test, pass gate
    5 of 5
    and zero CRITICAL or HIGH on the correct contract
    plugins/dimensional-analysis/tests/fixtures/ANSWER-KEY.md · PR #371
  10. The same workflow now runs in Codex as well as Claude Code, with five subagents that scan, discover units, annotate, propagate and validate. It needs a writable target and native agent dispatch. Version 3.1.0 ships the new test without a published score on it. The change is pull request 371 in the Trail of Bits skills repository.

    Slide text
    trailofbits/skills · dimensional-analysis 3.1.0 merged Oct 14, 2026
    arithmetic-scanner dimension-discoverer dimension-annotator dimension-propagator dimension-validator

    Needs a writable target and native agent dispatch. No published score on the new test yet.

    github.com/trailofbits/skills/pull/371
    plugins/dimensional-analysis/README.md · SKILL.md · PR #371

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built with ❤️ in New York