hotpath 0.29 counts exact instructions per function with Callgrind

2 hours ago

hotpath 0.29 can count the exact instructions each measured function runs, by profiling under Valgrind's Callgrind. A single feature flag now turns on a profiling mode. And two command-line changes break scripts: one command is renamed, and hotpath-utils is removed.

Ask about this presentation

Chapters

  1. 0:00hotpath 0.29
  2. 0:19A new icount report
  3. 0:39The same count every run
  4. 1:00Turning it on
  5. 1:21Where counts are missing
  6. 1:41One flag per mode
  7. 1:59log = true keeps coercion
  8. 2:13init-ci is now init-cloud
  9. 2:28hotpath-utils is removed
  10. 2:48Upgrading
Show transcript
  1. hotpath 0.29 can count the exact instructions each measured function runs, by profiling under Valgrind's Callgrind. A single feature flag now turns on a profiling mode. And two command-line changes break scripts: one command is renamed, and hotpath-utils is removed.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade
    hotpath's Ferris mascot, a crab on fire

    release v0.29.0

    hotpath 0.29 counts exact instructions per function with Callgrind

    Also: one feature flag per profiling mode, and two command-line changes that break scripts.

  2. Here's the timing table hotpath prints today. Under Valgrind, those durations stop meaning anything. So a build with the new hotpath-callgrind feature reports an icount section in its place: the inclusive instructions each measured function executed, with calls, an average, a total, and a share of the run.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    Under Valgrind, the timing table becomes an instruction count

    today's timing section

    hotpath timing report: per-function calls, average, P95, total and percent of total

    replaced by section

    icount

    “Inclusive instructions executed per function (Callgrind Ir).”

    Calls, average, total and percent of total for each measured function.

  3. Instruction counts are deterministic, so they make a steady number to compare in CI. To keep them that way, hotpath turns time sampling off under Valgrind, since a random draw would change a caller's count between runs. The JSON report records which Callgrind version counted, and hotpath's own drain benchmark now runs this way.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    Same code,
    same count,
    every run

    Sampling off

    Every time sampling rate is forced to 0.0 under Valgrind.

    Version on record

    JSON reports carry tool, e.g. callgrind-3.24.0. Counts from different versions don't compare.

    Used in CI

    hotpath's own drain benchmark now runs under Callgrind, with Valgrind pinned.

  4. To turn it on, add a feature to your crate that enables hotpath-callgrind. Then run under Valgrind's Callgrind tool. The repo's example sets Cargo's runner to do it. With a custom output file, set HOTPATH_CALLGRIND_OUT_FILE to match. Run normally, the same build prints timing, for one extra call per measured invocation.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    Add the feature, then run under Callgrind

    Cargo.toml

    [features]
    hotpath-callgrind = ["hotpath", "hotpath/hotpath-callgrind"]

    the repo's own example run

    CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUNNER="valgrind -q --tool=callgrind" \
      cargo run --release -p test-callgrind --example instructions \
      --features hotpath,hotpath-callgrind

    Custom --callgrind-out-file? Set HOTPATH_CALLGRIND_OUT_FILE to the same value.

  5. Counting only works on x86_64 Linux. And a few functions are left out. When one macro_rules definition expands several measured functions from the same spot, they share a marker, so hotpath warns instead of merging their counts. Synchronous track_caller functions and measure_block get no counts.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    counts only on

    x86_64
    Linux

    Every other target reports that it is not under Valgrind.

    left out of the icount table

    Functions one macro_rules! expands from one spot

    They share a marker, so hotpath prints a warning instead of a merged count.

    Sync #[track_caller] functions

    No marker, so caller tracking keeps working.

    measure_block!

    Gets no marker.

  6. In 0.28, profiling allocations meant passing two features, hotpath and hotpath-alloc. Now every profiling feature turns on hotpath by itself, so one flag is enough. The README lists hotpath first in your own feature, too. Old two-flag commands still build.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    Every profiling feature now turns on hotpath itself

    0.28

    cargo run --features='hotpath,hotpath-alloc'

    0.29

    cargo run --features='hotpath-alloc'

    your Cargo.toml, as the README now writes it

    hotpath-alloc = ["hotpath", "hotpath/hotpath-alloc"]
  7. A fix for log equals true. This function returns a reference to a String where its signature promises a string slice, inside an Option. The logging wrapper now keeps that coercion to the declared return type.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    log = true keeps return type coercion

    crates/test-tokio-async example

    #[hotpath::measure(log = true)]
    fn label_parts(&self)
        -> Option<(usize, &str)> {
        Some((self.0.len(), &self.0))
    }

    &self.0 is a &String. The signature says &str, inside an Option.

  8. The first breaking change renames init-ci, the command that sets up hotpath Cloud CI through your coding agent. It's now init-cloud, with the same agent option and the same forks flag. Update any script or doc that calls the old name.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade
    breaking

    init-ci is renamed init-cloud

    0.28

    $ hotpath init-ci --agent claude

    0.29

    $ hotpath init-cloud --agent claude
    $ hotpath init-cloud --agent claude --forks
  9. The second removes the hotpath-utils binary and the utils feature that built it. Its two commands go too: profile-pr, which profiled a pull request against main and posted a GitHub comment, and compare, which diffed two JSON reports. For pull request comments, the docs describe hotpath Cloud.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade
    breaking

    The hotpath-utils binary and its utils feature are removed

    hotpath-utils profile-pr

    “Profile a PR, compare with main branch, and post a GitHub comment”

    hotpath-utils compare

    “Compare two JSON reports and print the diff”

    For pull request performance comments, the docs describe hotpath Cloud: hotpath init-cloud.

  10. To upgrade, install the CLI at 0.29 and bump the dependency to 0.29. Hold off if a script still calls hotpath-utils or init-ci. Instruction counts need x86_64 Linux. Everywhere, 0.29 takes one feature flag where it took two, and on Linux it now counts the instructions your measured functions run.

    Slide text
    ~$ hotpath-rs
    added changed fixed breaking upgrade

    Upgrade to 0.29

    the CLI

    $ cargo install hotpath --version '^0.29'

    your Cargo.toml

    [dependencies]
    hotpath = "0.29"

    wait if a script calls

    hotpath-utils

    hotpath init-ci

    hotpath's Ferris mascot

What happened in AI today, in your inbox

One short email each evening. Read it and you’re caught up.

Built in New York with ❤️