Bad key
gstack's eval gate stops rerunning 131 cases on a rejected API key
401 on every case131 rerunsINFRA · no rerun
Bad keyStill redClock
garrytan/gstack · CHANGELOG.md · 1.91.183.0
When gstack's eval gate ran with a rejected Anthropic API key, all 131 cases failed on a 401, counted as assertion failures, and all reran. Now each one counts as INFRA, and none reruns.
Still red
A rejected key is still a red
Failure line: credentials rejected: Invalid API key
An assertion failure that mentions a key is still rerun
Known gap: local --case runs and bisect trials
Bad keyStill redClock
garrytan/gstack · CHANGELOG.md · 1.91.183.0
No pass or fail changes; the failure line now says the credentials were rejected. A real assertion failure that only mentions a key is still rerun. Local single-case runs and bisect trials still classify the old way.
Clock
The /office-hours lifecycle test stops racing the clock
real sleeps vs 300 ms deadline 2 judge starts
fake timers, 8 busy loops 20/20 pass
Bad keyStill redClock
garrytan/gstack · CHANGELOG.md · 1.91.184.0
The /office-hours attempt test used real sleeps against a mocked deadline, so a loaded machine could fail it. It now runs on fake timers, and passed twenty of twenty runs under heavy load.
Your runs
The skills you run behave the same
CHANGELOG.md1.91.183.0 · 1.91.184.0Has a bad API key ever failed a whole run for you?
Bad keyStill redClock
garrytan/gstack · CHANGELOG.md · 1.91.184.0
Both releases change gstack's tests, not its skills, and the changelog has the details. Has a bad API key ever failed a whole run for you?
