On 29 July, the same day a rewrite of our sync layer was verified in production, someone opened Claude Code's usage screen and cookd side by side:
cookd displayed 26%
Claude reported 56%Same window, same moment. More than a factor of two apart.
cookd's premise is that it knows how much of your Claude usage window you have burned. The app's access timer, the social feed, the lock screen and the roast all take that number as input. And the number is inferred. There is no API for your quota, no local file recording your limit, and no reliable artefact when you hit it.
An earlier article, Measuring a limit nobody publishes, covers how we built the inference in June: weight four token counters by relative price, then calibrate the ceiling from each user's own rate-limit history rather than shipping a constant. This one is about the day that inference was checked against ground truth, and what made the failure survivable.
The error, worked backwards
Working back from the window's weighted total at that moment, Claude's 56% implied a true limit less than half the size of ours. Our calibrated limit was roughly 2.15× too high.
It was also unstable. Between two syncs an hour apart, the inferred limit had moved by about a quarter. The estimator was not just wrong. It did not hold still.
There are two candidate causes, and both were documented as risks before the reading.
Calibration. The limit is inferred from the rolling-window value at past rate-limit hits, or failing that from clustered pauses. A user who rarely hits the wall gives the estimator few anchors, and everything between anchors is extrapolation.
The weights. The unit is a cost-weighted sum:
export const TOKEN_WEIGHTS = {
input: 1, output: 4, cache5m: 1.25, cache1h: 2.0, cacheRead: 0.1,
} as const;Those five numbers are relative list prices standing in for an undocumented accounting formula. The ADR records them as a named assumption so they can be pointed at when they break. This is when.
Telling them apart takes more than one reading. The method is to collect several paired readings and solve for the weights and the limit together, allowing for the possibility that the weights are not linear at all. Whatever that fit produces, the next section is why it can be applied to every number we have ever stored.
Why it was a constant and not a catastrophe
Until July, the server stored the weighted total: one number per user, already multiplied out. Under that design a 2.15× error is permanent. Every historical row is the product of a wrong constant, and the inputs are gone.
A separate piece of work had just changed storage to raw components (the five counters, unweighted, in 15-minute buckets) so the server could derive the window when it is read. The comment above the weights in the client says what that buys:
They are a guess, which is precisely why buckets store raw components: these five numbers can be re-tuned later and every stored bucket re-derives correctly, including all history.
So the eventual fix is a constant and a re-derivation. No migration, no backfill, no note to users about historical numbers.
We would like to say we planned that. We did not. The buckets were built to fix a lockout (a stale client was locking people out of the app while they worked), and raw components were chosen because they made the merge order-independent. Retroactive correction was listed as a benefit, in one sentence. It turned out to be the one that mattered most.
What else we had wrong
The same stretch of work turned up three more errors in the measurement, each found a different way.
We were reading about half the corpus. Transcript discovery walked one directory level. Subagent transcripts live deeper. Every session that delegated work had its most expensive part invisible, and two statistics, agent runs and the agent-driven share of usage, were zero by construction for every user. Fixing discovery raised everyone's numbers by roughly 8% overnight, and by about 21% in agent-heavy sessions. That discontinuity needed a product answer as well as a code change.
Deduplication has to come first. The same assistant message can appear more than once across the corpus, and every scan re-reads the same files. De-duplicating by message ID before events are folded into buckets was measured to prevent a 347 million weighted-token over-count. Once events are in a bucket their identity is gone, so the ordering is written into the design as a rule that may not be removed.
The scan has a budget. On a corpus of about 150 MB, two measured scans took 3.94 s and 4.20 s. The design sets 5 s at p95 as the trigger to revisit how the corpus is read, and the CLI records every scan's duration so the trigger has data to fire on. The obvious speed-up, an offset cursor, was considered and rejected, because whole-corpus visibility is what makes message-ID dedup and gap-based calibration work. The trade is written down with its trigger, so revisiting it is a measured decision rather than a surprise.
Write the fork before the experiment
One habit from June is worth pulling out, because it is the part of this work we would copy elsewhere.
The plan for the measurement component opened with a phase that wrote no production code. Its job was to answer three questions (does our arithmetic match anything, what is the weighting, is there a reset signal at all) and it wrote the decision down before the answer was known:
A. Reset signal exists in transcripts → the detector prefers the captured reset timestamp when present, falling back to window start plus five hours. B. No reset signal → the detector uses window start plus five hours only.
A watcher tailed transcripts for anything matching rate.?limit|429|usage limit|resets? at|retry.?after
on a machine about to be rate-limited. The answer was B.
It did not stay B. The companion later found that transcripts do carry rate-limit entries, flagged
isApiErrorMessage with a reset time in the text, and the shipped calibration now prefers them.
The gate still did its job. It turned an open-ended spike into a decision with a stated default,
and when the premise changed there was a written premise to revisit.
What it costs to infer instead of read
- Error bars we cannot state. We know the limit was 2.15× off on one machine on one day. We do not know the distribution across machines. We have one hand-collected reading from July and three from June.
- Ground truth needs a human. The only source is a percentage in another tool's interface, read by eye. Collection cannot be automated, which caps how many samples we will ever have.
- The formula is fitted to a usage mix. Cache reads dominate raw totals. If the mix shifts (a new model, different session shapes, more or less caching), a fit to today's mix drifts, invisibly, until someone takes another paired reading.
- The server cannot check any of it. The numbers come from a program on the user's machine reading the user's files. The server stores and re-derives them and can verify none of them. That property has its own article.
What we would do differently
- Take a paired reading every week, from day one. The 26% against 56% took one person about a minute. It had been available for seven weeks. A calendar reminder would have found the error in June.
- Log the calibrated limit on every sync. The hour-to-hour swing was visible only because two syncs happened to be compared. A time series of the estimate is the cheapest instability detector there is.
- Name the assumption in the code, not just the ADR. The weights carry an identifier in the decision record. The constant in code should carry the same one, so whoever re-tunes it lands on the reasoning.
Store what you saw, not what you concluded
Three June readings agreed with each other and were off by a factor of two. Internal consistency is not accuracy, and the original notes had a caveat predicting exactly this failure that nobody acted on.
What made it survivable was not a better estimate. It was having stored the observations instead of the conclusion, so that when the conclusion broke, the data underneath it was still whole.