Token savings, measured before they are claimed

A coding client that says it saves tokens should be able to say how much, on what work, and where the number stops being true. Warden’s figures are now kept that way. Each one below comes from live requests or from replaying real recorded sessions, and each is filed with its evidence and its limits.

The prompt cache, live

On a long document of about 41,000 tokens, each follow-up request read 99.5 to 99.7 percent of its input from the provider’s prompt cache, on par with the provider’s own coding client given the same prompts. That rests on two live sessions per client, a small sample, and it is stated as one.

Real sessions, replayed

Small live tasks finish before context handling has anything to do, so the bigger test is a replay: 287 older coding sessions from real daily work, re-billed as if Warden had been managing what stays in each conversation. Pooled, input tokens came out 14.9 percent lower and dollars at API list prices 5.8 percent lower, with nearly all of the saving in sessions of 100 requests or more. Of the 287, 58 got cheaper, 223 came out unchanged, and 6 got dearer, the worst by 21 percent.

A replay assumes the model would have made the same requests afterwards. A real model might read a file again instead, so these are upper bounds, not promises. The replay also paid for itself: it caught an earlier default that made some sessions cost more, and the figures above are for the corrected one.

What is not claimed

Running TheAuditor beside a session is meant to cut what a model has to read. That has not been measured yet, so no number is offered for it. Seeing what a session costs tells you what you are spending; this record is about what the handling saves, and where it does not.

Warden is a working proprietary development build. Supported providers, models, and cost behavior will be stated with its public release.