The cost was the prompt, not the model
A contact-enrichment loop was re-sending the same five thousand token ruleset on every single call. Caching it cut input cost by about 55 percent and changed the output not at all.
The problem
Enriching a 250,000-contact database meant one model call per contact, and every one of those calls carried the same long instruction block. The block never changed. It was being paid for every time, as if it were new.
What we did
Prompt caching is a billing mechanism, not a quality trade-off. The stable ruleset moves into a cached prefix and only the per-contact payload stays variable, so the model reads the same instructions at a fraction of the price. The accuracy of the extraction is untouched, because the prompt is untouched.
The result
- About 55 percent cut in input cost, with no measurable change in accuracy.
- Per-contact cost logged, so a rollout across the full database could be forecast before it was authorised rather than discovered afterwards.
- The same pattern now applies to every long-ruleset job in the studio.
The same engagement also produced the internal A&R and reporting dashboard the team runs on, a custom sequence engine with per-rep authenticated send and open tracking, and a rebuild of several reports after an upstream API deprecated the endpoints they depended on.