BLOG
OpenAI’s prompt caching guide describes a 12-hour retention option for eligible API requests. For a business that repeats a stable instruction or reference prefix, it may change the cost of some calls. It is not a promise that every prompt becomes cheaper, and it does not mean customer data will be deleted after 12 hours. This article explains how to evaluate the option using actual usage, without quoting an invented saving.
What the 12-hour option changes
Measure total usage before and after changing the configuration, using the same representative workload.
Caching can reuse a matching beginning of a prompt instead of treating every repeated prefix as entirely new input. The longer option extends the period in which an eligible cache entry may be reused. Confirm supported models, request parameters, and current rates in the official guide and pricing table. The value depends on repeated use of the same eligible prefix within the applicable window.
Find the genuinely reusable prefix
If each request has a different instruction, a large changing document, or a different opening section, the expected reuse may be small. Put stable instructions and approved reference material before variable customer details when the API guidance recommends that structure. Do not rearrange sensitive information merely to chase a cheaper token rate; apply the company’s data rules first.
Who can use it?
API availability is tied to the account, model, project limits, and supported regions. The United States is on OpenAI’s supported-country list, but that alone does not prove a particular model and cache configuration are enabled in your project. Check the current Responses API request reference before implementing the parameter.
This is an API configuration, distinct from a ChatGPT application subscription. A team using a third-party automation platform also needs to know whether that platform exposes the relevant setting. The published API feature cannot be assumed to appear in every connector immediately.
| Item | What to check |
|---|---|
| Model | Current eligibility and configuration |
| Prompt | Stable prefix and variable sections |
| Billing | Cache writes, reads, ordinary input, and output |
| Data | Retention terms beyond cache lifetime |
Estimate cost from the whole request
A cache write, cached input read, ordinary input, and generated output can carry different rates. The cheaper read rate is not a price for the entire request. Build an estimate from a representative workload: how many tokens form the stable prefix, how often it repeats during 12 hours, how much variable input follows, and how much output is produced. Include failed or repeated calls.
Check recorded cache hits
Run a small pilot and inspect the usage data returned by the API. Compare calls that did and did not reuse the prefix, then check the invoice against the model’s current price. A spreadsheet based only on theoretical perfect cache hits can understate the bill. If the application sends requests sporadically, the longer window may still produce few hits; measure rather than assume.
Example: an internal support draft
Consider a service company that prepares internal response drafts from incoming inquiries. The workflow uses a stable instruction describing tone, permitted claims, escalation rules, and output fields. Each inquiry then provides a different message. The stable instruction may be a candidate for caching; the customer message should remain separate and only contain necessary details.
First, verify that the instruction is approved and versioned. Then test several inquiries, including a missing fact and a request that must be escalated. Confirm the drafts still meet the team’s rules before measuring cost. Save a prompt version with each result so an editor can investigate a changed answer. If a policy changes, expect the cache behavior and comparison to change too.
Do not use the cache as a knowledge base for facts that need frequent updates. An outdated price, service condition, or customer record must be corrected at the source. Refresh the relevant reference before asking the model to draft an answer. The OpenAI data guide addresses broader retention and handling; a cache duration is not a deletion guarantee.
When is a test worthwhile?
Make the pilot reproducible. Save the prompt template without customer secrets, a set of authorized sample inputs, the model identifier, cache setting, and the date of the pricing page you used. Run a control with the same workload and no longer cache option. Then compare total billed usage and approved output quality over several hours rather than one immediate replay. A quick repeated call may exaggerate what a normal workday will achieve. Include the person who maintains the integration in the review, because added configuration and troubleshooting have a cost even when token spending falls.
Keep changing facts fresh
If a prompt includes retrieved documents, identify which part is genuinely stable. A policy that changes weekly may not belong in a long-lived prefix. An appointment schedule or account balance certainly should not be assumed stable. Design the request so the latest fact is fetched and checked at the time of use. Monitor for stale responses after updating a source document. When a customer-facing answer depends on current information, human approval and a direct source link remain important regardless of how the prompt was cached.
Test the 12-hour option when many API calls share a substantial, approved prefix and the team can inspect actual cache hits. Keep the original request route available and compare quality as well as cost. If the implementation becomes more complex than the saving it produces, a shorter prompt or simpler process may be the better choice. Recheck current documentation before deployment because rates, model support, and parameters can change.
