What this prompt does
This prompt makes the AI a senior AI engineer who optimises LLM cost at scale, designing a Gemini context-caching strategy and returning working SDK code, not just theory. You provide the [model], the repeated [stable_context], the [query_volume], and the [tenancy_model], and it returns SDK code samples for create, query, and refresh plus a short savings-tracking snippet.
The six deliverables build the caching strategy end to end: identify stable prefixes worth caching such as system instructions, reference docs, and schemas; create cached content via the API with an appropriate TTL; query using the cachedContent reference and merge per-request variable input; measure savings as cached versus uncached token counts and cost per call; manage expiry and refresh when the cached source changes; and decide per-user versus shared cache with the security and cost tradeoffs. The structure works because re-sending the same long context on every call is pure wasted spend, and caching is the highest-leverage cost lever before you ever touch model choice.
When to use it
- The same long context - a manual, schema, or system prompt - is re-sent on every call.
- You have high
[query_volume]over a stable prefix and want to cut token cost. - You need to decide between a shared cache and a per-user cache for security reasons.
- You want to measure cached versus uncached cost rather than guess at savings.
- Your cached source changes and you need expiry and refresh handling to avoid stale data.
Example output
You get SDK code for three operations - create cached content with a TTL, query using the cachedContent reference while merging per-request variables, and refresh when the source changes - plus a short savings-tracking snippet comparing cached versus uncached token counts and cost per call, all shaped around your [stable_context], [query_volume], and [tenancy_model].
Pro tips
- Watch cache expiry closely; a stale cache silently serves old reference data and looks like a model bug, not a cache bug.
- Cache only genuinely stable prefixes from
[stable_context]- system instructions, reference docs, schemas - not anything that varies per request. - Choose
[tenancy_model]deliberately; a shared cache for public docs is cheapest, but private data belongs in a per-user cache for security. - Size the TTL against how often the cached source changes, balancing freshness against re-creation cost.
- Use the savings-tracking snippet to confirm the cache is actually being hit, not just configured.
- Reach for caching before changing
[model]; it is usually the higher-leverage cost lever and lower risk. - Refresh the cache the moment the underlying
[stable_context]changes, not on a fixed timer, so updated docs reach users without a stale window. - Keep the per-request variable input small and outside the cached prefix, since only the stable portion benefits from caching and mixing them in breaks the cache hit.
