What this prompt does
This prompt implements Gemini context caching to cut costs and latency where repeated system instructions and large reference documents quietly dominate the token bill. It moves through six steps: auditing usage to find cacheable content, creating cached contexts, building a cache lifecycle layer, designing request routing, implementing invalidation triggers, and building a cost-monitoring dashboard. Caching is the cleanest win available, but the lifecycle, TTLs, and invalidation are fiddly to get right.
The variables tailor the cache layer. [endpoint_count] and [min_cache_size] scope the audit, [document_count] plus the TTLs ([system_ttl], [document_ttl], [tool_ttl]) define what gets cached and for how long, and [sdk_language] with [cache_store] build the lifecycle layer. [conversation_types] and [content_source] drive routing and invalidation, while [cache_discount] and [target_hit_rate] feed the cost math. The TTL choices per content type are where most of the tuning effort goes, since they trade freshness against savings.
When to use it
- Your Gemini bill is dominated by repeated system instructions or large reference documents.
- You include the same
[document_count]documents in nearly every prompt. - You want to cache content above
[min_cache_size]tokens to reduce input costs. - You need a cache lifecycle layer that creates, refreshes, and expires caches cleanly.
- You want request routing that picks the right cache per conversation type.
- You need invalidation that triggers when source documents or instructions change.
Example output
Expect a usage audit across [endpoint_count] endpoints identifying cacheable content and estimating savings. The implementation produces CachedContent resources for system instructions, the [document_count] documents, and tool declarations with their respective TTLs; a [sdk_language] lifecycle layer that creates or refreshes caches and stores cache IDs in [cache_store]; request routing that selects caches by [conversation_types]; invalidation triggers tied to [content_source]; and a cost dashboard comparing cached vs uncached usage, tracking hit rate, and alerting when it drops below [target_hit_rate]. It blends planning, code, and dashboards, with the lifecycle logic written to renew caches before they expire so cold starts don't reintroduce latency.
Pro tips
- Only cache content that is genuinely stable and above
[min_cache_size]; small or frequently changing content won't pay off. - Set TTLs to match volatility: long
[system_ttl]for stable instructions, shorter[document_ttl]for content that updates. - Implement proactive refresh in the lifecycle layer so caches renew before expiring and you avoid cold-start latency.
- Wire invalidation to
[content_source]so updated documents recreate the affected caches automatically with zero downtime. - Validate the
[cache_discount]and[target_hit_rate]math against real usage; pricing and hit-rate assumptions need tuning before you trust the savings. - Track cache storage costs too, since caching itself has a price that can offset savings at a low hit rate and even reverse them.
