# Do Dynamic Tools Break Prompt Caching?
> A common objection to dynamic tool activation is that swapping tool schemas destroys the provider prefix cache. I audited 10,186 turns across OpenAI, Gemini, and OpenRouter to measure the real cache miss rate, root causes, and token economics.
*Mohammed Reschreiter · 2026-08-30*
Tags: Prompt Caching, AI Agents, Architecture, Telemetry, Token Economics
---
**Short answer: No.**
Dynamic tool activation does not destroy provider prompt caching. In an audit of 10,186 assistant turns across OpenAI, Google Gemini, and OpenRouter, tool swapping caused a cache miss on **only 2.4% to 3.4% of turns**, while avoiding **71.2 Million unused schema tokens** and reducing average turn costs by **32.5%**.
Over 82.5% of prompt cache misses stem from routine multi-file reads, context compaction, and cloud provider 5-minute idle timeouts, not tool switching.
> This study is the empirical follow-up to my architecture deep-dive: [How I Cut 80%+ of Context Overhead in My Coding Agent](/en/blog/how-i-cut-80-percent-context-overhead-dynamic-tools).
---
When I published my architecture for dynamic tool activation in AI coding agents, the most common counterargument on Hacker News was immediate:
> *"Adding tools dynamically seems fine, but removing them is a terrible idea. Swapping schemas changes the prompt prefix and constantly nukes your provider prompt cache."*
On paper, this objection sounds logical. Major LLM providers (Anthropic, OpenAI, Google) cache prompt prefixes. If you modify the system prompt or tool schemas, the prefix hash changes, and the model must write a fresh cache entry at standard or elevated write pricing.
To see whether dynamic tool activation actually harms prompt caching in production, I audited **93 multi-turn coding sessions and 10,186 assistant turns** in Pi across OpenAI, Google Gemini, and OpenRouter.
Here is what the empirical telemetry reveals.
---
## The dataset
I analyzed all interactive coding sessions over an 11-day period where dynamic tool activation was active in daily engineering workflows.
* **Total Sessions:** 93 sessions
* **Total Assistant Turns:** 10,186 turns
* **Total Tool Calls:** 11,461 executions
* **Models Tested:** OpenAI (GPT-5.6 Sol, GPT-5.6 Luna, Codex), Google Gemini 3.7 Flash, and OpenRouter community endpoints.
* **Total Tokens Processed:** 1.22 Billion tokens
---
## 1. How often do agents actually call standby tools?
The core assumption behind the "cache nuke" concern is that an agent constantly swaps tools in and out every two turns.
In practice, software development follows a strict power-law distribution.
Across 11,461 total tool executions:
* **Core 4 tools (`bash`, `read`, `edit`, `write`):** 10,805 calls (**94.28%**).
* **Standby tools activated on demand:** 656 calls (**5.72%**).
```text
Core tools (bash, read, edit, write): ██████████████████████████████ 94.28%
Standby tools (browser, loops, image): █ 5.72%
```
Over 94% of an agent's work consists of reading files, editing lines, and running shell commands. Because the 4 default tools never leave the prompt, the tool schema prefix remains 100% identical and cached for over 85% to 90% of the entire session.
---
## 2. What actually causes cache misses?
Across all 10,186 assistant turns, there were **2,030 cache miss or zero-cache events** (a 19.93% total miss rate, giving an overall 80.08% cache hit rate).
I categorized every single cache miss by its technical trigger:
*Figure 1: Root causes of cache misses across 10,186 assistant turns. Over 82.5% of misses stem from routine file operations and cloud TTLs, not tool switching.*
| Root Cause | Cache Misses | Share of Misses | Share of **ALL 10,186 Turns** |
| :--- | :--- | :--- | :--- |
| **Large File Reads & Context Compaction** | **1,422** | **70.05%** | **13.96%** |
| **Tool Activation & TTL Expiration** | **354** | **17.44%** | **3.47%** |
| **Provider Idle Timeouts (>5 min)** | **150** | **7.39%** | **1.47%** |
| **Turn 0 Session Warmup** | **89** | **4.38%** | **0.87%** |
| **Model Switching** | **15** | **0.74%** | **0.15%** |
| **Total** | **2,030** | **100%** | **19.93%** |
### Key takeaways from the miss data:
1. **Tool activation accounts for only 3.47% of turns.**
Across all providers and models, swapping tools or letting a tool expire after its 2-turn TTL caused a cache transition in only 354 turns. The other **96.53% of turns** ran with zero tool-related cache disruption.
2. **On OpenAI models, the tool miss rate was 2.40%.**
In 4,164 OpenAI turns, tool activation caused exactly 100 cache transitions.
3. **Over 82.5% of cache misses had nothing to do with tools.**
The primary driver of cache churn is dumping 2,000-line files into the prompt, which shifts the context boundary and forces session compaction.
---
## 3. Why cache hit rates vary by provider
When comparing hit rates across providers, the differences come down to how each vendor structures its caching engine:
*Figure 2: Empirical cache hit rates across models and provider endpoints.*
| Provider / Model | Total Turns | Cache Hit Rate | Minimum Token Threshold | Documented Cache TTL |
| :--- | :--- | :--- | :--- | :--- |
| **OpenAI (GPT-5.6 / Codex)** | 4,164 | **86.5%** | **1,024 tokens** | 30 minutes |
| **Google (Gemini 3.7 Flash)** | 3,683 | **78.8%** | **32,768 tokens (32k)** | 1 hour |
| **OpenRouter / Free Endpoints** | 2,237 | **65.2%** | Variable | None / Node cycling |
### The Google Gemini 32k threshold
Google Gemini's 78.8% hit rate initially looked lower than OpenAI's 86.5%. Looking into Google Cloud's documentation explains why:
* OpenAI begins caching prompts automatically as soon as the input exceeds **1,024 tokens**.
* Google Gemini's context caching engine requires prompts to exceed **32,768 tokens** before caching activates.
In short sessions or early turns where context was under 32k tokens, Gemini returned `cacheRead: 0` by design. Out of 772 Gemini cache misses, **471 misses (61.0%)** occurred solely because the prompt had not yet reached Google's 32k threshold. Once sessions grew past 32k tokens, Gemini's cache hit rate climbed past 90%.
---
## 4. The financial math: cache writes vs. schema dragging
Prompt cache reads are heavily discounted, but they are not free. OpenAI charges **10% to 50% of base input price** for cached reads ($0.30 to $1.25 / MTok on GPT-5.6 / GPT-4o).
When you keep 79 static tools loaded in an environment like Codex, you send roughly **12,000 extra schema tokens on every turn**.
Here is the exact financial balance sheet from my 4,164 OpenAI turns:
| Strategy | Extra Schema Tokens Dragged | Cache Rebuild Cost | Cache-Read Fees | Net Cost Impact |
| :--- | :--- | :--- | :--- | :--- |
| **Static Tools (79 tools loaded 100% of the time)** | **49.96 Million tokens** | $0.00 | +$62.45 | **+$62.45 penalty** |
| **Dynamic Tools (4 core + on-demand standby)** | **0 tokens** | ~$0.80 (100 writes) | $0.00 | **-$61.65 net savings** |
Dragging 79 static tools forces you to pay a cache-read fee on 12,000 unused tokens on every single turn for the entire day.
By pruning standby tools after 2 turns of idle time, I spent **~$0.80** on 100 cache rebuilds to save **$62.45** in cache-read fees. That is a **~77x return on investment**.
Across all models and sessions, my average cost per turn dropped from **$0.0609 to $0.0411 per turn (a 32.5% pure invoice reduction)**.
---
## How to maximize cache stability in your agent
Based on these 10,000 turns, here are the three rules for maintaining prompt cache stability:
### 1. Promote high-frequency search tools to core
In my initial setup, `web_search` and `web_fetch` were on standby. Because web search accounted for 36% of all tool activations, promoting both to the default tool set eliminated 36 cache transitions for just ~350 extra tokens.
Keep heavy engines (`browser_use`, multi-agent loops, image generators) on standby, and leave lightweight, high-frequency lookups in core.
### 2. Enforce deterministic prompt sorting
Standby tool lists should be sorted alphabetically (`standby.sort()`) before prompt injection. If tool discovery returns items in varying order across runs, the byte string changes and breaks prefix matching.
### 3. Use bounded reads instead of full file dumps
Large file reads cause 70% of all cache misses. Adding a single guideline to your project instructions stops agents from dumping 2,000-line files:
> Search with `rg -n` to locate target lines first, and use `read` with `offset` and `limit` (100–200 lines) instead of loading entire large files into context.
---
## Conclusion
Dynamic tool activation does not destroy prompt caching.
Because 94% of coding tasks only use basic file and shell tools, your prompt prefix stays stable across the vast majority of turns. The few cache transitions that do occur cost pennies in cache writes, while saving tens of millions of billed schema tokens and keeping the model's context window clean.