A multi-layer semantic caching architecture for production LLM systems reduced costs by 45-50% and improved P95 latency to under 2 seconds, from a previous approach where P95 latency approached 5 seconds at millions of queries per month. The system caches at multiple granularities within the agentic pipeline rather than end-to-end response caching alone. Caching the agent's planning decisions achieved a 50% hit rate even with conservative similarity thresholds. A single query involves multiple LLM calls: Planning/Tool Selection (~8,500 input tokens), Tool Execution (minimal cost), and Summarization (~24,000 tokens), plus smaller models for rewriting and tool selection. Traditional string-based caching fails because semantically identical queries like "weather in san francisco" and "what's the weather in sf" produce different cache keys; initial exact string matching achieved only ~15% hit rate. The architecture implements caching at three layers: Layer 1 (Planner Cache) caches tool calls with a 50% hit rate, Layer 2 (Summarization Cache) caches only when tools use static data with a 35% hit rate, and Tool Execution has no caching as it requires fresh data.