NVIDIA’s blog post examines how attention design—specifically group size, head dimension, and sequence length—shapes dense attention performance for long-context inference. As context lengths increase, attention’s share of inference time grows, rising from 18% to 85% of prefill time for DeepSeek-R1 at 4K, 32K, and 128K context lengths. The analysis distinguishes prefill, which processes full prompts in parallel with large GEMM-M matmuls and is compute-bound, from decode, which generates one token at a time with small GEMM-M matmuls and is memory-bound by KV cache reads from high-bandwidth memory. Speculative decoding can increase GEMM-M and shift decode toward compute-bound. The post grounds its findings in analytical formulas from GEMM-shape arithmetic and measured data from prefill and decode kernels with FP8 for both attention compute and the KV cache. It distills the analysis, including how attention is parallelized across GPUs, into four practical guidelines—a co-design checklist for model developers to raise inference throughput and interactivity on NVIDIA GPUs. The post notes that with prefix caching, common in agentic and multiturn apps, a new turn may have a short input sequence length while attending to a large prefix cache, causing prefill to behave like decode. A future post will cover sparse attention.