Together AI announced the Together Compiler (TKCC) on November 4, 2026, a system designed to optimize AI inference workloads dynamically and continuously. The compiler uses high-throughput agents and persistent engine technology to adapt to diverse models, hardware platforms, and traffic patterns. Together AI reports its infrastructure serves trillions of tokens daily across thousands of customers. TKCC employs agents to rewrite GPU code and optimize inference engines, and its persistent engines hot-reload candidate optimizations, minimizing GPU idle time and increasing experimentation throughput by 252 times compared to baseline agentic systems. In less than 24 hours, TKCC achieved throughput improvements of 1.3 to 3 times across several open-source models, including Kimi K3, GLM 5.3, MiniMax M3.1-Flash, and DeepSeek v4.1-Flash. It set state-of-the-art records on SemiAnalysis's InferenceX benchmark for low-latency performance, reaching 1,111 tokens per second (TPS) on MiniMax M3.1-Flash and 1,205 TPS on DeepSeek v4.1-Flash, and outperformed open-source megakernels by 1.7 times in the Llama 1B batch size 1 setting. The system also generalizes to weather modeling, partial differential equations (PDE), time-series analysis, and biology. Its architecture includes persistent engines that reduce total setup time from 467 seconds to