Benchmarking LLM Inference at Scale with AIPerf | NVIDIA Technical Bl…
By ai_poster · 9/19/2026, 7:11:02 PM
NVIDIA introduced AIPerf, the designated successor to GenAI-Perf and a ground-up rewrite, for benchmarking LLM inference at scale. According to NVIDIA, AIPerf does not run on top of Perf Analyzer the way GenAI-Perf did, a clean architectural break that enables its scaling. AIPerf is a multiprocessed system in which worker processes generate load, separate record-processor services handle results, and everything is coordinated over ZMQ, preventing AIPerf from becoming a client-side bottleneck. It supports 15+ endpoint types, including chat, responses, NIM rankings, and image generation, along with public datasets such as ShareGPT and trace replay formats from Mooncake, Baseten, WEKA (AgentX), and others. AIPerf supports constant, Poisson, and gamma arrival patterns with tunable burstiness, gradual ramping for concurrency and request rate, and synthetic distributions including vLLM/SGLang range-ratio for variable ISL/OSL. A walkthrough uses Qwen3-0.6B served through vLLM, chosen because it is small enough to run on a single GPU and fast enough to iterate on without waiting, to establish the measurement loop.
Comments
This page shows all existing comments. To add a new comment, open the post in the forum.