AMD EPYC CPUs are optimized for enterprise AI CPU inference and excel for small and medium language models that a business might run on-premises. The 5th Gen AMD EPYC™ 9005 series server CPUs are the most recent generation, offering high CPU frequencies, high core counts, large cache, and high memory bandwidth. Compared to Intel Xeon chips, two-socket servers using 192-core AMD EPYC 9965 CPUs can achieve up to 1.9x the inference throughput when running XGBoost with the Higgs boson data set compared to two-socket servers using 128-core Intel Xeon 6980 CPUs. Additionally, a 2P high-frequency AMD EPYC 9575 CPU-based server with eight GPUs can deliver up to 6% higher overall inference throughput compared to a similar eight GPU server powered by two Intel Xeon 6960P CPUs, as well as 13% faster time-to-first-token. Small inference models typically have between one and four billion parameters, while medium AI inference can extend into tens-of-billions of parameters, such as Qwen’s QwQ-32B with 32 billion parameters.