Intel Counters AMD’s Data Center AI Claims: 5th Gen Xeon vs. EPYC Turin

In a battle of data center giants, Intel and AMD have been trading blows over AI performance claims. At Computex 2024, AMD unveiled its 5th Gen EPYC CPU family, codenamed Turin, featuring the latest Zen 5 core architecture. The company showcased impressive numbers, particularly in AI throughput workloads, where its 5th Gen EPYC Turin CPUs outperformed Intel’s 5th Gen Xeon family (codenamed Emerald Rapids). However, Intel has now responded, asserting that the right optimizations can tip the scales in favor of its Xeon processors.

The Benchmark Showdown

AMD’s spotlight was on the Llama2-7B Chatbot benchmark, which focused on INT4 inference throughput at a 50ms latency. In a dual-socket configuration, AMD’s 5th Gen EPYC CPUs with 128 cores each achieved up to 671 tokens/s. In contrast, Intel’s 5th Gen Xeon Platinum 8592+ chips, with 64 cores each, managed only 125 tokens/s—a significant 5.4x advantage for AMD.

However, Intel contends that these benchmarks were conducted without the proper software suite for the 5th Gen Xeon SKUs. AMD’s footnotes didn’t provide details about the Intel configuration used.

Intel’s Response

Intel fired back with its own performance benchmarks, revealing a different story. Leveraging the Intel Extension for PyTorch (P99 Latency), the 5th Gen Emerald Rapids Xeon CPUs achieved an impressive 686 Tokens/s—surpassing AMD’s 5th Gen EPYC Zen 5 CPUs. This boost wasn’t solely due to software optimizations; Intel’s 5th Gen Xeons also benefit from added AI hardware accelerators within the Emerald Rapids family.

But there’s more. In translation and summarization workloads, Intel saw gains of 1.2x to 2.3x compared to AMD’s Computex 2024 presentation results.

Memory Matters

Memory plays a crucial role in these benchmarks. While Intel’s current-gen Xeon “Emerald Rapids” CPUs support up to 8 DDR5 memory channels, AMD’s next-gen EPYC Turin family boasts a 12-channel DDR5 interface. Looking ahead, Intel’s upcoming Granite Rapids “Xeon 6700P/6900P” CPUs will match this 12-channel interface and feature the same 128 core count, with P-Cores and up to 288 E-Cores (expected with the Xeon 6900E family in early 2025). The Granite Rapids Xeon 6700P CPUs, available in Q3 2024, will offer up to 86 P-Cores.

The AI Battle Continues

The tech industry’s major players—NVIDIA, AMD, Intel, Qualcomm, and NVIDIA—are fiercely competing in the AI arena. AI TOPs (trillions of operations per second) are the buzz, and demand for capable hardware is soaring. As these companies vie for supremacy, users can expect more exciting developments and intense competition in the data center and AI space. Stay tuned! 🚀🔥