At Computex 2024, AMD (NASDAQ: AMD) made a significant impact with its announcement regarding the expansion of the AMD Instinct accelerator family. The keynote, delivered by Chair and CEO Dr. Lisa Su, highlighted a comprehensive multiyear roadmap that promises to deliver industry-leading AI performance and memory capabilities on an annual basis.

The newly updated roadmap is set to kick off with the AMD Instinct MI325X accelerator, which will be available in the fourth quarter of 2024. This will be followed by the AMD Instinct MI350 series, which is slated for release in 2025. The MI350 series will feature the new AMD CDNA 4 architecture, boasting a remarkable 35-fold increase in AI inference performance compared to the previous MI300 series with AMD CDNA 3 architecture.

By 2026, AMD plans to roll out the MI400 series, powered by the forthcoming AMD CDNA ‘Next’ architecture, further enhancing its performance capabilities.

Brad McCredie, Corporate Vice President of Data Center Accelerated Compute at AMD, emphasized the robust adoption of the AMD Instinct MI300X accelerators by major industry players such as Microsoft Azure, Meta, Dell Technologies, HPE, and Lenovo. He attributed this adoption to the exceptional performance and value proposition offered by these accelerators. “With our updated annual cadence of products, we are relentless in our pace of innovation, providing the leadership capabilities and performance the AI industry and our customers expect to drive the next evolution of data center AI training and inference,” said McCredie.

AMD’s AI Software Ecosystem

In conjunction with hardware advancements, AMD’s AI software ecosystem would also be maturing. The AMD ROCm 6 open software stack continues to evolve, enhancing the performance of the AMD Instinct MI300X accelerators. On servers equipped with eight MI300X accelerators and ROCm 6 running Meta Llama-3 70B, customers can achieve significant improvement in inference performance and token generation according to AMD. Similarly, a single MI300X accelerator with ROCm 6 would offer the highest performance on Mistral-7B. Additionally, Hugging Face, one of the largest repositories for AI models, is now testing 700,000 of their models nightly to ensure compatibility with AMD Instinct MI300X accelerators. AMD is also continuing its work on integrating with popular AI frameworks such as PyTorch, TensorFlow, and JAX.

During the keynote, AMD revealed details of its new accelerators and the annual cadence roadmap. The new AMD Instinct MI325X accelerator will feature 288GB of HBM3E memory and 6 terabytes per second of memory bandwidth, utilizing the same Universal Baseboard server design as the MI300 series. This model will be generally available in Q4 2024, promising industry-leading memory capacity and bandwidth.

The AMD Instinct MI350X accelerator, the first in the MI350 series, will be based on the AMD CDNA 4 architecture and available in 2025. It will also use the Universal Baseboard server design and be built using advanced 3nm process technology. Supporting the FP4 and FP6 AI datatypes, it will come with up to 288 GB of HBM3E memory.

A Look Forward to 2026

Looking ahead to 2026, the AMD Instinct MI400 series will be powered by the AMD CDNA ‘Next’ architecture, providing cutting-edge features and capabilities to enhance performance and efficiency for both inference and large-scale AI training.

AMD also highlighted the growing demand for its MI300X accelerators, with numerous partners and customers integrating them into their AI workloads. Microsoft Azure is using these accelerators for Azure OpenAI services and the new Azure ND MI300X V5 virtual machines. Dell Technologies is incorporating them into the PowerEdge XE9680 for enterprise AI workloads. Supermicro is providing multiple solutions with AMD Instinct accelerators, while Lenovo is using them to power Hybrid AI innovation with the ThinkSystem SR685a V3. HPE is leveraging the accelerators to boost AI workloads in the HPE Cray XD675.

This ambitious roadmap would underscore AMD’s commitment to maintaining its leadership in AI performance and memory capabilities, ensuring that its accelerators continue to meet the evolving demands of the AI industry. The annual cadence of new products would reflect AMD’s relentless drive for innovation, aiming to propel the development of next-generation AI models and redefine the standards for data center AI training and inference.