Table of Contents
AMD had previously announced a special AI event for December 6, 2023, at which it would launch the next-generation Instinct MI300 data center GPU accelerator family and highlight its growing momentum with AI hardware and software partners. AMD’s website and YouTube channel broadcast the event live.
AMD AI Event: Instinct MI300: The World’s Most Powerful AI Accelerator
The main attraction of the AMD AI event was the Instinct MI300, the world’s most powerful AI accelerator, which is based on the new cDNA 3 data center architecture. The Instinct MI300 is designed to deliver unparalleled performance and efficiency for generative AI applications, which create new content or data from existing ones, such as images, text, audio, or video.
The Instinct MI300 offers 2.4 times more memory capacity (192 GB) and 1.6 times more memory bandwidth (5.3 TB/s) than the competition, with HBM3 technology. It also supports sparsity and the latest data formats, including FP8, which reduce memory and bandwidth needs and increase performance.
The Instinct MI300 delivers 1.3 times more teraflops of FP8 and FP16 performance (2.6 and 1.3 petaflops, respectively) than the competition, with cDNA 3 compute units and memory density. The Instinct MI300 is built on the most advanced process technologies and 3D packaging, which enable higher performance and power efficiency.
AMD AI Event: How Instinct MI300 is Built and Integrated
The Instinct MI300 is composed of four IO dies (IODs) in the base layer, which have 256 MB of Infinity Cache and all the next-gen IO features, such as 128-channel HBM3 interfaces, PCIe Gen 5 support, and 4th Gen Infinity Fabric that connects multiple Instinct MI300s for 896 GB/s of total system bandwidth.
On top of the IODs, there are eight cDNA 3 accelerator chips or XCDs, which deliver 1.3 petaflops of FP16 and 2.6 petaflops of FP8 performance. These 304 compute units are connected with dense through-silicon vias (TSVs) that support up to 17 TB/s of bandwidth2.
The Instinct MI300 is integrated into an OCP-compliant design, which means it can easily fit into any OCP-compliant system, which is the majority of AI systems today. This makes it very easy for customers to adopt the Instinct MI300 and replace their existing boards. The Instinct MI300 also supports all the connectivity and networking capabilities of the competition, such as PCIe Gen 5 support for 400 Gb Ethernet.
AMD AI Event: How Instinct MI300 is Supported by Open-Source Software
The Instinct MI300 is not only a hardware powerhouse, but also a software-friendly platform. It is supported by ROCm, a modular and open-source software stack that enables broad user accessibility and rapid contribution by the open-source and AI community.
ROCm is optimized for genAI, especially large language models, and has powerful new features, library optimizations, expanded ecosystem support, and performance improvements. ROCm 6, which will be shipping later this month, supports FP16, BF16, and the new FP8 data pipe, incorporates advanced graph and kernel optimizations and optimized libraries, and ships state-of-the-art attention algorithms, such as Flash Attention and Page Attention, which are critical for performing LLMs and other models.
ROCm also contrasts with CUDA, which is proprietary and closed. ROCm leverages the open source community, which moves at the speed of light in deploying and proliferating new algorithms, models, tools, and performance enhancements. ROCm has established tremendous ecosystem momentum and delivers for AI developers.
What Else to Expect from the AMD AI Event
The AMD AI event is not only about the Instinct MI300 but also about the company’s overall AI strategy and partnerships. AMD will showcase how it is advancing AI in various domains, such as gaming, content creation, healthcare, education, and more. AMD will also highlight how it is collaborating with leading AI hardware and software partners, such as Microsoft, Google, Amazon, IBM, and others, to deliver innovative and scalable AI solutions.
In addition, the AMD AI Event revealed some details about its other data center and AI products, such as the MI300A, the industry’s first data center APU for AI and HPC, which began volume production earlier this quarter and is now being built into what we expect to be the world’s highest performing system. The MI300A brings the CPU and the GPU together into one package, sharing a unified pool of memory, and delivers impressive performance for high-precision operations, such as 61 teraflops of FP64 and 122 teraflops of FP32.
AMD AI Event teased some of its upcoming consumer products, such as the Hawk Point Ryzen 8040 series mobile processors, which combine all of its industry-leading performance and battery life, and increase AI TOPs by 60% compared to the previous generation. Ryzen 9 8945, the top of the stack, is significantly faster than the competition in many areas, delivering more performance for multi-threaded applications, 1.8x higher frame rates for games, and 1.4x faster performance across content creation applications.
The AMD AI event was an exciting and informative showcase of the company’s vision and achievements in the AI field. If you are interested in learning more about the Instinct MI300 and other AMD products and solutions, make sure to tune in on AMD’s website or YouTube channel.
