Unlocking Performance: The Azure Maia 300 Architecture Explained

The Azure Maia 300 architecture exists to tackle the growing demands of long-context, reasoning-intensive, and decode-heavy inference tasks. By evolving from its predecessors, it introduces a Software Defined Locally Accessed Dataflow Architecture that separates control path programming from data path execution. This separation is crucial for optimizing performance and efficiency in modern AI workloads.

At the core of Maia 300's design is its multi-chiplet architecture, which utilizes TSMC N2P technology to enhance performance by strategically placing compute resources. This setup allows for a non-uniform memory access (NUMA) organization, enabling software to place data closer to the compute units that require it. The architecture also introduces optimized low precision inference formats, such as MXFP4 and sub-3-bit storage, which significantly reduce memory footprint and data movement. With a memory hierarchy designed for bandwidth per watt, Maia 300 combines on-die SRAM with eight HBM4 stacks, delivering an impressive 288 GB of capacity and 18 TB/s of bandwidth per accelerator.

In production, Maia 300's architecture can double the rack capacity to 96 accelerators, increase compute FLOPs by 36%, and provide 2.5 times the HBM bandwidth. This means you can handle more complex workloads efficiently. However, be mindful of the architectural changes when migrating from previous generations, as they may require adjustments in your deployment strategies.