New chip design cuts AI processing delays in half while saving power
Researchers have redesigned the core computing circuits used in AI accelerators and wireless systems to dramatically reduce latency and energy consumption. The innovation could accelerate deployment of AI on edge devices and lower operating costs for data centers running machine learning workloads.
Originaltitel: Scalable Low-latency Systolic Arrays using Toroidal and Bi-Directional Dataflows
<p>Systolic arrays (SAs) for matrix multiplication are commonly used in machine learning (ML), wireless communication, and signal processing. Inherently offering high throughput with good data reuse, they are well-positioned for both low-power edge devices and accelerator applications in high-performance computing. Current realizations suffer from startup latency, defined as the time required to fully utilize all processing elements (PEs). In this work, this issue is addressed by introducing bidirectional systolic arrays with connected edges that form toroidal dataflows. The proposed systolic arrays significantly reduce computational and readout latency from 4n−2 to 2.5n−1 clock cycles for an n×n matrix multiplication, while simultaneously reducing energy per operation by up to 43% compared to conventional SAs. Moreover, a variety of differently shaped SAs are synthesized in a 22 nm CMOS technology, and it is shown that the toroidal designs offer a 5%−12% lower silicon area cost.</p>