SK hynix has introduced an upgraded version of its Accelerator-in-Memory based Accelerator (AiMX) card at the AI Hardware & Edge AI Summit 2024. The new prototype offers double the capacity of last year's model, aiming to improve AI performance in data centers and edge devices with high bandwidth and energy efficiency.

SK hynix has showcased its latest Accelerator-in-Memory (AiMX) card at the AI Hardware & Edge AI Summit 2024, held from September 9-12 in San Jose, California. The annual summit, organized by Kisaco Research, is a gathering place for AI and machine learning industry leaders to discuss breakthroughs and advancements in the field.

This year, the focus was on exploring cost and energy efficiency throughout the technology stack. SK hynix, marking its fourth year at the summit, highlighted the capabilities of its AiMX products in enhancing AI performance across various platforms.

The upgraded AiMX prototype, a 32 GB card, was presented with a focus on its ability to provide high bandwidth and exceptional energy efficiency, which are crucial for the operation of large language models (LLMs). As LLMs continue to expand with larger datasets, the need for efficient solutions grows. The new AiMX card combines multiple GDDR6-AiMs to meet this demand.

Demonstrations at the summit included a showcase of the AiMX prototype's advanced processing capabilities in a multi-batch environment using the Llama 3 70B model, an open-source LLM. The demonstration emphasized AiMX's role as an effective attention accelerator in data centers.

In addition to data centers, AiMX addresses the challenges of cost, performance, and power consumption associated with LLMs in edge devices and on-device AI applications. For instance, in mobile on-device AI applications, AiMX can improve LLM speed by three times compared to mobile DRAM while maintaining the same power consumption.

SK hynix also presented its vision for the future of AiMX in the AI era, with plans to develop AiM products for on-device AI based on mobile DRAM. Euicheol Lim, a research fellow and head of the Solution Advanced Technology team, discussed the company's strategy and the importance of collaboration with companies involved in the development and management of data centers and edge systems to further the advancement of AiMX products.

The AI Hardware & Edge AI Summit 2024 served as a platform for SK hynix to demonstrate the AiMX solution's applications in LLMs across various sectors, positioning it as a key player in the advancement of AI and LLM technologies with its low-power, high-speed memory solutions.