Samsung, SK hynix push new memory layer as AI outgrows HBM alone

South Korean semiconductor giants Samsung Electronics and SK hynix are advancing a new class of memory technology designed to address the growing limitations of traditional high-bandwidth memory (HBM) in artificial intelligence (AI) computing. The move comes as AI applications increasingly require vast amounts of memory beyond what can be efficiently integrated onto individual chips. At the HPE Discover conference in Las Vegas earlier this summer, SK hynix unveiled its second-generation CMM-DDR5 memory modules, built on the Compute Express Link (CXL) 3.2 standard. These modules offer twice the capacity of the first-generation version, reaching 256 gigabytes. The demonstration took place within a pooled-memory server from Liqid, showcasing how CXL memory can provide a scalable solution for AI infrastructure without requiring additional processors. Samsung is also working on its own version of the CXL-based memory module, known as the CMM-D. According to recent reports, the company aims to start production by year-end, although delays in supporting server platforms, primarily from Intel and AMD, could push the launch into 2027. Despite potential setbacks, Samsung has stated it remains committed to its production schedule. The transition to CXL memory represents a strategic shift in AI hardware design. Traditionally, AI systems relied heavily on HBM, which provides high-speed access to memory directly adjacent to the graphics processing unit (GPU). However, as AI models grow in complexity, the demand for memory capacity exceeds what can be physically attached to a single chip. This limitation forces system designers to either add more expensive processors or find alternative ways to scale memory resources. CXL memory addresses this challenge by enabling a shared memory pool accessible to multiple processors. Unlike conventional DRAM, which is tightly coupled to a specific CPU, CXL allows for a more flexible architecture where memory can be dynamically allocated across different components. This approach reduces the need for additional compute resources while maintaining sufficient performance levels for real-time computations. Testing conducted by Samsung demonstrated that a 1-terabyte CXL memory pool achieved approximately 92 percent of the performance of direct DRAM access across eight GPUs. Similarly, SK hynix’s research indicated a 35.7 percent increase in throughput when using CXL as a shared tier for inference tasks. These results highlight the potential of CXL memory to enhance AI system efficiency without compromising computational speed. Despite these advancements, CXL memory is not intended to replace HBM entirely. Instead, it complements HBM by providing a larger, more cost-effective memory tier for less critical operations. This dual-layer approach allows high-speed HBM to handle computationally intensive tasks while CXL manages the broader memory requirements of AI models. The adoption of CXL memory faces challenges, particularly in terms of software optimization. Since CXL memory operates at a lower speed compared to local DRAM, effective management of data placement across different memory tiers becomes crucial. This necessity has led to increased competition not just in hardware but also in the development of memory controllers, firmware, and management tools. As the technology matures, the market for CXL memory is expected to grow significantly. Industry experts predict that demand will surge starting around 2026 as AI applications continue to evolve and require more sophisticated memory solutions. For now, however, the success of CXL memory hinges on the timely availability of compatible server platforms and robust software support. Meanwhile, in a related development, China’s ChangXin Memory Technologies (CXMT) is preparing for a landmark initial public offering (IPO) on the Shanghai STAR Market. Valued at approximately $85.2 billion, the IPO will provide substantial funding for CXMT to expand its production capabilities and close the technological gap with global leaders such as Samsung, SK hynix, and Micron Technology. CXMT aims to achieve a 15 percent share of global DRAM bit shipments, a threshold that would secure its position among the top-tier suppliers. Currently holding about 9 percent, the company projects reaching 11 percent by 2028. With plans to invest in advanced manufacturing processes and higher-value memory products, CXMT is positioning itself as a formidable player in the global memory market.