How CXL Breaks the Memory Bottleneck

The Memory Expansion Paradox

As AI and real-time processing workloads continue to grow, systems increasingly rely on more powerful CPUs and accelerators to process large amounts of data. As compute capability increases, system architects are hitting the memory wall faster than ever.

To keep up with this growing demand, systems require larger amounts of high-capacity, low-latency DRAM. However, memory expansion through traditional DIMM slots is limited by the number of memory channels supported by the CPU. Once these slots are fully populated, scaling memory capacity becomes significantly more difficult.

This challenge is especially noticeable in edge systems, where platform size, power budgets, and DRAM costs place additional constraints on system design. When the bottleneck shifts to memory processing, and with DRAM price volatility and supply fluctuations, traditional expansion methods may fall short. While these challenges reveal clear limitations, they also suggest alternative ways to move forward—and a different way to think about memory expansion.

Considerations for Traditional DIMM-Based Memory Expansion 

NUMA Effects

In dual-CPU or multi-socket systems, each processor has its own local memory — an architecture known as NUMA (Non-Uniform Memory Access). Each CPU and its attached memory form a NUMA node. 

While a CPU can access its local memory directly, accessing memory attached to another CPU requires traveling through the inter-processor interconnect, resulting in higher latency.

If memory is not evenly distributed across nodes, the operating system may place application data in a remote NUMA node, increasing latency and reducing performance.

Beyond inter-socket latency, memory performance is also shaped by “how DIMMs are physically arranged” on the motherboard.

DIMM Topology

Memory configuration is also influenced by DIMM topology, which refers to how memory channels and slots are electrically routed on the motherboard.

Depending on the layout, some motherboards perform best when the DIMM slot closest to the CPU is populated first. Installing a second DIMM may reduce the memory frequency.

Other designs are optimized for balanced configurations, where DIMMs are populated symmetrically across memory channels. Installing only one DIMM may not deliver the best performance.

To get the most out of a system, users need to understand their motherboard's DIMM topology and configure memory accordingly. These design considerations have also driven interest in new approaches to memory expansion — which is where technologies like CXL come into play.

What Is CXL, and Why Was It Created?

CXL (Compute Express Link) is an open industry standard for high-speed interconnects, first released in 2019 and backed by a consortium of major technology companies including Intel, AMD, ARM, Samsung, Micron, and others. It defines both the protocol layers that govern how devices communicate and the physical layer, which is built on top of the PCIe electrical interface. This means CXL devices plug into standard PCIe slots while enabling a fundamentally different style of memory access. 

CXL's Three Sub-Protocols

CXL defines three complementary sub-protocols that work together to support different communication patterns:

CXL.io handles basic device communication. It is responsible for identifying, initializing, and managing devices — similar to how PCIe devices are detected and configured by the system. In simple terms, this is what allows the system to recognize a CXL device and make it usable.

CXL.mem is the key enabler for memory expansion. It allows the CPU to access memory on a CXL device as if it were part of the system’s memory. From the operating system’s perspective, this memory simply appears as additional RAM.

CXL.cache enables a CXL device to cache host CPU memory. It ensures that both the CPU and the device are always working with up-to-date data, avoiding inconsistencies when data is shared between them.

CXL Device Types

Devices built on the CXL protocol can be broadly categorized into three types based on their primary function. Type 1 devices are accelerators without onboard memory. Type 2 devices are accelerators that include onboard memory. Type 3 devices focus purely on memory expansion.

Innodisk’s CXL product family, including the CXL Add-In Card (CXL AIC) and the CXL Memory Module (CMM), are both Type 3 devices, allowing processors to access external memory as a native extension of the host’s address space.

How Innodisk CXL Solutions Solve the Memory Bottleneck

Innodisk’s CXL products are designed to push past the limitations of traditional memory architecture. The CMM provides 64GB, 96GB, or 128GB of expansion per module, while the CXL AIC scales up to 256GB via two on-card DIMM slots.

CXL Memory Module                                                                                  CXL Add-in-Card

Beyond simply bypassing the motherboard's DIMM slot count, Innodisk CXL solutions offer several key architectural advantages:

1. Memory Pooling: Addressing the NUMA Challenge

CXL enables multiple hosts to share a unified memory pool, making resource allocation more flexible and helping mitigate latency and imbalance issues commonly seen in NUMA architectures. This capability is particularly valuable for multi-host edge servers operating in unpredictable environments.

Real-world workloads often experience significant fluctuations in data processing demand. According to well-known 5G network traffic reports, daily traffic peaks can reach 2–4× the average load, typically occurring during evening hours, when large numbers of users are simultaneously streaming video, gaming, or accessing online services. Special events such as major sports broadcasts can push network traffic to even higher peaks.

Financial trading infrastructure shows similar patterns. Exchange data indicates that market activity follows a well-known U-shaped intraday trading volume pattern, with the highest activity occurring near market open and close. During major market events, message rates in trading systems can surge dramatically, sometimes reaching 10× spikes as large volumes of market data and trading orders are processed in real time.

5G Traffic Pattern                                                                                  Trading Message Rate

These workload spikes create a fundamental challenge for system architects. Memory capacity must be provisioned for peak demand, even though much of that capacity may remain underutilized during normal operating periods.

This is where memory pooling becomes essential. Instead of overprovisioning memory on each server, organizations can achieve higher resource utilization while still maintaining the ability to handle sudden workload surges.

2. Leveraging the PCIe Interface with Memory Semantics

Although the CXL AIC connects through a PCIe connector, memory semantics allow the CPU to access CXL memory more directly, enabling memory-like access.

Most importantly, because the CPU treats the CXL AIC as a single CXL.mem device, it bypasses traditional DIMM-level topology issues (such as frequency drops). The result is predictable performance and high signal integrity, regardless of your motherboard's internal layout.

Extending CXL from Data Center to Edge

CXL memory expansion can be realized through different form factors, each optimized for specific system architectures.

Innodisk’s CXL Memory Module (CMM) is purpose-built for data center environments, offering high-density memory expansion in a compact EDSFF E3.S form factor. It has also been recognized with industry awards, including FMS Best of Show and the Taiwan Excellence Award 2025.

Building on this success, Innodisk extends the same CXL capability with the CXL AIC for edge deployments. Both the CMM and AIC connect via a standard PCIe connector, but the AIC uses a card-based design that offers greater flexibility in system configuration and budget control.

AIC’s TCO Advantage

Memory pricing is not linear. As density increases, the cost per GB often grows exponentially. High-capacity modules (like 256GB DIMMs) carry a significant "density premium."

By leveraging the CXL AIC, organizations can achieve a significant TCO (Total Cost of Ownership) advantage. For example, populating two 128GB DIMMs on your motherboard plus two 128GB DIMMs on a CXL AIC typically costs less than purchasing two 256GB modules for the motherboard alone.

Flexible Deployment

The CXL AIC enables flexible memory scaling based on actual system requirements, without modifying existing DIMM configurations. This makes it especially suitable for conservative industrial environments.

Depending on your objective, you can optimize your setup:

• For Maximum Expansion: 128GB x2 (Host) + 128GB x2 (AIC). This pushes your edge device to its absolute limits.
• For Cost Efficiency: 64GB ×2 (Host) + 64GB ×2 (AIC). This delivers significant capacity using mainstream, budget-friendly modules, while still allowing future upgrades to higher-capacity configurations as system demands grow.
• For Pure Signal Integrity & Low Latency: Keep the Host empty + 128GB x2 (AIC). By moving the memory workload to the AIC, you bypass complex motherboard topology for predictable, high-speed performance.

Beyond capacity flexibility, the CXL AIC also provides flexibility in physical deployment. It can be paired with Innodisk’s industrial-grade DDR5 RDIMM VLP modules to further reduce system footprint. In addition, while maintaining a half-length design, the AIC supports both full-height and half-height brackets, enabling seamless deployment in space-constrained environments such as edge servers and industrial platforms.

Rethinking Memory Expansion

As the memory wall continues to challenge modern systems, expanding memory is no longer just about adding more DIMMs. It's about choosing the right architecture. CXL opens the door to a more flexible, scalable approach to memory, and Innodisk's CXL solutions bring that capability into real-world deployment, helping you build systems ready for the next generation of workloads.

Related Press Release

1. Innodisk Launches CXL Add-In Card for Scalable Edge AI Memory Expansion

2. Innodisk Unveils Advanced CXL Memory Module to Power AI Servers

Related Products

1. CXL Add-in Card (CXL AIC)

2. CXL Memory Module (CMM)