ipXchange, Electronics components news for design engineers 1200 627

Why Memory Is Becoming a Bottleneck in AI SoCs

ipXchange, Electronics components news for design engineers 310 310

By Dawn Maindidze


Products


Solutions


Published


20 November 2026

Written by


AI processors continue to gain compute capability, but that compute only matters if data can reach it quickly enough.

As system-on-chip designs integrate more processors, accelerators and memory-intensive workloads, the memory subsystem is becoming a significant performance constraint.

At SEDEX 2026 in Seoul, we spoke with OPENEDGES about why memory bandwidth, latency and on-chip data movement now need to be considered alongside raw compute.

Why AI puts pressure on memory

Neural-network workloads regularly move large quantities of weights, activations and intermediate data between compute blocks and memory.

Adding more compute therefore does not automatically increase application performance.

If the processor spends time waiting for data, additional arithmetic capability can remain underutilised.

This becomes particularly relevant in physical AI and edge systems, where camera and sensor inputs generate continuous data streams that must move through preprocessing, inference and control pipelines.

The engineering problem shifts from simply increasing compute throughput to ensuring that the complete memory path can sustain it.

A memory system is more than DRAM

OPENEDGES approaches this problem through three interconnected pieces of IP.

The OMC DDR memory controller manages DRAM transactions and scheduling.

The OPHY DDR PHY provides the electrical interface to external memory.

The OIC Network-on-Chip interconnect moves traffic between compute blocks and the memory subsystem inside the SoC.

All three influence effective memory performance.

A high-bandwidth PHY provides limited value if the memory controller cannot schedule transactions efficiently. Likewise, an efficient controller cannot remove congestion inside an undersized or poorly optimised interconnect.

OPENEDGES therefore designs these components as a combined memory subsystem rather than treating them as isolated blocks.

Why integration matters

Historically, SoC developers could license a memory controller, PHY and interconnect from separate suppliers.

That approach remains possible, but the integration work sits with the SoC development team.

Each IP block may have different assumptions around latency, quality of service, physical architecture and traffic behaviour.

For complex AI SoCs, those interactions increasingly affect performance.

OPENEDGES addresses this with technologies such as ActiveQoS, where the NoC and memory controller exchange information to prioritise latency-sensitive traffic and reduce congestion.

This allows optimisation to happen across the memory subsystem rather than inside individual blocks.

DDR5 brings physical challenges

The interview also demonstrated OPENEDGES’ DDR5 PHY technology.

At high memory speeds, analogue effects become increasingly important. Voltage characteristics can shift with operating conditions, while timing relationships can drift.

A DDR PHY therefore needs to maintain the correct signalling margins while communicating with the external memory device.

OPENEDGES’ OPHY architecture is designed to track impedance and clock-phase changes without interrupting normal memory traffic.

That physical layer sits underneath the controller and interconnect, but its behaviour directly affects the performance and reliability available to the rest of the SoC.

PPA is no longer the whole physical-design problem

Power, performance and area remain fundamental semiconductor design metrics.

Large AI devices introduce another challenge: physical distance.

As chips grow, high-bandwidth traffic may need to travel further across the die. Long paths complicate timing closure and can require additional pipeline stages, registers and routing resources.

OPENEDGES’ OIC includes long-distance interconnect technology intended to address this type of physical implementation problem.

The company describes a source-synchronous approach where clock and data travel together, helping compensate for timing variation over longer paths.

For large SoCs, the design of the memory system therefore extends beyond DRAM bandwidth. Floorplanning and physical implementation can determine whether that bandwidth can be delivered to the blocks that need it.

Evaluating the complete system

Another important part of the workflow happens before physical implementation.

OPENEDGES provides customers with tools to explore and optimise the memory-system architecture during evaluation.

That allows designers to model memory devices, controller behaviour and interconnect requirements while the wider SoC architecture is still being defined.

This is particularly useful when the workload is known but exact memory requirements are not yet fixed.

Engineers can identify the real bottleneck before committing to the final architecture.

Compute needs data

AI hardware discussions often focus on TOPS, core count or accelerator architecture.

Those metrics remain important, but effective performance depends on feeding that compute.

Memory bandwidth, utilisation, latency, on-chip routing and physical implementation all contribute to how much accelerator performance an application can actually use.

As AI SoCs become larger and increasingly specialised, memory architecture is moving closer to the centre of the system-design problem.

Comments

No comments yet

Comments are closed.

    Find out how we value your privacy

    Get the latest disruptive technology news

    Sign up for our newsletter and get the latest electronics components news for design engineers direct to your inbox.