Purpose-Built for AI: Unifying KVCache Reuse and GPU Memory Expansion Using CXL to Address One of AI's Most Persistent Infrastructure Challenges

As AI workloads evolve beyond static prompts into dynamic context streams, model creation pipelines, and long-running agents, infrastructure must evolve, too.

PEAK:AIO, a company that provides software-first infrastructure for next-generation AI data solutions, announced the launch of its 1U Token Memory Feature. This feature is designed to unify KVCache reuse and GPU memory expansion using CXL, addressing one of AI's most persistent infrastructure challenges.

As AI workloads evolve beyond static prompts into dynamic context streams, model creation pipelines, and long-running agents, there is a pressing need for infrastructure to evolve at an equal pace. However, vendors have been retrofitting legacy storage stacks or overextending NVMe to delay the inevitable as transformer models grow in size and context. This approach saturates the GPU and leads to performance degradation.

"Whether you are deploying agents that think across sessions or scaling toward million-token context windows, where memory demands can exceed 500GB per fully loaded model, this appliance makes it possible by treating token history as memory, not storage. It is time for memory to scale like compute has," said Eyal Lemberger, Chief AI Strategist and Co-Founder of PEAK:AIO.

In contrast to passive NVMe-based storage, PEAK:AIO's architecture is designed with direct alignment to NVIDIA's KVCache reuse and memory reclaim models, providing plug-in support for teams building on TensorRT-LLM or Triton. This support accelerates inference with minimal integration effort. Furthermore, by harnessing true CXL memory-class performance, it delivers what others cannot: token memory that behaves like RAM, not files.

"While others are bending file systems to act like memory, we built infrastructure that behaves like memory, because that is what modern AI needs. At scale, it is not about saving files; it is about keeping every token accessible in microseconds. That is a memory problem, and we solved it at embracing the latest silicon layer," Lemberger explained.

The fully software-defined solution utilizes standard, off-the-shelf servers and is expected to enter production by Q3. For early access, technical consultation, or to learn more about how PEAK:AIO can support any level of AI infrastructure needs, please contact sales at sales@peakaio.com or visit https://peakaio.com.

"The big vendors are stacking NVMe to fake memory. We went the other way, leveraging CXL to unlock actual memory semantics at rack scale. This is the token memory fabric modern AI has been waiting for," added Mark Klarzynski, Co-Founder and Chief Strategy Officer at PEAK:AIO.

About PEAK:AIO

PEAK:AIO is a software-first infrastructure company delivering next-generation AI data solutions. Trusted across global healthcare, pharmaceutical, and enterprise AI deployments, PEAK:AIO powers real-time, low-latency inference and training with memory-class performance, GPUDirect RDMA acceleration, and zero-maintenance deployment models. Learn more at https://peakaio.com