Disaggregated inference:
The right chip for the
right workload

The AI agent inference factory

Agent workloads are rewriting the rules of inference. SambaNova's disaggregated architecture gives inference providers faster large-model experiences, more users per unit of compute, and economics built to scale.

GPUs for prefill, RDUs for decode

Adding more GPU scale does not solve the problem. Agent tasks consume large token volumes across many turns, and over those turns decode becomes the bottleneck.

  • Prefill is compute-heavy and highly parallel, which is exactly where GPUs perform best today.

  • Large-model decode depends on memory bandwidth, chip-to-chip communication, and the fast performance at scale.

  • SRAM alone is not enough when the architecture can't scale to large models and long contexts, but RDUs can deliver the memory bandwidth and performance at scale. 

If you are an inference service provider, it’s time to talk to our experts about unlocking faster large-model experiences at a lower cost-to-serve.

Connect with SambaNova Experts

DevTalks: Designing Effective AI Agents

Join SambaNova and CrewAI for an in-depth webinar on Designing Effective AI Agents — exploring how developers, enterprise AI teams, and entrepreneurs can build, orchestrate, and deploy agentic systems that deliver real results.

In this live session, Justin Woo (SambaNova) and Shane K. Johnson (CrewAI) will demonstrate how to combine CrewAI’s powerful orchestration framework with SambaCloud’s blazing-fast inference to create agents that are both intelligent and efficient.

Date: November 18th

Time: 10am PT | 1pm ET 

What You’ll Learn:

  • What AI agents are and why they matter

  • How CrewAI enables multi-agent orchestration and workflow automation

  • How SambaCloud powers scalable, high-performance inference for agents

  • Core principles of effective agent design 

  • Best practices for building and deploying real-world agentic systems

2x the speed of B200-only configurations

Using Nvidia's B200 GPU for prefill and SambaNova's SN40 RDU for decode, the system delivers double the inference speed of B200-only setups.* 

The world's first heterogeneous disaggregated inference 

This demo showcases the world's first heterogeneous disaggregated inference system that combines SambaNova's RDU processors with Intel CPUs and NVIDIA GPUs. SambaNova co-founder & CEO Rodrigo Liang compares two setups side-by-side — one using the disaggregated approach with different chip types working together, and another using GPUs alone.

This complementary approach results in dramatically reduced end-to-end latency. Testing shows disaggregated inference to be 2-3 times faster than GPU-only solutions, making it optimal for premium inference.*

* Verified by Artificial Intelligence