TL;DR
- At RAISE Summit 2026, SambaRack SN50 ran the fastest MiniMax M2.7 inference in the world, as benchmarked by Artificial Analysis.
- The setup is heterogeneous and disaggregated: one NVIDIA H200 rack (four GPUs) for prefill, one SambaRack SN50 with 16 RDU chips for decode.
- Decode speeds reach up to 850 tokens per second on short-context workloads and over 450 tokens per second on long-context workloads.
- It builds on the COMPUTEX blueprint (NVIDIA B200 for prefill, SambaNova SN40 for decode), now moved forward with SN50 for agentic inference.
- This is the first preview of a roadmap that scales from 16 RDU chips to 128, and up to 256, balancing latency, throughput, and cost-to-serve.
At RAISE Summit 2026, SambaNova is showing the next preview of premium inference: SambaRack SN50 running the fastest MiniMax M2.7 in a heterogeneous, disaggregated inference setup with one NVIDIA H200 rack using four GPUs for prefill and one SambaRack SN50 with 16 RDU chips for decode. The demo builds on the COMPUTEX blueprint we showed live with NVIDIA B200 GPUs for prefill and SambaNova SN40 RDUs for decode, and moves that architecture forward with SN50 for agentic inference.
As benchmarked by Artificial Analysis, the demonstration on MiniMax M2.7 reaches decode speeds up to 850 tokens per second (t/s) on short-context workloads and over 450 t/s on long-context workloads. For inference providers, that matters because even existing H200 infrastructure can be used in the stack for compute-heavy prefill while SN50 adds purpose-built decode capacity for a premium inference experience.
A Preview of SN50 Scale-Up
Today, we are showing what a single SambaRack SN50 with 16 RDU chips can do in a disaggregated setup with one NVIDIA H200 rack using four GPUs for prefill. But the real power of SambaRack SN50 comes from its ability to scale up to 256 chips. Over the coming months, we will show how SambaRack SN50 can scale from 16 RDU chips to 128 RDU chips, giving providers a path to balance low latency, high throughput, and cost-to-serve across frontier models and agent workloads.
For a visual walkthrough of how SambaNova systems fit into real datacenters, explore the SambaNova Datacenter Walkthrough.
Why This Demonstration Matters for Agents
Premium inference means fast, responsive generation on large models, with the throughput and cost-to-serve profile inference providers need to operate at scale. This is incredibly important for agents as they turn token speed into product experience. They do not just answer one prompt. They plan, generate, call tools, search context, inspect files, run tests, validate outputs, revise, and keep going.
Short-context speed makes short turns feel immediate. Long-context speed matters when the agent is carrying the repo, diffs, tool output, logs, tests, retrieved docs, prior decisions, and the plan. That is when sustained decode speed becomes the difference between an agent that feels useful and one that feels stuck. This preview shows why SN50 matters for that phase: It is designed to keep decode moving with an architecture that goes beyond just SRAM.
Inside the Disaggregated Inference Factory
Agentic inference has two very different phases.
Prefill loads the context. It reads the prompt, codebase, retrieved content, files, tool state, and prior turns into the KV cache. That work is compute-heavy and highly parallel, so GPUs are a strong fit. Prompt caching helps reduce repeated prefill work when the same context shows up again.
Decode is the experience. After the context is loaded, the model still has to generate the answer token by token. That path is memory-bound, especially as context grows, and it determines the sustained speed users feel while the agent plans, writes, validates, and repairs.
Disaggregated inference maps compute-heavy prefill to GPUs and memory-bound decode to SambaNova RDUs so tokens can keep moving at premium speed.
Just like we showed at COMPUTEX with SN40, this SN50 preview shows how the same architecture can fit within a disaggregated inference setup. Existing infrastructure like H200 GPUs can keep building context through prefill, while SN50 RDUs keep decode tokens streaming.
For inference providers, that creates a path to use existing GPU infrastructure for a premium inference experience instead of forcing every phase of inference onto the same accelerator. Moreover, this allows inference providers to leverage their existing air-cooled data centers to deliver this premium inference.
Built with the Open-Source Inference Ecosystem
Inference providers and enterprises are increasingly building around open-source serving systems like vLLM and SGLang. These projects shape much of the modern production serving experience: familiar APIs, continuous batching, KV-cache-aware execution, and scheduling behavior designed for high-volume LLM serving.
Disaggregated inference has to fit that world. Developers should not need to learn a new serving model just because the backend is getting smarter. The serving layer should stay familiar while the infrastructure underneath maps each phase of inference to the hardware best suited for it.
RDUs naturally fit within that existing ecosystem. This demonstration was built and measured on vLLM. This shows a path where providers can keep building in the ecosystem they already understand while moving decode-heavy work onto purpose-built RDU infrastructure.
SambaNova’s inference stack connects RDU systems with orchestration, model serving, APIs, and agent frameworks.
The Takeaway
MiniMax M2.7 is the first preview of a broader SambaRack SN50 roadmap. SambaNova is showing fast MiniMax inference on a 16-chip SN50 today, with up to 850 t/s on short-context workloads and over 450 t/s on long-context workloads.
The market does not just need bigger accelerators. It needs architectures that make agentic AI feel fast, stay fast as context grows, and remain economically viable for the providers serving it. This preview shows where SambaRack SN50 is headed: from low-latency decode today to broader scale-up demonstrations across frontier models.