MiniMax M2.7 is a model used by many of our customers around the world that helps augment their coding and agentic workflows using the fast inference speed of SambaNova’s SN40 to accelerate their tasks. SambaNova’s SN50 chip improves running this model significantly, which we previewed at RAISE with MiniMax M2.7 running over 800 tokens/second as validated by Artificial Analysis.
SemiAnalysis has now benchmarked SambaRack SN50 on MiniMax M2.7 as well, demonstrating two configurations of the same model with different chip parallelism strategies. In the chart below, SN50 is deployed using vLLM appearing with ~800 tokens/second at the fastest interactivity and ~400 tokens/second with throughput starting to approach the fastest throughput of B200s running the same model. This is early validation for SambaRack SN50 on MiniMax M2.7, showing fast decode performance in the part of the stack users feel the most.
Speed Users Feel, Balanced with Throughput Inference Providers Need
This chart matters because it shows the two things inference providers have to optimize at the same time. The x-axis is interactivity, or tokens per second per user, which is the speed developers feel when a coding assistant or agent is generating responses in real time. The y-axis is output token throughput per chip in the chart’s normalization, which points to how much work each accelerator can sustain as providers scale demand. Moving right means a more responsive experience; moving up means more throughput capacity by serving more requests.
As benchmarked by SemiAnalysis, SN50 moves far to the right on interactivity. The furthest-right point uses Tensor Parallel (TP) across 16 chips: The model work for one request is split across the rack so all 16 RDUs contribute to the same decode stream, pushing per-user speed as far right on the Pareto curve as possible. The second point, at roughly 400 tokens per second, uses Tensor Parallel across 8 chips and uses the remaining 8 chips for Data Parallel. Data Parallel replicates the serving workload across chip groups so the rack can serve more concurrent users and increase aggregate throughput while keeping each user responsive.
GPUs + RDUs Improve Inference Performance
This demonstration is not just a speed result; it is a preview of how heterogeneous, disaggregated inference can fit into real provider infrastructure. SambaNova first showed this pattern at COMPUTEX with GPUs for prefill and SN40 RDUs for decode, then moved the architecture forward at RAISE with MiniMax M2.7 running on NVIDIA H200 GPUs for prefill and SambaRack SN50 RDUs for decode.
That architecture matters because inference has two very different phases. Prefill reads the prompt, code, retrieved content, tool state, and prior turns into the KV cache; it is compute-heavy and highly parallel, so GPUs are a natural fit. Decode produces the answer token by token; it is memory-bound, it is the part users feel, and it becomes harder as more users and longer agent workflows are active. SambaRack SN50 is designed for that decode phase, giving providers a way to add purpose-built token-generation capacity while continuing to use existing Hopper/H200 infrastructure for prefill. The Artificial Analysis chart below shows the RAISE setup on MiniMax M2.7, and the SemiAnalysis result above adds the next view of how SN50 decode can be configured for speed, throughput, and concurrency.
Scaling Up from 16 Chips to 256
The current MiniMax demonstration runs today on one 16-chip SambaRack SN50. With 16 chips, the rack can be configured for different decode goals: use all 16 chips in Tensor Parallel to maximize per-user interactivity, or split the rack into 8-chip Tensor Parallel groups with Data Parallel serving to increase aggregate throughput and concurrency.
As SN50 scales to 64 and 256 chips, the same idea gets more powerful. Larger RDU fabrics open the path to expert-parallel configurations, where different model experts are distributed across chips and tokens route to the right experts while decode stays fast. For providers, that is the path from one fast MiniMax endpoint to provider-scale clusters that balance interactivity, throughput, concurrency, and cost-to-serve.
For a visual walkthrough of how scale-up and scale-out clusters can work with SambaRacks in real data centers, explore our virtual SambaNova Data Center Walkthrough.
Ready for Premium Inference?
The demand for AI is here, and agentic workloads are making faster tokens more important than ever for interactive use cases. As models get larger, running them responsively and economically on existing infrastructure becomes harder.
As we noted as an InferenceX supporter, “Premium Inference is a new category, driven by agents that need fast, interactive tokens with the best per-chip throughput on intelligent frontier models. InferenceX™ is one of the few benchmarks measuring this chip performance in the open, across every major accelerator. We look forward to participating in the official InferenceX with SN50 and showing our premium decode performance on frontier open models tracked transparently."