SambaRack™
Run the largest models at fast inference speed for agentic ai, right inside the air-cooled data center you already operate.
SambaRack™ SN50 is SambaNova's fifth-generation AI inference rack: state-of-the-art, air-cooled, and purpose-built for agentic AI data centers. It’s energy efficient architecture allows it to generate the most tokens per second per megawatt (MW) compared to any other air-cooled architecture on the market
Built for Performance inference.
SambaRack is the core foundation of the SambaNova Platform: air-cooled racks powered by the SN50 RDU, with compute, memory, networking, storage, and power built in.
The RDU's acceleration libraries map your entire model across multiple RDU nodes, linked rack to rack by a scale-up network so they run as one. This custom dataflow architecture keeps data moving instead of paying a memory round-trip between operations. With the hottest data closest to compute, you get fast inference in racks that need no datacenter rebuild, inside the power, space, and cooling you have today.
What is inside every rack
Compute
RDU server nodes running SN50 processors, with an Intel Xeon host server that orchestrates them.
Memory
The host system memory stages and continuously feeds data to the RDUs.The RDU's three-tier memory architecture optimizes data flow across SRAM for hot local data, HBM for active model weights, and DDR for prompt caches and larger model pools.
Networking
A high-bandwidth data switch for inference traffic, a separate management switch for telemetry and hardware control, and console access for recovery. Inference traffic and management traffic stay separated.
Storage
NVMe on the host node caches prompts across long sessions, which cuts prefill time and keeps agents responsive.
Power
Redundant power distribution, and the whole rack is air-cooled, so it fits in a standard data center with no liquid cooling.
The software that ships with It
SambaRack Manager
Manages, monitors, and controls the RDU nodes. Whatever serving software you run on top, this keeps the hardware underneath healthy.
Acceleration libraries
Compile and map each model onto the reconfigurable hardware, which is what lets the rack run your models at all.
Why a rack of RDUs decodes faster
Decode, the phase that generates each token, is memory-bound. A GPU goes back to memory between operations, so every token pays for the trip. The RDU chains operations in a single kernel and streams data straight through, so there is less data movement, lower latency and less energy per token.
Traditional GPU flow
|
Data detours through memory at each step
|
|
More data movement and overhead
|
|
Higher latency per token
|
SambaRack RDU dataflow
|
Operations chained in a single kernel
|
|
Less memory movement
|
|
Faster inference and better energy efficiency
|
Independently benchmarked with reference architectures built for scale.
SemiAnalysis validation.
Third-party benchmark of SambaRack SN50 on MiniMax M2.7.
World-record speeds.
SN50 runs the fastest MiniMax speeds recorded.
Data center walkthrough.
Video, as evidence the deployment story is real.
Same rack. Four different problems it solves.
Neoclouds and inference providers
Every neocloud rents out the same GPUs, so the tokens all look alike and price is the only thing left to compete on.
- Serve more users: sell a speed tier your competitors cannot list
- Maximize your return: charge premium rates against a cost per token they cannot reach
- Grow without rebuilding: more tokens from the power you have already secured
Model labs
Every GPU serving production traffic is a GPU not training the next model, and compute is what your roadmap is waiting on.
- Serve more users: serve production traffic without taking GPUs off training
- Maximize your return: more research throughput from the fleet you already have
- Grow without rebuilding: keep your training platform, frameworks, and checkpoints where they are
Enterprises
The pilot worked. What stalled it was the bill at production scale, and the fact that regulated data cannot leave your perimeter
- Serve more users: frontier-model speed that holds when the whole company is on it, not just at low load
- Maximize your return: a cost you can budget instead of a usage bill that grows with adoption, on hardware that keeps earning after it has paid for itself
- Grow without rebuilding: your weights, your logs, your data lineage, inside your own perimeter and air-gapped when the data requires it
Government and public sector
Agencies need AI for public services and research, but the data cannot leave the country or, in some cases, the building. Public cloud APIs are not an option.- Serve more users: serve more agencies, more researchers, and more citizen services from the same deployment
- Maximize your return: public funding that goes further, because the hardware pays for itself in months and keeps running for years
- Grow without rebuilding: air-gapped in your own facilities, with no data collection, no prompt logging, and nothing shared with a third party
Deployments come in Scalable Units
A Scalable Unit is the smallest configuration that runs the largest models across a tightly coupled scale-up network, with a spare rack for failover. When you need more capacity, you add another unit.
What's in a Scalable Unit
Required
16 RDU racks1 spare rack for failover
4 scale-up networking racks
What your facility needs
Air-cooled throughout, at roughly 20 kW per rack, with no liquid cooling and no facility retrofitStandard data center rows, deployed alongside the GPU racks you already run
Scale up to deploy bigger models.
Scale out to grow.
Every deployment scales up. The minimum configuration includes four scale-up networking racks that connect RDUs tightly enough to run one or many models across all of them, which is what lets a Scalable Unit serve the largest models. Scale-out is added when you outgrow a single scalable unit; typically when demand grows faster than one unit can serve, when you want to run more models or when more copies of them are needed at once.
Scale up
Four scale-up networking racks at 800G connect RDUs across racks tightly enough that a single model can be split across them and stay synchronized. This is the baseline networking configuration any deployment needs.
Scale out
Two scale-out networking racks at 400G with RoCE are added when demand outgrows a single scalable unit. They connect independent RDU nodes so more requests run in parallel, which raises token throughput and the number concurrent users. Most customers already have a front-end network management rack, if not the Core Services Rack is an optional rack that SambaNova can provide.
GPU clusters requires both
A GPU cluster deployment is disaggregated inference: GPU nodes handle prefill, RDU nodes handle decode, and storage nodes hold the KV cache between them. Scale-up networking runs the model across the RDUs and scale-out networking carries the traffic between the GPUs, the storage, and the RDUs, so both are required. This is the configuration for operators who already run GPUs and want to add decode capacity without replacing them.
Two ways to serve Inference.
SambaRack runs either way. The question is who builds and operates the software on top of it.
Build your own
You want full control of the inference serving software stack. You have platform engineers who know inference infrastructure, and you are prepared to own the upgrades, and the scaling logic. SambaRack runs the open-source engines your team already works with, including vLLM and Dynamo.
License SambaStack
You would rather run a proven and validated software stack than build and maintain one. SambaStack is SambaNova's licensed software stack for SambaRack: serving, monitoring, and user management, validated and supported, ready to run.
FAQs
What is SambaRack?
SambaRack is an AI inference rack powered by the SN50 RDU, and it is the hardware foundation of the SambaNova Platform. Compute, memory, networking, storage, and power are integrated into one air-cooled system that fits an existing data center footprint without liquid cooling.
How is SambaRack different from a GPU rack?
SambaRack is powered by RDUs. The RDU's dataflow architecture chains operations into a single kernel and streams data through, which removes the memory round-trips a GPU makes between operations. Its three-tier memory holds model weights, KV cache, and prompt caches at the right distance from compute. SambaRack handles both prefill and decode, and in a GPU cluster deployment it serves as the decode node.
Do I have to replace my GPUs?
No. SambaRack adds decode capacity to a GPU fleet without changing it. In a disaggregated deployment, the GPUs continue to handle prefill.
Does SambaRack work in disaggregated infrastructure deployments?
Yes. In a disaggregated deployment, GPU nodes handle prefill, RDU nodes handle decode, and storage nodes hold the KV cache between them. vLLM and NVIDIA Dynamo both support splitting prefill from decode.
What inference software runs on SambaRack?
SambaRack runs the open-source inference serving software and engines most developers use, including vLLM and NVIDIA Dynamo. Teams that would rather not build and maintain their own stack can license SambaStack, SambaNova's validated inference serving software.
Can I run my own models on SambaRack?
Yes. Developers bring up models with RDX, SambaNova's toolkit for compiling and tuning models to run on RDUs. Because the RDU is reconfigurable, support for new model families arrives as software rather than new hardware.
What does SambaRack need from my data center?
Standard air-cooled rows at roughly 20 kW per rack. No liquid cooling, no facility retrofit, and it can sit in the same rows as the GPU racks you already run.
How many racks are in a SambaRack deployment?
Deployments come in Scalable Units. The minimum configuration is 16 RDU racks, one spare rack for failover, and four scale-up networking racks. You can increase capacity by adding more units or scaling out.
How does SambaRack scale?
Every Scalable Unit includes four scale-up networking racks at 800G, which connect RDUs so a model can run across many of them. Scaling out adds two scale-out networking racks at 400G with RoCE on top of the Scalable Unit, connecting independent nodes to serve more requests in parallel. A GPU cluster deployment requires both.
What is the Core Services Rack?
It serves as the front-end network management layer for an RDU cluster. Most customers already run their own, which is why the Core Services Rack is optional. SambaNova can supply it for customers who do not.
Can SambaRack run air-gapped?
Yes. SambaRack deploys into isolated facilities where data cannot leave the building, with no data collection and no prompt logging. It is in production at U.S. national laboratories and at sovereign providers in the UK, the EU, and Australia.
Is SambaRack energy efficient?
Designed for maximum token generation per megawatt, its dataflow architecture reduces memory round-trips to save energy.
