Because prefill and decode are different problems. GPUs excel at compute-bound prefill; decode is memory-bound and latency-sensitive. Adding GPUs to reduce decode latency raises power draw, networking complexity, and cost-to-serve. RDUs add purpose-built decode capacity, so a neocloud can serve a premium speed tier without over-provisioning the whole fleet.

No. SambaStack is available to orchestrate SambaRack systems alongside your existing GPUs and within your existing inference platform, serving everything behind a single OpenAI-compatible API endpoint. Disaggregated inference routes compute-heavy prefill to GPUs and latency-sensitive decode to RDUs, which improves utilization across the fleet rather than replacing it.

In nearly all cases, yes. SambaNova’s Dataflow Architecture minimizes memory movement on the RDU chip, which substantially lowers power draw per token. SambaRack systems are designed to operate within standard air-cooled power envelopes, so a power-constrained facility can add trillion-parameter-class inference capacity without a liquid-cooling retrofit or a new build.

 SambaRack SN50 scales to 256 networked accelerators, supporting models up to 10 trillion parameters and context lengths up to 10 million tokens. That covers the frontier open models commanding premium pricing today, with headroom for the next generation.

SambaNova sells the infrastructure that lets neoclouds serve premium inference themselves. Our business is enabling providers, not competing with them for their customers’ token spend.