Solving the AI Datacenter Power Crisis Without New Construction

Delivering on the promise of AI is no longer constrained by the technology that enables it, but by the ability of modern datacenters to power the systems it runs on. The result is a growing gap between what the most advanced AI systems demand and what today's facilities can actually deliver.

TL;DR:

• The AI datacenter power crisis is the growing mismatch between the power next-generation AI systems require and what existing data centers can supply.
• Existing datacenters hold more than 80% of AI capacity but support only 30 kW or less per air-cooled rack, while the latest GPU-based racks draw 120 kW and future generations may reach 1 MW per rack.
• A new 1 GW AI datacenter costs an estimated $47 billion to build and takes roughly three years to stand up.¹ Power shortages are projected to delay or cancel 30% to 50% of AI datacenters planned for 2026.⁵
• Fewer than 8% of enterprise datacenters will meet next-generation power and cooling requirements when NVIDIA Rubin Ultra ships, according to Teradata.
• SambaNova's RDU platform runs air-cooled at about 10 kW per rack (SambaRack™ SN40-16) and 20 kW per rack (SambaRack SN50), so it drops into existing datacenters, on its own or alongside GPUs.

Existing datacenters make up more than 80% of the available capacity for AI systems, yet most cannot run next-generation systems like the NVIDIA VR200 NVL72 because of power limitations. Instead, running these powerful systems requires the construction of entirely new facilities, a costly process that can take years to complete.

Foxconn estimates that a 1GW datacenter built to power NVIDIA VR200 NVL72 systems would cost $47 billion to construct, with an annual electric bill of approximately $1.3 billion.¹ NVIDIA CEO Jensen Huang has confirmed, “If you want to build a datacenter here in the United States from breaking ground to standing up an AI supercomputer is probably about three years”.²

Three forces are driving this crisis: a widening gap between the power available in existing datacenters and what next-generation systems require, growing public and regulatory pushback against datacenter expansion, and the need for sovereign AI in regions with limited resources. The sections below examine each of these challenges and how SambaNova’s 20kW air-cooled RDU platform is uniquely positioned to solve them.

The Power Crisis

“AI infrastructure powered by advanced GPUs challenges the power and cooling capacities of current enterprise data centers. To address this, organizations must adopt design and operational practices to deploy AI infrastructure effectively within existing spaces, ensuring performance, scalability, and energy efficiency.”

Gartner, AI Infrastructure Guide for Power-Constrained Data
27 June 2026 - ID G00850629

Rising Power Demands Per Rack

The latest GPU-based systems are voracious consumers of power and water. Each new generation of GPUs has driven power demands higher. The DGX H100 and H200 systems, each with eight GPUs, only consume 10.2 kW of power, making them easily compatible with most existing datacenters.

However, the 72 GPU Blackwell DGX GB200 NVL72 draws up to 120 kW per rack, a more than 10X increase that puts it out of reach for most existing data centers. Power requirements for the latest Vera Rubin NVL72 racks operate at 120-130 kW, while the Rubin Ultra NVL576 power consumption skyrockets to as much as 600 kW per rack, which is enough to power 400 homes.

Feynman, the system expected to follow Rubin, will have configurations that may require as much as 1 MW per rack. The core problem is that most current datacenters can only support 30 kW or less, per air-cooled rack. These datacenters need systems that can operate within that range.

AI's Grid-Level Power Gap

Gartner estimates that meeting the incremental power needs of AI datacenters in 2027 will be 500 terawatt-hours (TWh) per year. This is a 2.6X increase of the power requirements in 2023 and nearly as much as Germany’s entire power consumption in 2022.

Emerging Tech: Generative AI Power Challenges Cannot Be Solved by Semiconductors Alone, ID G00809650

Datacenter Readiness

Delivering the power required to operate the latest GPU-based systems is a significant challenge for existing datacenters. Most enterprise datacenters were built to support rack-level power requirements of 5 kW-8 kW and typically only offer air cooling – a far cry from the 600 kW some Vera Rubin systems demand.

Power is only part of the problem. Any system consuming that much electricity generates a proportional amount of heat, which is why these systems require sophisticated liquid cooling rarely found in existing datacenters.

Given the significant power and cooling demands, most traditional datacenters are ill-equipped to support the latest GPU-based platforms. Teradata estimates that by the time Vera Rubin systems ship, fewer than 8% of enterprise datacenters will meet the power and cooling requirements needed to run them.

Enterprise readiness for Rubin deployment. AI training clusters approach universal liquid cooling, but overall data center penetration and 800V DC adoption lag far behind. Fewer than 8% of enterprise facilities will have both capabilities when Rubin Ultra ships in H2 2027.

Source:Teradata

The Great Datacenter Build Out

The exploding demand for faster AI services, combined with the need for datacenters that can support next-generation GPU systems, has triggered a massive wave of proposed new datacenter construction. Pew Research counts more than 1,500 new datacenters currently in various stages of development in the U.S. alone.³

Much of the construction is shifting to rural areas. While 87% of existing datacenters are located in urban areas, 67% of those being built are in rural areas, and 39% are in areas with no datacenters today.

Construction alone won’t solve the problem. These new facilities also need access to sufficient power., and many rural sites lack grid connections capable of meeting their needs.

According to Gartner: “Datacenters that demand huge amounts of power can be built far faster than power utilities can expand their capacity. Delivering increased power to datacenter locations often requires new transmission lines from existing generation facilities (which can take years for permits), but can even necessitate building new power plants, which can take decades".⁴

That gap is already stalling projects. One report estimates that the inability to secure power will delay or cancel 30%-50% of AI datacenters planned for deployment in 2026.5 Of the 16 GW of capacity slated to come online in 2026, only 5 GW are actively under construction. With build times of 12-18 months, it is unlikely that the remaining sites will become operational within their target dates. An additional 16 GW has been announced with no signs of progress beyond the “announcement” phase.

AI Datacenters Planned for the U.S.

Water Consumption

Power is not the only resource under strain. Every watt consumed by a GPU generates heat that must be removed, and that cooling demands enormous quantities of water.

Closed-loop systems can cut water consumption by up to 70%, and immersion cooling can also reduce water use. But both add design complexity, introduce the potential for leaks, and raise costs. These are trade-offs operators will need to weigh as they build next-generation facilities.

Sovereign AI

This power crisis carries even higher stakes for sovereign AI deployments. Sovereign AI refers to national AI deployments built to benefit a particular nation. Under these programs data is stored, processed, and managed within the borders of a given country.

While AI can offer a clear national benefit, every nation is different and many are constrained by existing datacenter capacity and the availability of power to operate new facilities.

Aging power grid infrastructure compounds the problem. Capgemini reports that “outdated grids are a top concern for reliable data-center power delivery, cited by 74% of electricity executives globally (79% in US and 75% in Europe).” The same report projects European datacenter power demand will grow from 96 TWh in 2024 to 168 TWh by 2030, with Germany, France, UK, Ireland, and Netherlands accounting for more than 60% of that consumption.

Much of that energy will travel over aging infrastructure. The European Commission estimates 40–55% of European power lines will be more than 40 years old by 2030, and that upgrading the grid will require an investment of more than €1.2 trillion ($1.39 trillion) by 2040.⁹

The SambaNova Solution

The root cause of the datacenter power crisis is the mismatch between the power and cooling demands of next-generation GPU-based systems and what existing datacenters can deliver. There is no doubt that new datacenters with significant power capabilities are coming, but they will take years, and meeting today’s AI demand requires a different approach now.

SambaNova created the Reconfigurable Dataflow Unit (RDU) to solve this exact problem. Purpose-built to run AI inference with ultra-low latency, RDUs move data more efficiently than GPUs, delivering exceptional performance while consuming significantly less power.

SambaNova SN50 RDUs deliver faster inference to more users than comparable generation GPUs.

Air-Cooled Inference at 10 to 29 kW Per Rack

SambaNova SambaRack SN40 systems deliver exceptional performance at an average of just 10 kW per rack. SambaRack SN50 systems deliver 5X the compute power of the SN40, while still averaging only 20 kW per rack.

With a low power footprint and air-cooling, SambaNova systems can be used in existing datacenters. They can be used for high-performance inference on their own or alongside GPU deployments.

Deploying RDUs Alongside Existing GPUs

In disaggregated inference deployments, for example, GPUs handle the prefill phase and RDUs power the decode part of the process. The result is a solution that delivers extreme performance for premium inference, powered by the GPUs already in the datacenter and RDUs that work seamlessly with the infrastructure available today.

The purpose-built design of the SN50 RDU delivers superior throughput and performance compared to equivalent-generation GPUs, all while running air-cooled, making it a true drop-in solution for existing data centers, with no new construction required.