The Server Rack Just Got 15 Times Hotter, and Heat Exchangers Have to Catch Up

A single AI server rack built on NVIDIA's newest platform can draw up to 246 kW. That's more than fifteen times what a mainstream rack pulled a couple of years ago. Average rack density sat around 16 kW in 2025 and is running near 27 kW this year, with the top of the market well past that. This is the number reshaping every conversation about how heat leaves a data hall.

Heat exchangers used to be a background component in facility design. In 2026 they've become the constraint the whole building is designed around. No single design wins, either. Different workloads, sites, and climates each pull the answer in a different direction.

Rear-Door Exchangers Buy Time for Existing Halls

Operators with rooms full of legacy air-cooled infrastructure can't rip everything out to chase the next GPU generation. Rear-door heat exchangers are the pragmatic bridge. A chilled-water coil bolts to the back of the cabinet and captures the hot exhaust before it ever reaches the room, so a hall built for 10 kW racks can host 40 or 50 kW racks without a full mechanical redesign.

The design pressure here is coil performance at low approach temperatures. There isn't much room, the fan power is limited, and the water has to do most of the work. Fin geometry, circuiting, and surface area per cubic inch decide whether the door keeps up or the rack throttles. That's why microchannel construction has moved into serious consideration for this niche, because the density of surface area is what makes the retrofit math work.

Direct-to-Chip Loops Change What the Coil Even Looks Like

Once rack power crosses roughly 50 kW, air alone stops being credible and coolant meets the silicon directly. Cold plates sit on the GPUs, a facility water loop takes the heat out to a coolant distribution unit, and a secondary loop rejects it outside. The building's heat exchangers no longer condition air. They move BTUs between two liquid loops at very tight temperature differences.

That shifts the design problem in three ways worth naming:

• Approach temperatures shrink. Chips want warm-but-stable coolant, so the exchanger has to move a lot of heat with only a few degrees of difference between loops. Surface area and flow distribution matter more than raw size.
• Fouling tolerance drops. Narrow passages boost performance and punish poor water chemistry. Filtration, treatment, and material selection become part of the exchanger spec, not an afterthought.
• Redundancy moves into the coil. A failed liquid-to-liquid exchanger doesn't warm a room. It strands a rack running a training job worth millions. Parallel paths and isolation valves are now baseline.

Warm-Water Rejection Kills the Chiller (Sometimes)

One of the more counterintuitive moves in AI cooling is running the loop hotter, not colder. NVIDIA's Vera Rubin NVL72 platform is designed for warm-water cooling at a 45°C supply temperature, which is warm enough that ambient air can carry the heat away through dry coolers most of the year without a mechanical chiller in the loop at all.

For heat exchanger design, this rewrites the assumptions. Dry coolers and fluid coolers stop being shoulder-season equipment and become the primary rejection path. Coil face area, fan selection, and coating durability under year-round outdoor duty all get more scrutiny. Designers who used to size for a worst-case wet-bulb day are now sizing for a worst-case dry-bulb day plus a much larger annual runtime.

Water-Scarce Sites Push Design Toward Closed Loops

Evaporative cooling is efficient, but AI-scale water use is drawing real scrutiny in siting decisions and permitting. When a facility can consume meaningful volumes of water per kWh of compute, hyperscalers are increasingly willing to trade a little thermodynamic efficiency for a closed-loop design that doesn't touch municipal supply.

The design tradeoff lands squarely on the heat exchanger. A closed-loop dry system needs substantially more air-side surface area to reject the same load an evaporative tower would handle, which favors high-density coil designs and, increasingly, compact microchannel heat exchangers that pack more capacity into the same footprint. The plate count goes up, the fin spacing tightens, and corrosion protection becomes non-negotiable because these coils live outdoors and run constantly.

Where This Leaves Designers

There isn't going to be one heat exchanger for the AI era. Retrofits will keep leaning on rear-door coils. New hyperscale builds will lean on liquid-to-liquid exchangers matched to warm-water direct-to-chip loops.

Water-constrained sites will spend the extra capital on dense dry coolers. The common thread is that the coil is no longer a commodity. Its geometry, materials, and manufacturing tolerance decide whether the racks behind it can run at full power.

For anyone specifying equipment right now, the useful shift in mindset is putting the heat exchanger inside the compute stack, not the building. The chip roadmap tells you what next year's thermal load looks like. The exchanger has to be sized for that, not for the rack sitting in the hall today.