Power Management in AI Data Centers

­AI’s capabilities are revolutionizing industries and contributing to a dramatic surge in data center power consumption. Fueled by the rapid growth in AI workloads and other GPU and CPU power-intensive applications, data centers’ electricity consumption is forecast to double from 2022 to 2026, reaching more than 1,000 TWh per year.

This increase in power usage puts pressure on data center operators to reduce electricity consumption, both from a cost and environmental perspective. This poses a tough challenge as the power consumption of high-density server racks rises.

While liquid-cooled racks already support densities over 100 kW, Data Center Frontier highlights industry projections that future rack densities could surpass 500 kW per rack.

How can electricity usage be controlled in power systems? The key factor is efficiency. By reducing losses, an improvement in efficiency means less power is wasted.

Reducing losses also reduces the heat generated in power systems, meaning that data center cooling systems can be smaller and use less power.

Power systems must be as compact as possible to fit into a data center's available spaces. With smaller power supplies, data centers can move to different power architectures, such as an intermediate bus converter (IBC) that uses multiple power conversion stages at both rack and board levels.

The IBC Architecture

The core of a modern processor chip may run on a low voltage of only around 400mV to 900mV. Instead of generating this from the incoming AC supply in one step, which would require bulky and costly wiring, this is often done in stages using an intermediate bus.

First, the incoming supply is typically converted to 48VDC or 54VDC for distribution around the server rack. This improves electrical safety and keeps currents relatively low, allowing cable sizes to be smaller than would be needed to handle high currents while minimizing ohmic (I²R) losses.

Within each server blade in the rack, an IBC transforms the 48V or 54V input to an intermediate bus voltage, typically 12V. This voltage passes through voltage regulator modules to achieve the low voltages at very high currents required for multiple boards and chips.

A Range of Embedded Solutions

Flex offers many embedded power solutions to address data center requirements. These include AC/DC power supplies and shelves, DC/DC converters, and energy storage systems (see diagram). These products improve power supply efficiency and reduce space usage while helping customers simplify design and reduce time to market.

Click image to enlarge

Figure 1 - Flex’s range of embedded power solutions

The range includes a comprehensive portfolio of board-mounted DC/DC modules for data center applications. These modules are provided in compact form factors to save space and typically run on a narrow 40V to 60V supply voltage. They can meet power requirements up to 3kW using individual modules, but higher power can also be achieved through paralleling multiple modules.

Exceptionally high-power density levels can be achieved by using inductor-inductor-capacitor (LLC) and switched capacitance converter (SCC) topologies and cutting-edge components. LLC resonant converters operate with zero-voltage switching and zero-current switching, reducing switching losses and electromagnetic interference while improving thermal performance, making them ideal for high-efficiency applications. In contrast, SCCs utilize charge redistribution among capacitors to achieve high step-up or step-down voltage conversion ratios with smaller inductor sizes, thereby improving power density. Integrating these topologies with components such as wide-bandgap semiconductors and high-frequency switching controllers enables power designers to further optimize efficiency, reduce the footprint, and enhance the overall power-handling capability of modern electronic systems.

Another area of innovation is vertical power delivery (VPD), which places the voltage regulator modules directly under the processors on the bottom of a printed circuit board (PCB). By minimizing the distance that power travels across the PCB, VPD significantly reduces energy losses and enhances overall efficiency. The proximity of regulators with VPD reduces power plane resistance and increases current density, thus enabling a more reliable, efficient power supply to deliver high current, low voltage, and extremely fast load transient response. In contrast, traditional lateral power delivery methods often lead to considerable power dissipation across the PCB, increasing energy costs and complicating thermal management.

In addition to converters and modules, Flex offers a range of other custom power products, such as power shelves, which can be customized to fit our customers’ specific requirements. This includes the GB200, which consists of six power supply units with a maximum total output power of 33kW. The solution can provide DC power to all the payloads inside a rack, including the most demanding GPUs.

Designed to integrate with server rack power systems, the company’s Capacitive Energy Storage System (CESS) helps address significant utility power challenges in AI data centers. The CESS enhances energy efficiency and ensures more stable power management by balancing peaks in data center utility power during large transients or surges. 

The Broader Picture

In addition to embedded power management, data center providers are looking for suppliers to provide solutions for all parts of the power supply. This includes switchgear, busway, power distribution, advanced liquid cooling systems, and more.

Flex is the only manufacturer of data center infrastructure with a power portfolio that extends from grid to chip. In fact, with the acquisitions of critical power providers, including Anord Mardix and Crown Technical Systems, Flex meets more than 80 percent of data center needs with an extensive product portfolio supported by advanced manufacturing and lifecycle services.

Power distribution in data centers can present technical challenges that call for engineered critical power solutions to prevent downtime and manage efficiency. The risks of power failure, load imbalances, and inefficient distribution can result in significant consequences, from service outages to overheating and equipment failure.

Scalability is another significant challenge. Traditional power infrastructure can struggle to keep up with data center expansion, which often requires costly and time-consuming modifications. Prefabricated modular systems such as those developed by Anord Mardix can streamline this process. Essentially, these units integrate components, such as transformers, switchgear, and uninterruptible power supply devices, into a compact, pretested assembly. The benefits of these systems are that they can be deployed quickly, reducing installation time and minimizing disruptions, especially for fast-track data center builds or locations with space constraints.  

Flexibility in power distribution is essential. Many facilities still depend on cable-based systems, which can result in tangled wiring, voltage inconsistencies, and limited adaptability. Busway systems offer a more dynamic alternative, enabling power tap-offs to be added or reconfigured without shutting down the system. This solution is particularly useful in hyperscale environments where IT loads frequently shift.

In summary, as data centers evolve to accommodate increasingly complex workloads and higher computational demands, the need for scalable, adaptable, and energy-efficient power solutions is more pressing than ever. With a full portfolio of vertically integrated solutions that optimize power distribution from grid to chip, Flex enables companies to address compute and power challenges at scale and support the rapid expansion of AI data centers.

Liquid Cooling 

With more power delivered per rack comes more heat, which will continue to scale in parallel with compute infrastructure expansion and advances, requiring innovative approaches to cooling technology beyond traditional air cooling. Direct-to-chip cooling transfers heat from processors to liquid-cooled cold plates. This approach in data centers can significantly outperform forced air cooling in heat transport/removal and make board design more flexible, because processors and heatsinks do not have to be placed near an air exit. JetCool uses microjet technology to precisely target hot spots and improve cooling efficiency at the chip level.

Flex Power Modules