Since the introduction in the early eighties of distributed power architectures within the telecom and datacom industries (recently called Information and Communication Technology), the power demand per board in the data-server segment has increased from 300W to 1200W and is foreseen to grow exponentially reaching 3000W per board by 2015.
In addition to the electrical challenges for power designers to develop highly efficient topologies and products, delivering this level of power from isolated bricks, at 3kW per board, will require systems designers to consider a number of things beyond powering it. Another major concern is how to cool this type of board efficiently to make sure all processors can be operated without overheating.
Some may question why the data-server industry is reaching this level of power per board: why not split the task into three smaller boards? While many have considered this approach, the solution is not that easy, and power per board by some studies will reach up to 5kW each by 2020. Before addressing how best to power kilowatt boards, it is interesting to understand what is behind this level of power and why these boards will continue to require even more power.
Growing Traffic
The amount of data transferred through networks is increasing tremendously, with huge growth in mobile traffic and data centers. The mobile industry, which accounted for 6.6 billion subscribers at the end of Q3 2013, generates 1.9 Exabytes of data per month and is estimated to increase around 10x between 2013 and 2019 reaching 19 Exabytes per month (see Figure 1).
Click image to enlarge
Figure 1 – Global mobile traffic per month expected to grow up to 19 Exabytes per month
In data centers, a recent report estimated that the annual IP traffic (estimated to be 2.6 Zettabytes in 2012) will reach 7.7 Zettabytes by the end of 2017. The report explains that ICT manufacturers need to quickly develop efficient solutions to manage this level of traffic, while also needing to reduce energy consumption.
To respond to this demand, data center operators are considering two approaches:
•Modernization of existing data-farms (upgrade)
•Development of new data centers using the latest technologies that offer higher performance in terms of energy utilization, efficiency and a reduction in the Total Cost of Ownership and also environmental impact (see Figure 2). In term of power per board, the two solutions are very different.
Click image to enlarge
Figure 2 – A new generation of data centers will include new power architectures for reduced power consumption
Modernization
In the case of modernization, most installed data centers have been already through different phases of systems upgrades and migrations to higher computing power and are now reaching some limitations in terms of increased power per board. Chassis and cabling is already installed (see Figure 3), power supplies and power backup is often limited to what is already in place, and refrigeration systems have limited room for extra cooling. This means that boards have to be extremely energy efficient, and that data center operators could have to select a critical part of the system for upgrade, especially when new cards might require 1.2 to 1.5kW maximum, which is almost twice that of today’s average power per board.
Click image to enlarge
Figure 3 – Data center modernization will need to take already-installed chassis and cabling into consideration
Most of the existing boards used in applications are powered by two 450W quarter-brick modules connected in parallel, delivering an average power of 900W, which comes in about 800W when thermal derating is taken into account (see Figure 4). Because board power designers have no room for additional power modules and have to stick to existing footprints, when considering new generations of equipment they often opt for higher power quarter-bricks modules. These are now delivering up to 860W (see Figure 5) in their datacom version (45V to 60V input) or stacked modules delivering the same level of power across a wider input range (see Figure 6) to meet telecom standards (36V to 60V/75V). These solutions are making upgrades possible, but probably do not offer the most efficient power architecture when considering power per board demands in the 3 to 5kW range.
Click image to enlarge
Figure 4 – Today, two quarter-bricks in parallel deliver 900W to IP switches and routers
Click image to enlarge
Figure 5 – Datacom-input-range high-power-density 864W quarter-brick module featuring double power pins and optimized design for cold-wall cooling
Click image to enlarge
Figure 6 – Wide-input-range telecom stacked power modules
Meanwhile, routers have also gained in computing power: as they demand more power per board, the PCBs have become thicker adding heavy copper layers to carry the 100A (1200W / 12V) delivered by board-mounted power sources to the loads. The number of PCB layers has reached 30 in many applications and therefore the soldering of power modules to the boards – and rework in the case of defects – has become a critical point. In addition to this, board power architects have started to question the reasoning behind a single pin per polarity to deliver the power to the intermediate bus.
To address this concern, back in 2003 power supplies makers worked in close cooperation with customers, contract electronics manufacturers and connector manufacturers to develop suitable technology that guaranteed a sustainable solution. The basis of a new power brick concept emerged from ‘The Double-P’ (Thermally Enhanced Double Power Pin) initiative and featured two power pins and a layout to optimize power dissipation through pins and ferrites (see Figure 7), foreshadowing the high-power-density power modules of today that power 1.2kW routers.
Click image to enlarge
Figure 7 – The first ‘The Double-P’ high-power-density module launched in 2003
Ignoring for one moment the challenge to obtain the necessary power from bricks, the migration from 800W to 1.2KW per board represents another challenge for systems architects in terms of cooling. The conventional method in place in data centers is usually based on air circulation, which might not be enough in the case of an almost doubling of power consumption. This is triggering the use of heat-pipes, moving hot spots from processors and other dissipating components to cold plates and heatsinks to improve cooling. In these systems, the board-power module baseplates are also connected to master cooling elements, reducing thermal stresses and improving reliability. All these techniques are acceptable when data center operators consider upgrading existing installations, but it might not be suitable for the multi-kilowatt boards required by the next generation of routers and associated equipment.
New Developments
Today, systems architects are used to dealing with processors and data-speed switching capacity that is approaching hundreds of Terabytes per second; 200-Gbit/s optical networks are not a dream anymore and Internet-switch processors now offer an amazing level of computing power with the ability to manage significantly higher traffic capacity per second than the previous generations of IP switches and routers. But systems architects now have to reconsider routing overall, shortening connections between different packet-processors and increasing the number of I/Os per board to limit interconnection losses which introduce latencies – the nightmare of data center operators.
In response to these issues, systems architects are considering new mechanical platforms based on bigger boards and a chassis designed to accommodate new cooling methods such as refrigerated cold-walls, making it possible to cool systems with more efficiency than that offered by conventional ventilation. Much research is being conducted in this area and we should expect a great deal of innovation here.
In terms of powering boards, while paralleling today’s quarter-bricks could be the smart choice for 1.2kW boards, and even if a 1kW quarter-brick is conceivable for powering 3kW and perhaps up to 5kW, systems and board-power architects are questioning the electrical and thermal benefits to parallel this type of brick. They are increasingly inviting power supplies manufacturers to rethink board-power modules, which haven’t evolved a great deal in terms of packaging since the first high-power-density bricks that were introduced in the late nineties.
In addition to the complexity involved in soldering power-bricks to boards that could have as many as 45 layers, board power architects have to consider how to connect the output of these power bricks to additional power rails that they may have to add to distribute the 250A, and later 415A, to end loads. To complete the picture, if one considers new cooling methods employed in other industries and applications such as RF power amplifiers, which are operated in free air convection most of the time, power architects are likely to demand that board-power manufacturers design a new generation of products for cold-wall conditions. Considering electrical and thermal constraints, new multi-kilowatt boards from board power manufacturers could result in a totally new type of board-power source, taking up more space to facilitate thermal exchange, with interconnections that are better designed for very high currents, and with brand new assembling techniques.
If such modules do not exist at the time of publication, power supply manufacturers in close cooperation with semiconductor and other component manufacturers could develop a the final product that could be similar to the 3D simulation shown in Figure 8. Modules will likely include larger transformers with interconnection to high-power press-fit sockets pre-installed during board preparation.
Click image to enlarge
Figure 8 – Simulation of 1.5kW isolated power modules designed for efficient cooling and easy assembling required to power multi-kilowatt boards
Conclusion
When the ‘The Double-P’ initiative started in 2003, powering 800W boards was challenging enough and nobody could have believed only 10 years later boards would be migrating to 1.2kW and quarter-bricks would be reaching 864W, getting closer to the almost mythical kilowatt power source. Power designers have proven that extreme challenges are highly motivational to explore new way of doing things. And powering a 3kW board, or potentially a 5kW board by 2020, presents a new challenge from which a totally new technology could emerge combining unprecedented levels of innovation.
PDF
