I had the privilege of leading Vertiv’s latest Engineer’s Frontier webcast with Sandro Gatto, Vertiv Business Development Manager for chillers and speaking about the transition from Air cooled to Liquid cooled Data centers. With increasing densities on IT hardware and more concern over sustainability, we expect an increasing interest in the deployment of liquid cooling technology.
IT Power Trends
Artificial Intelligence (AI), automation, HPC (High Performance Computing), and Machine Learning increasing demand on processing resulting in higher heat densities per chip. This contributes to servers generating more heat that must be removed, and in some applications the heat generated attains a level at which air cooling is not enough to permit sufficient cooling to these high-density racks.
There are multiple options in terms of thermal management technology to consider depending on design requirements within a specific range of rack densities. As rack densities approach and exceed 30 kilowatts (kW), maintaining the processor working temperature (case temperature), may not be adequately cooled with air cooling, with the result that liquid cooling is the only option for removing heat efficiently.
This is due to the thermal heat carrying properties of liquids, which are much more effective for the heat transfer from the chip which can be done via Direct to Chip (D2C) or Immersion Cooling methods.
Efficiency
Liquid cooling is a more efficient way of cooling the data center for two reasons:
-
The supply temperature of the working fluid can be elevated to maintain processor operating conditions, without increasing fan power as the cooling is directed at the hottest components, typically the processor. We acknowledge that the heat transfer between the processor and the cold plate is much more efficient than the heat transfer between the processor and the surrounding air in case of a traditional air cooling system, thus benefitting the overall efficiency of the datacenter.
-
The return water temperature from the racks, being elevated compared to traditional air cooling, could enable the heat recovery in a much efficient way than in an air cooled data center.
With the constant increase of applications requiring increased computing power, there will be a move to innovative thermal management technologies to better manage these higher power densities. We expect many data centers will introduce hybrid thermal management systems, including both air and liquid cooling.
Liquid Cooling Technologies
The transition to support a Direct-to-Chip liquid cooling system is close to what we are familiar with a standard rack, with the difference being that the rack design will include a liquid manifold to distribute the working liquid to the Direct-to-Chip enabled IT equipment (ITE) the plumbing inside the rack will be matched to liquid distribution manifolds at the row to deliver the liquid to the rack and all controlled by a Cooling Distribution Unit (CDU). Provisioning for this requires changes to the facility technical space which requires appropriate planning.
An immersion cooling system requires a datacenter layout that is different, the standard rack layout is replaced by tanks which are horizontal compared to vertical traditional racks. In this form of Data Center, the ITE is completely immersed in a thermally conductive dielectric liquid or fluid (a fluid that is not electrically conductive). Immersion cooling is a promising technology, but not completely mature yet as there is still a lack of comprehensive standards. There is a need to up-skill datacenter operations teams, as they are typically unfamiliar with the technology and require transferring knowledge prior to installation and operation.
The common approach for both technologies is to utilize a CDU with a liquid-to-liquid plate heat exchanger between the TCL (Technology Cooling Loop) and the FCL (Facility Cooling Loop), to transfer the heat between the TCL and the FCL, the TCL has stringent requirements in terms of cleanliness and filtration requirements are much higher than the ones required for the FCL.
Both technologies can use a single-phase fluid or phase change fluid, known as two-phase fluids, which are more efficient as the heat transfer is benefitted as the liquid vaporizers to a gaseous state.
Metrics
Hybrid data center design is expected to be popular, therefore metrics become important to measure the efficiency of the entire datacenter. We need to consider metrics that go beyond the traditional PUE (Power Usage Effectiveness) metric, in order to take into account, the expected reduction in fan power via liquid cooling systems.
Together with our partners, Vertiv developed a study showing different metrics such as TUE (Total Usage Effectiveness), that reflects the total power reduction for the whole design of the data center.
Read more about the TUE metric in our blog: Quantifying the Impact on PUE and Energy Consumption When Introducing Liquid Cooling Into an Air-cooled Data Center.
Also, learn more about the Vertiv™ Liebert® XDU Coolant Distribution Units, which are suitable for chip and rear door cooling applications and are easy and cost-effective to deploy in any data center.
Finally, don’t forget to listen to the entire Engineers Frontier Consultant’s Series for insight on the latest technologies from the experts at Vertiv.
