Equinix Begins Running Liquid-Cooled Servers in Production on Metal

Feb. 1, 2023
Equinix has begun using liquid-cooled servers in production environments for its Metal service, taking the next step in a shift to broader use of liquid cooling across its infrastructure.

Digital infrastructure heavyweight Equinix has begun using liquid-cooled servers in production environments for its Metal service, taking the next step in a shift to broader use of liquid cooling across its infrastructure, the company said this week. 

In a blog post, Equinix said it is running a full rack of equipment in its NY5 data center in Secaucus, N.J. using a system from ZutaCore that uses two-phase liquid cooling to remove heat from its servers. 

"It wasn’t just any workload we were dealing with; we were going all in with a full rack of equipment powering our production control plane," said My Truong, Field CTO for Equinix. "That’s right, we’ve taken liquid cooling to production with our customer portal and the Metal API!"

Equinix has been testing the use of liquid cooling in its data centers for several years at its Co-Innovation Facility in Ashburn, Virginia, where it works with partners (including ZutaCore) on next-generation data center technologies. As it moves into production on Metal, Equinix provided a detailed overview of its installation, with extensive technical details and the lessons it has learned. 

The company's goal is to help its customers and the data center industry get comfortable with liquid cooling, which is likely to become an essential technology for managing the growing adoption of artificial intelligence (AI) and other high-density applications.

 "Equinix is at the beginning of a journey we expect most Fortune 2000 companies to make in the next couple of years," Truong writes. "Data center liquid cooling will go from being almost exclusively in the HPC realm to becoming a standard requirement for systems. Colocation operators like Equinix will be changing buildings and operational policies to enable customers to leverage it in pursuit of sustainability."

Preparing for Growth of Liquid Cooling

The leadership from Equinix is notable because the company is the largest provider of colocation and interconnection services, with 10,000 customers and 229 data centers across the globe. 

Interest in liquid cooling has been boosted by the growth of artificial intelligence, which relies upon powerful hardware that packs more computing power into each piece of equipment, boosting the power density – the amount of electricity used by servers and storage in a rack or cabinet – and the accompanying heat. These rising power densities are challenging traditional practices in data center cooling, and prompting data center operators to adapt new strategies to support high-density racks.

But interest in advanced cooling is no longer just about density and “hot hardware,” according to a recent Data Center Frontier roundtable of data center experts, who cite a confluence of factors that includes sustainability and edge deployments. Microsoft is using immersion-cooled servers in production, and believes the technology can also boost the efficiency and performance of its cloud data centers.

The Equinix-ZutaCore Implementation 

The Equinix blog post provides details of its approach, complete with videos, diagrams and photos. The company is using a ZutaCore design that brings dielectric fluid into the server, where it enters an evaporator chamber sitting atop the CPU. The heat produced by the processor boils the liquid - The two-phase fluid turns to vapor at about 92F (33C).- and the resulting vapor is returned to a condenser unit, where the coolant changes back to liquid. 

"If you build a tiny pool for the fluid on top of a processor and keep topping it up as the fluid evaporates, the temperature of the metal plate in the chip’s casing that’s in contact with the fluid (chipmakers call it the “Integrated Heat Spreader”) will generally remain at 92F," said Truong.

The Equinix installation used standard medium-density 19-inch servers, rather than the high-density racks that are the typical candidates for liquid cooling. The rack housed 20 1U servers along with a 6U liquid-to-air heat exchanger. The company said the installation has been stable and operational since getting commissioned in June 2022 with no issues.

About the Author

Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

Courtesy of AFL

Hyperscale Rising

Alan Keizer and Keith Sullivan of AFL explore the growth and evolution of hyperscale computing from being a nice-to-have to a must-have.

White Papers

Dcf Service Express Sr Cover2023 07 07 15 37 53

Top Methods To Modernize and Balance Your Infrastructure

July 10, 2023
The growing number of connected devices, the increased use cases around mobility and a greater need for data center reliability are all driving growth in cloud and data center...