Meta Rethinks its Data Center Design for an AI-Powered Future

Dec. 15, 2022
Meta is revamping its data center design to optimize for artificial intelligence, and is pausing construction to "rescope" a number of its projects. We look at this strategic shift, and its potential connection to Meta's planned shift to liquid cooling.

The data center of the future will be different than today's facilities. That's why Meta has decided to overhaul its data center design to optimize its facilities for artificial intelligence.

Meta is still working on what this next-generation computing facility will look like. For the moment, it is pausing construction on a number of its data center projects so it can "rescope" its new data centers with the pending design focused on AI, which is the lynchpin of Meta founder Mark Zuckerberg's vision for an immersive metaverse community.

The shift at Meta was first reported by DCD, which noted a construction halt to an expansion at the company's data center campus in Odense, Denmark and learned that the move is part of a larger rethinking of the company's infrastructure. 

Industry sources confirm that Meta is making a strategic shift in its data center designs, which will focus on optimizing new facilities to better support artificial intelligence. The expansion in Odense - which would have added a fourth and fifth building on the campus - is the only construction project being halted completely.

But the shift will impact the pace of construction at some campuses around the world. Meta typically builds between five and nine data center buildings on each of its huge campuses. Data centers that are under construction will be completed, but work on future phases may be paused so they can conform to the new design standard, sources say. 

Why Meta's Strategy Shift Matters

Meta's decision is significant on a number of fronts, extending well beyond its own infrastructure. Meta operates 18 data center campuses around the globe to support its Facebook, Instagram and Messenger services. These data centers represent an investment of $16 billion and more than that span 40 million square feet of space.

Some possible implications:

  • The new Meta design could provide a boost to liquid cooling. Although Meta is not revealing details of its new design, it has recently outlined plans for a gradual shift to a liquid-cooled infrastructure. Accelerating these plans would provide a boost to the evolving commercial ecosystem around liquid cooling.
  • Wall Street may welcome a slowdown in Meta's construction timeline, as securities analysts have been wary of the company's infrastructure spending to support its metaverse vision. Shares of Meta plunged 25% in a single day in October after the company reaffirmed its plans to invest heavily in metaverse infrastructure.
  • Meta is one of the largest employers of data center construction teams, with multi-phase campus buildouts that keep construction teams busy for years. Any pause in its construction projects would be closely watched by a data center development community that's struggling to find enough workers to meet delivery timelines.
  • New designs from Meta could reinforce the relevance of the open hardware movement, as the company shares most of its innovations in hardware and data centers with the Open Compute Project and its growing ecosystem.

How AI Might Shift Meta's Design

Here at DCF the data center has always been our prism for analyzing changes in technology and business. While many headlines are focusing on the construction halt in Odense, we think the upcoming new design merits attention, and likely reflects an ongoing shift in which data centers are taking on more characteristics of high-performance computing (HPC). 

In artificial intelligence (AI), computers are assembled into neural networks that emulate the learning process of the human brain to solve new challenges. It’s a process that requires lots of computing horsepower, which is why the leading players in the field have moved beyond traditional CPU-driven servers.

A key question: What elements of an AI-focused data center are prompting Meta to rescope its construction? One possibility is the need to incorporate systems to deliver water to the rack. The current design was developed by Facebook in 2011, and features racks sitting on a slab floor, with cool air being dropped into the data hall from a second story "cooling penthouse."

In October Meta announced a roadmap for a gradual shift to a water-cooled AI infrastructure, using cold plates to provide direct-to-chip cooling for AI workloads on its GPU servers. This would initially be accomplished using a rear-door heat exchanger and air-assisted liquid cooling, followed by a shift to a “facility water” strategy as thermal loads exceed the limits of the rear-door heat exchanger. That next phase will likely require the addition of piping to bring chilled water to the rack. 

Here's what we wrote about this strategy at the time:

Meta’s strategy allows it to add higher-density workloads within its current data centers, while working out the details of a next-generation design to transition to facility water supplies and the additional infrastructure that will require. 

Earlier this year Meta revealed a new facility to house its Research SuperCluster (RSC), which may become the fastest AI system in the world when it is completed. Much of the GPU-powered infrastructure in that system is air-cooled, but the facility’s InfiniBand network uses a liquid-to-liquid cooling distribution unit. 

Photos of the RSC show that racks are deployed on a raised floor, a departure from the traditional Meta slab design. Many data center shells can be adapted to use air cooling in either a slab or raised floor approach, but adding water could involve under-floor piping and connections to exterior water infrastructure. 

The RSC design used largely off-the-shelf components, as noted by The Next Platform, leaving open the possibility that a next iteration will feature the kind of custom hardware and design common to Meta and OCP. 

In brief public comments thus far, Meta has not shared any timetable on its rescoped design and what it means for its construction schedule. We will keep our readers updated as more details become available.

About the Author

Rich Miller

I write about the places where the Internet lives, telling the story of data centers and the people who build them. I founded Data Center Knowledge, the data center industry's leading news site. Now I'm exploring the future of cloud computing at Data Center Frontier.

Sponsored Recommendations

Get Utility Project Solutions

Lightweight, durable fiberglass conduit provides engineering benefits, performance and drives savings for successful utility project outcomes.

Guide to Environmental Sustainability Metrics for Data Centers

Unlock the power of Environmental, Social, and Governance (ESG) reporting in the data center industry with our comprehensive guide, proposing 28 key metrics across five categories...

The AI Disruption: Challenges and Guidance for Data Center Design

From large training clusters to small edge inference servers, AI is becoming a larger percentage of data center workloads. Learn more.

A better approach to boost data center capacity – Supply capacity agreements

Explore a transformative approach to data center capacity planning with insights on supply capacity agreements, addressing the impact of COVID-19, the AI race, and the evolving...

JDzacovsky/Shutterstock.com

Coolant Distribution Units: The Heart of a Liquid Cooling System

nVent's Abhishek Gupta explains why CDUs are at the core of driving the efficiencies that liquid cooling can bring to data centers, so choosing the right one is critical.

White Papers

Dcf Vertiv Wp Cover 2022 08 15 13 47 38

Vertiv’s Approach to Environmental, Social and Governance Matters

Aug. 16, 2022
Vertiv presents their approach to environmental, social, and corporate governance (ESG) including what they’re doing for the planet, for their people, and for their neighbors....