AI Driving Data Centers to the Edge

Sept. 20, 2024
Joe Reele, VP Solutions Architects at Schneider Electric, explains how the demand for AI and the evolution of the data center are shaping the digital landscape.

In today’s rapidly advancing technology landscape, data centers are the cornerstone of our digitally-interconnected world. At the same time, the quick growth and application of artificial intelligence (AI) and Machine Learning (ML) is profoundly shaping the design and operation of data centers. Predictive algorithms are impacting industry sectors ranging from healthcare and finance to manufacturing, transportation, and entertainment. Once in production, AI models will be placed at the edge to reduce latency. 

AI workloads are estimated to represent 15% to 20% of total data center energy by 2028, based on internal Schneider Electric growth projections. These AI training workloads are expected to consistently operate at very high densities, which causes challenges for power, cooling, racks, and software management design in data centers. 

The distinction between training and inference is critical when designing AI systems. Training workloads are used to train AI models like large language models (LLMs). These models require massive amounts of data to be fed to specialized IT Clusters (concentrated high-performance compute) and network infrastructure – allowing them to operate in unison. Inference models could be deployed on edge devices or cloud servers, depending on the application's needs, and take the previously trained AI model into production to predict the output of new queries (inputs). The demand for high-performance computing has led to increased requirements for powerful servers with GPUs (Graphics Processing Units) that increase the IT Rack density up towards 100+ kw/rack.  This has changed the technology and architecture of power, cooling and network infrastructure in order to support this higher density requirement.  

Increased AI and ML usage is driving record demand for data centers, but power constraints and old infrastructure grids are inhibiting growth as the data processing requirements are heavily reliant on power-intensive technologies. The data center industry will need to carefully plan and manage environmental and regulatory concerns as companies continue to invest in renewable and energy-efficient fuel sources to mitigate power constraints to support market growth. 

The energy consumption of a generative AI query can vary greatly depending on the size of the inference model, the specific hardware it is running on, and where it is located.  

The Rapid Growth in Data Traffic 

It’s no secret that digital demand will continue to grow. We aren’t going back to beepers and pay phones. From the increasing demand for digital services, cloud computing, and data storage, (and don’t forget the advancements in smart end devices), all of this contributes to the rapid growth in data traffic, especially with AI. This, in turn, drives the need for more data centers, network, aggregation points and investments in digital infrastructure. This also means a greater demand for electricity to power this growth. We can see the future will be digital and with AI-driven automation in many verticals and use cases. Facility owners and operators from data centers, manufacturing, buildings will use this technology to automate routine tasks, reduce manual workloads and enhance overall efficiency in operations, risk, and energy.   

AI and Data Center Evolution 

The demand for AI and the evolution of the data center are interconnected elements shaping the digital landscape. Increased workloads, especially high-performance compute models, require significant computing resources to execute. This requires data centers that can support the performance requirements of AI workloads. As AI technology advances, it will continue to influence the design and operation of data centers. While these advancements bring efficiency and innovation, they also pose challenges related to energy consumption, monitoring, and control, and will require advancements in power and cooling systems. More sustainable approaches to cooling (moving to liquid cooling) and heat capture systems, as well as renewable and green sources of energy becoming available at scale, will help balance this demand vs capacity. 

This relentless advancement of AI is only going to continue, and to meet these evolving needs, the data center industry needs to adapt. Schneider Electric offers guidance on best practices for embracing scalable and flexible infrastructure design to support intensive AI workloads. Strategies go from grid to plug with energy management and controls systems (back of meter power plants & micro grids); deploying high-efficiency and high-capacity power systems; to chiller to chip with mechanical cooling/liquid cooling systems and building automation systems, as well as full suite of sustainability services and global software platforms. 

 

About the Author

Joe Reele

Joe Reele is Vice President, Solution Architects at Schneider Electric, responsible for bringing together the full suite of the company's products and services to provide complete solutions for its customers. Joe has more than 23 years of experience in strategic planning, business development, operations management, and system engineering strategies.

To learn more about Schneider Electric's data center solutions and networks, visit se.com.

Sponsored Recommendations

NECA Manual of Labor Rates Chart

See how Champion Fiberglass compares to PVC, GRC and PVC-coated steel in installation.

Electrical Conduit Cost Savings: A Must-Have Guide for Engineers & Contractors

To help identify cost savings that don’t cut corners on quality, Champion Fiberglass developed a free resource for engineers and contractors.

Conduit Sweeps and Elbows for Data Centers and Utilities

Data Centers and Utilities projects require a large number of electrical conduit sweeps and elbows. Learn why Champion Fiberglass is the best supplier for these projects.

Prefabricated Conduit Duct Banks Enable Smooth and Safe Electrical Installation for a Data Center

Prefabricated conduit duct banks encourage a smooth, safe electrical conduit installation for a data center.

Image courtesy of Submer
Image courtesy of Submer

The Future of Data Center Cooling: Addressing Jitter and Thermal Inconsistencies

Ryan Howard, Solution Engineer at Submer, explores the impact of jitter and thermal inconsistencies on various sectors and how innovative cooling solutions can improve efficiency...

White Papers

Dcf Server Tech Wp Cover 2023 01 19 9 00 45

Smart Cities Run on Smart Power

Jan. 23, 2023
A growing number of cities are getting “smart” and taking advantage of sensor-generated data to make day-to-day living comfortable, convenient, and efficient. Sensors can be used...