Keeping track of the AI market has become a fascinating exercise. The demand and interest in AI services is growing almost faster than it is possible to keep track of.
After years of slow, steady technology growth, both in hardware and software, it seems there are now almost daily announcements of new hardware and software, with what almost seems to be an exponential growth rate in performance and capabilities.
There was a time in the not-so-distant past whereby keeping track of what the hyperscalers were doing in any particular cloud market segment would give you an excellent idea of what was going on in any given space.
But AI seems to have broken that pattern.
Surfing the AI Tsunami
It’s not even that the interest in AI came as a complete surprise to the hyperscalers, though the rate of growth and demand most likely did -- but the rate of change in capability, available performance, and application suitability seems to have shocked everyone.
The software market alone for AI is predicted to reach over $125 billion by 2025 with a 38% CAGR, and it is likely that the vast majority of that software will be deployed in an as-a-Service model.
For businesses projecting their own investment in AI and AI-related services, this is going to be quite an adventure.
There’s always a certain amount of guesswork in any business projection. Now add predicting where and how AI is going to impact your business over the next few years, and you add another entire layer of complexity - and one without a huge amount of historical data to draw from.
Business adoption of AI has really highlighted some fundamental business issues, from fear of job displacement, to a basic AI skills gap and lack of knowledge of how to integrate AI and business processes.
There are also some fundamental problems, like AI bias, that are proving hard to address in a practical and short-term manner. Businesses also need to avoid the trap called “techno-solutionism,” which would give AI the appearance of being the solution and not just the tool it is, that can be used to address business issues.
Many of the AI solutions we’ve previously written about take very different approaches to how AI should solve problems and their underlying technology solutions. While the software side of the problem solving can be very different, there is little change to the issues that impact data centers. The bottom line is: supporting massive computational challenges requires equivalent resources of power and cooling.
How Data Centers Will Buoy AI Demand
For the data center industry - the people providing the infrastructure that AI projects will rest on - the necessary changes and adaptations are, on one hand, very straightforward.
Data centers need to be able to support AI hardware at scale, and this one demand will change the way data centers are designed from the ground up. How is the demand being addressed?
We’ve written quite a few stories about the increased demands on power and cooling that data centers will need to address. Adding AI to the mix has exacerbated those demands and, in many ways, made them more immediate.
The industry needs to be able to support the power and cooling requirements of AI hardware -- and to do it now, the solutions have to fit in the existing footprint of current data centers.
This has led to a number of colocation providers announcing that they have made changes to support higher power densities in existing facilities, have added rack and row level cooling capabilities, and have started to look at solutions that will enable the deployments of racks drawing hundreds of kW of power.
But being able to handle the deployment of power-hungry AI infrastructures is just the beginning.
For example, Google is combining purpose-built hardware with an optimized software stack to improve the efficiency of their data centers in meeting the demands of compute-intensive AI workloads.
Meanwhile, Microsoft not only announced their first dedicated AI chips and server CPUs, they also announced new rack designs with accompanying cooling systems to make these hardware infrastructures more practical for them to deploy.
For its part, a white paper released by Schneider Electric earlier this year addressed many of the issues that the industry will face in integrating AI support into the way data centers are designed, offering specific guidance addressing everything from power distribution to next-generation rack design suggestions, as well as pointing out likely problem areas, as the industry adopts and adapts to these forthcoming changes.
Sustainability Headwinds
While it’s clear that the industry has a path towards supporting AI to its fullest, the demand for AI has had an unfortunate convergence with a serious demand for sustainability in the data center industry.
It’s great to be able to support the potential future of AI hardware, but doing so while finding ways to more efficiently use power and cooling is not a simple task. Add to that sustainability issues such as decarbonization and meeting Scope 3 emissions, and the problems faced in addressing this technological future grow exponentially.
Starting with the basic problem - power - the industry is faced with two issues: contracting for sufficient power, and getting the power where it needs to be.
It’s not news to anyone that the power transmission infrastructure in the United States is due for a major overhaul, which doesn’t limit the demand for power.
But there is no quick fix here, and the solution for power demand, at least in the near term, may simply be the voluntary throttling of workloads by data centers in times of any sort of power availability crisis.
To wit, in October 2023, Google announced they were plotting their demand response power grid support utilizing their carbon-intelligent computing platform to move compute and power-hungry tasks to locations where carbon-free energy was available.
Plotting a Course for AI Power Demands
In the future, technologies like this will likely also drill deeper into the power demands of data centers, defining the necessity of tasks within a matrix focused on economizing the use of power in the data center as power demand increases.
Using AI as our leading example, consider that more than 80 percent of AI tasks are focused on inference, while only 15-20 percent is generally focused on training.
Customers don’t want their inference engines shut down (after all, you need to get your related product recommendations when you're shopping online) but slowing that part of the computing down by a few percent is likely not to be noticed, while saving power.
And if you need to simply stop running your current training model while power is in peak demand, the impact on the businesses is likely to be minimal. So intelligently throttling AI tasks may be low hanging fruit in the future of AI in the data center.
In Conclusion
It is fair to say that AI is having an impact on the way the data center business looks at itself, from the basics of large-scale design and implementation of data center facilities, to operation and daily management and automation.
The common goal of all of the ways AI is making the industry rethink itself is in delivering a more efficient, sustainable, and business and community-friendly existence.
In the following recent episode of the Data Center Frontier Show podcast, Sean Farney, Vice President for Data Center Strategy with JLL Americas, talks about the myriad ways AI will impact future data centers.
