Intel Engineers Multi-Pronged Approach to Gaining Data Center AI Market Share

April 12, 2024
Three years into the renewed stewardship of Intel CEO Pat Gelsinger, new GPUs, accelerators, CPUs, open standards, and silicon offerings now lead the way in a competitive AI infrastructure ramp-up.

It was bad enough for Intel when AMD started to make huge inroads into their server CPU market with their latest generations of EPYC CPUs -- but as the market shifted towards massive investment in AI, Intel was hard-pressed to offer any competition for Nvidia.

But five years after acquiring Israeli start-up Habana Labs, and its Gaudi AI chip family, Intel thinks they have a shot at competing in the AI chip marketplace with the Intel Gaudi 3 AI accelerator.

Inference Matters

While it is true that the AI marketplace is all about Nvidia, then everybody else, with Nvidia’s very aggressive release schedule for advanced AI products it is important to remember that the vast majority of AI-related tasks don’t require the absolute current pinnacle of performance.

While training large language models is the jewel in the AI crown, the majority of enterprise AI use is applying what is learned by those models as inference, most often seen by consumers as recommendation models when visiting web sites.

And the industry is realizing this, as is clearly identified by Cisco’s partnering with Nvidia to deploy capable Tensor core-based systems within their blade servers and Ethernet networks.

Competing in the Market

In its Gaudi announcement, Intel claims that the Gaudi 3, crafted using TSMC’s advanced 5nm technology, delivers 50% better inference and 40% better power efficiency than the previous top-of-the-line Nvidia H100 AI GPU.

What this means is that the large market of people for whom being absolutely cutting-edge is unnecessary will have serious competition.

And while pricing has not been announced, Intel's claim is the Gaudi 3 will be a fraction of the cost of the comparable Nvidia AI accelerators.

Gaudi 3 vs. Nvidia

Intel likes to point out that the previous generation Gaudi 2 is the only benchmarked alternative to the Nvidia H100 for generative AI performance, with results that are linked here.

And the performance claims for Gaudi 3 indicate a significant increase in performance, with 4X AI compute in BF16; a 1.5x increase in memory bandwidth; and a doubling of networking bandwidth driving the performance gains.

Additional details on the Gaudi 3 accelerator can be found here.

Performance comparisons against the shipping Nvidia H100 using common reference models make the following claims:

  • 50% faster time-to-train on average a cross Llama2 models with 7B and 13B parameters, and GPT-3 175B parameter model.
  • Inference throughput is projected to outperform the H100 by 50% on average1 and 40% for inference power-efficiency averaged2 across Llama 7B and 70B parameters, and Falcon 180B parameter models. 

 

CUDA vs. Open Standards and Ethernet Implications

Perhaps more importantly, with the new release Intel is taking on Nvidia’s developer dominance with CUDA by focusing on open standard, community-based software, reducing reliance on CUDA-specific development skills.

It is also making use of Ethernet-standard networking, allowing systems to scale from a single node to mega-clusters with thousands of nodes.

This means supporting inference and training models at a scale that can adjust to customer demands.

Will “Open” Really Make the Difference?

To help drive the concept of open platform AI development, Intel announced that they had joined with Anyscale, Articul8, DataStax, Domino, Hugging Face, KX Systems, MariaDB, MinIO, Qdrant, Red Hat, Redis, SAP, VMware, Yellowbrick and Zilliz, with the intention to create this open AI platform.

To get this effort going, Intel announced that they “will release reference implementations for GenAI pipelines on secure Intel Xeon and Gaudi-based solutions, publish a technical conceptual framework, and continue to add infrastructure capacity in the Intel Tiber Developer Cloud for ecosystem development and validation of RAG and future pipelines.”

The company is also working through the Ultra Ethernet Consortium to develop optimized Ethernet networking for an AI fabric.

Everyone Is Becoming An AI Company

Across many technical disciplines, according to Intel CEO Pat Gelsinger, every player is becoming an AI company.

Be it manufacturing components for AI-enabled infrastructures, actual AI accelerators, or using AI to deliver value to customers, everyone seems to be involved. And Gelsinger points out that “Intel is bringing AI everywhere across the enterprise, from the PC to the data center to the edge."

He continues, "Our latest Gaudi, Xeon and Core Ultra platforms are delivering a cohesive set of flexible solutions tailored to meet the changing needs of our customers and partners and capitalize on the immense opportunities ahead.”

Select hardware partners, including Dell Technologies, Hewlett Packard Enterprise, Lenovo and Supermicro, are sampling the Gaudi 3 this quarter, with the air-cooled version of Gaudi 3 expected to ship to customers in Q3 24, and the water-cooled version in Q4 24.

 

 

Keep pace with the fast-moving world of data centers and cloud computing by connecting with Data Center Frontier on LinkedIn, following us on X/Twitter and Facebook, and signing up for our weekly newsletters using the form below.

About the Author

David Chernicoff

David Chernicoff is an experienced technologist and editorial content creator with the ability to see the connections between technology and business while figuring out how to get the most from both and to explain the needs of business to IT and IT to business.

Sponsored Recommendations

The AI Disruption: Challenges and Guidance for Data Center Design

From large training clusters to small edge inference servers, AI is becoming a larger percentage of data center workloads. Learn more.

A better approach to boost data center capacity – Supply capacity agreements

Explore a transformative approach to data center capacity planning with insights on supply capacity agreements, addressing the impact of COVID-19, the AI race, and the evolving...

How Modernizing Aging Data Center Infrastructure Improves Sustainability

Explore the path to improved sustainability in data centers by modernizing aging infrastructure, uncovering challenges, three effective approaches, and specific examples outlined...

How Modern DCIM Helps Multi-Tenant Colocation Data Centers Be More Competitive

Discover the transformative impact of modern DCIM software on multi-tenant colocation data centers, enhancing competitiveness through improved resiliency, security, environmental...

ZincFive

Rethinking Resilience: Preparing Data Centers for the Next AI Wave

Tim Hysell, Co-founder and CEO of ZincFive, explains why the need for a resilient power infrastructure has never been more pressing for data center operators.

White Papers

Dcf A10 Sr Cover 2023 01 17 14 23 57

The Security Gap: DDoS Protection in a Connected World

Jan. 18, 2023
The world is in love with connectivity, but it comes with a whole host of challenges for data centers. As customers continue to shift to the cloud and colocation services, security...