* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Sunday, July 27, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Top 5 entertainment news: Sandeep Reddy Vanga regrets trimming Animal’s runtime by 7-8 minutes, Akshay Ku – Times of India

    Top 5 Entertainment Highlights: Sandeep Reddy Vanga Reveals Why He Trimmed Animal’s Runtime by 7-8 Minutes, Plus Akshay Ku Updates

    Cote de Pablo reveals how Michael Weatherly used his soap opera roots to put her at ease in “NCIS” love scene – yahoo.com

    Cote de Pablo Reveals How Michael Weatherly’s Soap Opera Background Made Their “NCIS” Love Scene Easier

    City of Pelham announces entertainment district plans for former Oak Mountain Amphitheatre site – WVTM

    Pelham Unveils Exciting New Entertainment District Plans for Former Oak Mountain Amphitheatre Site

    Black Box Players presents ‘The Three Musketeers’ – CBS 19 News

    Experience the Adventure: Black Box Players Bring ‘The Three Musketeers’ to Life!

    AP Entertainment SummaryBrief at 1:51 p.m. EDT – Channel 3000

    Entertainment Highlights: Key Updates You Can’t Miss

    ‘Devil Wears Prada 2’ casts Anne Hathaway’s love interest replacing Adrian Grenier’s Nate – Entertainment Weekly

    Devil Wears Prada 2′ Casts New Love Interest for Anne Hathaway, Replacing Adrian Grenier’s Nate

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology

    Eagle Unveils Revolutionary X-Ray Technology at Pack Expo

    Validea’s Top Information Technology Stocks Based On Peter Lynch – 7/25/2025 – Nasdaq

    Validea’s Top Information Technology Stocks Based On Peter Lynch – 7/25/2025 – Nasdaq

    WhoFi: New surveillance technology can track people by how they disrupt Wi-Fi signals – Tech Xplore

    WhoFi: New surveillance technology can track people by how they disrupt Wi-Fi signals – Tech Xplore

    Google Cloud Announced as a Key Technology Partner for Odoo Connect 2025 in San Francisco – GlobeNewswire

    Google Cloud Announced as a Key Technology Partner for Odoo Connect 2025 in San Francisco – GlobeNewswire

    Behind the Screens: The Impact of Technology on Real Estate – TRREB

    Behind the Screens: How Technology is Transforming the Future of Real Estate

    Sustainserv and Palau Announce Technology Partnership to Leverage Innovative AI Platform to Advance Sustainability Reporting – Business Wire

    Sustainserv and Palau Team Up to Transform Sustainability Reporting with Breakthrough AI Technology

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Top 5 entertainment news: Sandeep Reddy Vanga regrets trimming Animal’s runtime by 7-8 minutes, Akshay Ku – Times of India

    Top 5 Entertainment Highlights: Sandeep Reddy Vanga Reveals Why He Trimmed Animal’s Runtime by 7-8 Minutes, Plus Akshay Ku Updates

    Cote de Pablo reveals how Michael Weatherly used his soap opera roots to put her at ease in “NCIS” love scene – yahoo.com

    Cote de Pablo Reveals How Michael Weatherly’s Soap Opera Background Made Their “NCIS” Love Scene Easier

    City of Pelham announces entertainment district plans for former Oak Mountain Amphitheatre site – WVTM

    Pelham Unveils Exciting New Entertainment District Plans for Former Oak Mountain Amphitheatre Site

    Black Box Players presents ‘The Three Musketeers’ – CBS 19 News

    Experience the Adventure: Black Box Players Bring ‘The Three Musketeers’ to Life!

    AP Entertainment SummaryBrief at 1:51 p.m. EDT – Channel 3000

    Entertainment Highlights: Key Updates You Can’t Miss

    ‘Devil Wears Prada 2’ casts Anne Hathaway’s love interest replacing Adrian Grenier’s Nate – Entertainment Weekly

    Devil Wears Prada 2′ Casts New Love Interest for Anne Hathaway, Replacing Adrian Grenier’s Nate

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology

    Eagle Unveils Revolutionary X-Ray Technology at Pack Expo

    Validea’s Top Information Technology Stocks Based On Peter Lynch – 7/25/2025 – Nasdaq

    Validea’s Top Information Technology Stocks Based On Peter Lynch – 7/25/2025 – Nasdaq

    WhoFi: New surveillance technology can track people by how they disrupt Wi-Fi signals – Tech Xplore

    WhoFi: New surveillance technology can track people by how they disrupt Wi-Fi signals – Tech Xplore

    Google Cloud Announced as a Key Technology Partner for Odoo Connect 2025 in San Francisco – GlobeNewswire

    Google Cloud Announced as a Key Technology Partner for Odoo Connect 2025 in San Francisco – GlobeNewswire

    Behind the Screens: The Impact of Technology on Real Estate – TRREB

    Behind the Screens: How Technology is Transforming the Future of Real Estate

    Sustainserv and Palau Announce Technology Partnership to Leverage Innovative AI Platform to Advance Sustainability Reporting – Business Wire

    Sustainserv and Palau Team Up to Transform Sustainability Reporting with Breakthrough AI Technology

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Intel wants to run AI on CPUs and says its 5th-gen Xeons are ones to do it

December 14, 2023
in Technology
Intel wants to run AI on CPUs and says its 5th-gen Xeons are ones to do it
Share on FacebookShare on Twitter

Intel launched its 5th-generation Xeon Scalable processors with more cores, cache, and machine learning grunt during its AI Everywhere Event in New York Thursday.

The x86 giant hopes the chip will help it win over customers struggling to get their hands on dedicated AI accelerators, touting the processor as “the Best CPU for AI, hands down.” This claim is no doubt bolstered by the fact Intel is one of the few chipmakers to have baked AI acceleration, in this case its Advanced Matrix Extensions (AMX) instructions, into their datacenter chips.

Compared to Sapphire Rapids, which we’ll remind you only launched this January after more than a year of delays, Intel says its 5th-gen Xeons are as much as 1.4x faster in AI inferencing and can deliver acceptable latencies for a wide variety of admittedly smaller machine learning applications.

Before we dig into Intel’s CPU-accelerated AI strategy, let’s take a look at the chip itself. Despite this being a refresh year for the Xeon family, Intel has actually changed quite a bit under the hood to boost the chip’s performance and efficiency compared to last-gen.

Fewer chips, more cores and cache

Emerald Rapids brings several notable improvements over its predecessor, which we largely see in the form of higher core counts and L3 cache.

The new chips can now be had with up to 64 cores. For a chip launching on the eve of 2024, that’s not a whole lot of cores. AMD hit this mark in 2019 with the launch of Epyc 2, and most chipmakers, including several of the cloud providers, are now deploying chips with 72, 96, or 128 or more cores.

The good news is, unlike January’s Sapphire Rapids launch, the highest core count parts aren’t reserved for large four-or eight-socket platforms this time around. Previously, Intel’s mainstream Xeons topped out at 56 cores. The bad news is, if you did want a large multi-socket server, you’re gonna be stuck on Sapphire Rapids, at least until next year, as Intel’s 5th-gen Xeons are limited to just two-socket platforms.

While you might think Intel would be using more chiplets to increase core counts, similar to how AMD boosted their Epyc 4 parts to 96 cores last year, they’re not.

Intel's 5th-gen Xeons use fewer larger compute tiles than we saw with Sapphire Rapids earlier this year.

Intel’s 5th-gen Xeons use fewer larger compute tiles than we saw with Sapphire Rapids earlier this year. – Click to enlarge

Strip away the integrated heat spreader, and you’ll find a much simpler arrangement of chiplets compared to Sapphire Rapids. Instead of meshing together four compute tiles, Emerald Rapids pairs this back to two of what it calls XCC dies, each with up to 32 cores. 

There are a couple of benefits to this, namely fewer dies means less data movement and therefore lower power consumption. One consequence of this approach is that these extreme core count (XCC) dies, while fewer, are physically larger. Usually larger dies means lower yields, but the Intel 7 process tech used in both Sapphire Rapids and now Emerald Rapids is quite mature at this point.

For lower core count parts, Intel continues to employ a single monolithic die. These medium-core-count dies (MCC), as Intel calls them, can still be had with up to 32 cores. What’s new this generation is the availability of an even smaller die called EE-LCC which is good for up to 20 cores.

In addition to more cores, Emerald Rapids boasts a much larger L3 cache at 320MB. That’s up from 112.5MB of L3 last generation. This larger cache, combined with the simpler chiplet architecture, is largely responsible for the chip’s 1.21x performance gains over last gen.

Finally, to keep the cores fed, Intel has extended support to faster DDR5 memory, up to 5,600 MT/s. While the chip is still stuck with eight memory channels — four fewer than AMD’s Epyc 4 or AWS’s Graviton 4 — it’s now able to deliver peak bandwidth of 368 GB/s or about 5.75 GB/s per core on the top-specced part.

Take these with a grain of salt, but at least in a core-for-core comparison Intel says its Emerald Rapids Xeons offer up to 2.5x the performance of AMD's Epycs.

Take these with a grain of salt, but at least in a core-for-core comparison Intel says its Emerald Rapids Xeons offer up to 2.5x the performance of AMD’s Epycs. – Click to enlarge

Altogether, Intel claims its 5th-gen Xeons offer competitive advantage over AMD’s Epyc 4 processors in a variety of benchmarks pitting its 64-core part against a similarly equipped Eypc 9554. As usual, take these with a grain of salt. Although the benchmarks demonstrate a core-for-core lead, they don’t account for the fact AMD’s Epyc 4 platform is available with between 50 and 100 percent more cores. So, while Intel’s cores may in fact be faster, AMD can still pack more of them into a dual-socket server.

Can CPUs make sense for AI inferencing? Intel seems to think so

With demand for AI accelerators far outstripping supply, Intel is pushing its Emerald Rapids Xeons as an ideal platform for inferencing and has made several notable improvements to the silicon to bolster the capabilities of its AMX accelerators.

In particular, Intel has tweaked the turbo frequencies of its AVX-512 and AMX blocks to reduce the performance hit associated with activating these instructions. This, in addition to architectural improvements, translates into 42 percent higher inferencing performance in certain workloads, compared to its predecessor, the company claims.

However, with LLMs, like GPT-4, Meta’s Llama 2, and Stable Diffusion all the rage, Intel is also talking up its ability to run smaller models on CPUs. For these kinds of workloads, memory bandwidth and latency are major factors. Here, the chip’s faster 5600 MT/s DDR5 helps, but it’s no replacement for HBM. And while Intel actually has made CPUs with HBM on board, its Xeon Max series processors used in the Aurora and Crossroads supercomputers aren’t making a return this generation.

According to Intel, large language models are well within the capability of its 5th-gen Xeons up to about 20 billion parameters

According to Intel, large language models are well within the capability of its 5th-gen Xeons up to about 20 billion parameters – Click to enlarge

Even so, Intel says it can achieve next-token latencies — this is how quickly words or phrases can be generated in response to a prompt — of around 25 milliseconds in the GPT-J model using a dual-socket Xeon platform.

But as you can see from the chart, as the number of parameters increases so does latency. Even still, Intel says it was able to achieve latencies as low as 62 milliseconds when running the Llama 2 13B model, well below the 100 milliseconds the chipmaker deems adequate.

We’re told that Intel has been able to achieve acceptable latencies on models up to about 20 billion parameters. Beyond this, the company has demonstrated acceptable second token latencies by distributing models, like Meta’s 70 billion parameter Llama 2 model across four dual-socket nodes. 

Despite this limitation, Intel insists its customers are asking them for help running inference on CPUs, which we don’t doubt. The ability to run LLMs and other ML workloads at acceptable levels of performance has the potential to significantly reduce costs, especially given the astronomical price of GPUs these days.

However, for those looking to run larger models, like GPT-3 at 175 billion parameters, it seems that dedicated AI accelerators like Intel’s own Habana Gaudi 2 aren’t going anywhere any time soon.

AMD slaps together a silicon sandwich with MI300-series APUs, GPUs to challenge Nvidia’s AI empire

AWS unveils core-packed Graviton4 and beefier Trainium accelerators for AI

AWS unveils core-packed Graviton4 and beefier Trainium accelerators for AI

Like Microsoft, Google can’t stop its cloud from pouring AI all over your heads

The best is yet to come

Despite the improvements brought by Intel’s Emerald Rapids Xeons, much of the chip’s thunder has already been stolen by the vendor’s next-gen datacenter parts.

Intel has spent the last few months teasing its performance and efficiency core Xeons, codenamed Granite Rapids and Sierra Forest, respectively. The parts promise to include much higher core counts, support for more, faster memory, and will be among the first to employ Intel’s long-delayed 7nm (A.K.A. Intel 3) process tech.

Sierra Forest is due out in the first half of next year and will offer up to 288 efficiency cores in a single socket — 144 cores per compute tile.

Granite Rapids, meanwhile, is slated to arrive later in 2024. As we learned at Intel Innovation this summer, the chip will employ a new modular chiplet design with up to three compute tiles flanked by a pair of I/O dies on the upper and lower edges of the chip.

Intel has yet to say just how many more cores Granite Rapids will offer, but at Hot Chips this summer it did reveal we’d be getting 136 PCIe lanes and 12 memory channels with support for 8,800 MT/s MCR DIMMS. The latter will boost the chip’s memory bandwidth to roughly 845 GB/s, something that should help considerably with LLM inference performance.

Of course, these chips aren’t launching in a vacuum. AMD is expected to roll out its 5th-gen Epyc processors, codenamed Turin, sometime next year. Elsewhere, many of the major cloud providers have announced Arm-based CPUs of their own. ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2023/12/14/intel_xeon_ai/

Tags: Inteltechnologywants
Previous Post

Russia joins North Korea in sending state-sponsored cyber troops to pick on TeamCity users

Next Post

Google Will Turn Off Cookies for 30 Million People on January 4

At United Nations Summer Program, Computer Science Student Examines AI Ethics – Georgetown University

At United Nations Summer Program, Computer Science Student Examines AI Ethics – Georgetown University

July 27, 2025
Return of wolves to Yellowstone has led to a surge in aspen trees unseen for 80 years – Live Science

Wolves’ Return to Yellowstone Sparks Unprecedented Aspen Tree Boom After 80 Years

July 27, 2025
‘Bluey’ Is Growing Up: BBC Plots All-Ages Expansion For The Franchise As A Full-Fledged Lifestyle Brand – Cartoon Brew

‘Bluey’ Is Growing Up: BBC Plots All-Ages Expansion For The Franchise As A Full-Fledged Lifestyle Brand – Cartoon Brew

July 27, 2025
Katie Ledecky, Summer McIntosh and a race for the ages at World Swimming Championships – NBC Sports

Katie Ledecky and Summer McIntosh Ignite an Epic Showdown at the World Swimming Championships

July 27, 2025
5 ways Trump has shaped the economy in 6 months – The Hill

5 Game-Changing Ways Trump Transformed the Economy in Just 6 Months

July 27, 2025
Flutter Entertainment Announces Pricing Of $1.272 Bln Of Additional Senior Secured Notes Due 2031 – Nasdaq

Flutter Entertainment Raises $1.27 Billion in Major Senior Secured Notes Offering Due 2031

July 27, 2025
KFF Health Tracking Poll: Public Views on Recent Tax and Budget Legislation – KFF

How the Public Really Feels About the Latest Tax and Budget Changes

July 27, 2025
Dollar falls against yen as markets weigh new trade deal, Japanese politics – Reuters

Dollar falls against yen as markets weigh new trade deal, Japanese politics – Reuters

July 27, 2025
Clinch County Schools prepare for students’ return with new policies, technology – WALB

Clinch County Schools Prepare for Students’ Return with Exciting New Policies and Cutting-Edge Technology

July 27, 2025
Highlights: Xfinity Series at Indy on The CW – NBC Sports

Unforgettable Thrills Ignite the Xfinity Series at Indy

July 27, 2025

Categories

Archives

July 2025
MTWTFSS
 123456
78910111213
14151617181920
21222324252627
28293031 
« Jun    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (740)
  • Economy (765)
  • Entertainment (21,645)
  • General (16,132)
  • Health (9,803)
  • Lifestyle (773)
  • News (22,149)
  • People (767)
  • Politics (774)
  • Science (15,980)
  • Sports (21,262)
  • Technology (15,746)
  • World (748)

Recent News

At United Nations Summer Program, Computer Science Student Examines AI Ethics – Georgetown University

At United Nations Summer Program, Computer Science Student Examines AI Ethics – Georgetown University

July 27, 2025
Return of wolves to Yellowstone has led to a surge in aspen trees unseen for 80 years – Live Science

Wolves’ Return to Yellowstone Sparks Unprecedented Aspen Tree Boom After 80 Years

July 27, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version