* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Tuesday, July 8, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Longhua District and Max-Matching Entertainments, supported by RWS Global forge strategic partnership to develop international IP-themed entertainment complex – Amusement Today

    Longhua District and Max-Matching Entertainments, supported by RWS Global forge strategic partnership to develop international IP-themed entertainment complex – Amusement Today

    Government whip to withdraw Entertainment Complex Bill on July 9 – Nation Thailand

    Government whip to withdraw Entertainment Complex Bill on July 9 – Nation Thailand

    Magicians and Battlebots light up Las Vegas entertainment scene – KSNV

    Magicians and Battlebots Take Las Vegas Entertainment by Storm

    Max-Matching Entertainments & Longhua District form partnership for new entertainment complex – Blooloop

    Max-Matching Entertainments and Longhua District Unite to Launch Thrilling New Entertainment Complex

    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    MAY HER SOUL REST IN PEACE 🙏 Veteran entertainment columnist and talent manager Lolit Solis has passed away. She was 78 years old. https://tinyurl.com/6kumarkx | LatestChika.com – Facebook

    Beloved Entertainment Icon Lolit Solis Passes Away at 78 – A Life Remembered with Love and Respect 🙏

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Technology And Construction Names Join Top Stock Lists: Check Out Additions To IBD 50, Big Cap 20 And More – Investor’s Business Daily

    Technology and Construction Leaders Surge Into Top Stock Rankings: See the Latest Additions to IBD 50, Big Cap 20, and More

    Column: Teach kupuna new technology skills – Honolulu Star-Advertiser

    Empowering Kupuna: Unlocking New Technology Skills for a Connected Future

    EIFO invests $5 million in D3, the Ukraine-focused defence technology venture fund – sUAS News

    EIFO Pledges $5 Million to Supercharge Ukraine-Focused Defense Technology Fund

    New Technology for Water Efficiency and Working with Mexico on Screwworm – AG INFORMATION NETWORK OF THE WEST

    Revolutionary Water Efficiency Technology and Cross-Border Collaboration to Defeat Screwworm

    Environmental cognitive distance, R&D capability distance, and supply chain green technology innovation – Nature

    Bridging Gaps: How Environmental and R&D Differences Drive Green Technology Innovation in Supply Chains

    LG Innotek CEO Moon Hyuksoo: “Our Next-gen Substrate Technology Will Change the Industry Paradigm” – TechPowerUp

    LG Innotek CEO Moon Hyuksoo: “Our Next-Gen Substrate Technology Will Revolutionize the Industry” Revolutionizing the Future: LG Innotek’s CEO Unveils Game-Changing Next-Gen Substrate Technology

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Longhua District and Max-Matching Entertainments, supported by RWS Global forge strategic partnership to develop international IP-themed entertainment complex – Amusement Today

    Longhua District and Max-Matching Entertainments, supported by RWS Global forge strategic partnership to develop international IP-themed entertainment complex – Amusement Today

    Government whip to withdraw Entertainment Complex Bill on July 9 – Nation Thailand

    Government whip to withdraw Entertainment Complex Bill on July 9 – Nation Thailand

    Magicians and Battlebots light up Las Vegas entertainment scene – KSNV

    Magicians and Battlebots Take Las Vegas Entertainment by Storm

    Max-Matching Entertainments & Longhua District form partnership for new entertainment complex – Blooloop

    Max-Matching Entertainments and Longhua District Unite to Launch Thrilling New Entertainment Complex

    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    Kennedy Publishing, MGA Entertainment Launch Yummiland Magazine – License Global

    MAY HER SOUL REST IN PEACE 🙏 Veteran entertainment columnist and talent manager Lolit Solis has passed away. She was 78 years old. https://tinyurl.com/6kumarkx | LatestChika.com – Facebook

    Beloved Entertainment Icon Lolit Solis Passes Away at 78 – A Life Remembered with Love and Respect 🙏

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Technology And Construction Names Join Top Stock Lists: Check Out Additions To IBD 50, Big Cap 20 And More – Investor’s Business Daily

    Technology and Construction Leaders Surge Into Top Stock Rankings: See the Latest Additions to IBD 50, Big Cap 20, and More

    Column: Teach kupuna new technology skills – Honolulu Star-Advertiser

    Empowering Kupuna: Unlocking New Technology Skills for a Connected Future

    EIFO invests $5 million in D3, the Ukraine-focused defence technology venture fund – sUAS News

    EIFO Pledges $5 Million to Supercharge Ukraine-Focused Defense Technology Fund

    New Technology for Water Efficiency and Working with Mexico on Screwworm – AG INFORMATION NETWORK OF THE WEST

    Revolutionary Water Efficiency Technology and Cross-Border Collaboration to Defeat Screwworm

    Environmental cognitive distance, R&D capability distance, and supply chain green technology innovation – Nature

    Bridging Gaps: How Environmental and R&D Differences Drive Green Technology Innovation in Supply Chains

    LG Innotek CEO Moon Hyuksoo: “Our Next-gen Substrate Technology Will Change the Industry Paradigm” – TechPowerUp

    LG Innotek CEO Moon Hyuksoo: “Our Next-Gen Substrate Technology Will Revolutionize the Industry” Revolutionizing the Future: LG Innotek’s CEO Unveils Game-Changing Next-Gen Substrate Technology

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Intel wants to run AI on CPUs and says its 5th-gen Xeons are ones to do it

December 14, 2023
in Technology
Intel wants to run AI on CPUs and says its 5th-gen Xeons are ones to do it
Share on FacebookShare on Twitter

Intel launched its 5th-generation Xeon Scalable processors with more cores, cache, and machine learning grunt during its AI Everywhere Event in New York Thursday.

The x86 giant hopes the chip will help it win over customers struggling to get their hands on dedicated AI accelerators, touting the processor as “the Best CPU for AI, hands down.” This claim is no doubt bolstered by the fact Intel is one of the few chipmakers to have baked AI acceleration, in this case its Advanced Matrix Extensions (AMX) instructions, into their datacenter chips.

Compared to Sapphire Rapids, which we’ll remind you only launched this January after more than a year of delays, Intel says its 5th-gen Xeons are as much as 1.4x faster in AI inferencing and can deliver acceptable latencies for a wide variety of admittedly smaller machine learning applications.

Before we dig into Intel’s CPU-accelerated AI strategy, let’s take a look at the chip itself. Despite this being a refresh year for the Xeon family, Intel has actually changed quite a bit under the hood to boost the chip’s performance and efficiency compared to last-gen.

Fewer chips, more cores and cache

Emerald Rapids brings several notable improvements over its predecessor, which we largely see in the form of higher core counts and L3 cache.

The new chips can now be had with up to 64 cores. For a chip launching on the eve of 2024, that’s not a whole lot of cores. AMD hit this mark in 2019 with the launch of Epyc 2, and most chipmakers, including several of the cloud providers, are now deploying chips with 72, 96, or 128 or more cores.

The good news is, unlike January’s Sapphire Rapids launch, the highest core count parts aren’t reserved for large four-or eight-socket platforms this time around. Previously, Intel’s mainstream Xeons topped out at 56 cores. The bad news is, if you did want a large multi-socket server, you’re gonna be stuck on Sapphire Rapids, at least until next year, as Intel’s 5th-gen Xeons are limited to just two-socket platforms.

While you might think Intel would be using more chiplets to increase core counts, similar to how AMD boosted their Epyc 4 parts to 96 cores last year, they’re not.

Intel's 5th-gen Xeons use fewer larger compute tiles than we saw with Sapphire Rapids earlier this year.

Intel’s 5th-gen Xeons use fewer larger compute tiles than we saw with Sapphire Rapids earlier this year. – Click to enlarge

Strip away the integrated heat spreader, and you’ll find a much simpler arrangement of chiplets compared to Sapphire Rapids. Instead of meshing together four compute tiles, Emerald Rapids pairs this back to two of what it calls XCC dies, each with up to 32 cores. 

There are a couple of benefits to this, namely fewer dies means less data movement and therefore lower power consumption. One consequence of this approach is that these extreme core count (XCC) dies, while fewer, are physically larger. Usually larger dies means lower yields, but the Intel 7 process tech used in both Sapphire Rapids and now Emerald Rapids is quite mature at this point.

For lower core count parts, Intel continues to employ a single monolithic die. These medium-core-count dies (MCC), as Intel calls them, can still be had with up to 32 cores. What’s new this generation is the availability of an even smaller die called EE-LCC which is good for up to 20 cores.

In addition to more cores, Emerald Rapids boasts a much larger L3 cache at 320MB. That’s up from 112.5MB of L3 last generation. This larger cache, combined with the simpler chiplet architecture, is largely responsible for the chip’s 1.21x performance gains over last gen.

Finally, to keep the cores fed, Intel has extended support to faster DDR5 memory, up to 5,600 MT/s. While the chip is still stuck with eight memory channels — four fewer than AMD’s Epyc 4 or AWS’s Graviton 4 — it’s now able to deliver peak bandwidth of 368 GB/s or about 5.75 GB/s per core on the top-specced part.

Take these with a grain of salt, but at least in a core-for-core comparison Intel says its Emerald Rapids Xeons offer up to 2.5x the performance of AMD's Epycs.

Take these with a grain of salt, but at least in a core-for-core comparison Intel says its Emerald Rapids Xeons offer up to 2.5x the performance of AMD’s Epycs. – Click to enlarge

Altogether, Intel claims its 5th-gen Xeons offer competitive advantage over AMD’s Epyc 4 processors in a variety of benchmarks pitting its 64-core part against a similarly equipped Eypc 9554. As usual, take these with a grain of salt. Although the benchmarks demonstrate a core-for-core lead, they don’t account for the fact AMD’s Epyc 4 platform is available with between 50 and 100 percent more cores. So, while Intel’s cores may in fact be faster, AMD can still pack more of them into a dual-socket server.

Can CPUs make sense for AI inferencing? Intel seems to think so

With demand for AI accelerators far outstripping supply, Intel is pushing its Emerald Rapids Xeons as an ideal platform for inferencing and has made several notable improvements to the silicon to bolster the capabilities of its AMX accelerators.

In particular, Intel has tweaked the turbo frequencies of its AVX-512 and AMX blocks to reduce the performance hit associated with activating these instructions. This, in addition to architectural improvements, translates into 42 percent higher inferencing performance in certain workloads, compared to its predecessor, the company claims.

However, with LLMs, like GPT-4, Meta’s Llama 2, and Stable Diffusion all the rage, Intel is also talking up its ability to run smaller models on CPUs. For these kinds of workloads, memory bandwidth and latency are major factors. Here, the chip’s faster 5600 MT/s DDR5 helps, but it’s no replacement for HBM. And while Intel actually has made CPUs with HBM on board, its Xeon Max series processors used in the Aurora and Crossroads supercomputers aren’t making a return this generation.

According to Intel, large language models are well within the capability of its 5th-gen Xeons up to about 20 billion parameters

According to Intel, large language models are well within the capability of its 5th-gen Xeons up to about 20 billion parameters – Click to enlarge

Even so, Intel says it can achieve next-token latencies — this is how quickly words or phrases can be generated in response to a prompt — of around 25 milliseconds in the GPT-J model using a dual-socket Xeon platform.

But as you can see from the chart, as the number of parameters increases so does latency. Even still, Intel says it was able to achieve latencies as low as 62 milliseconds when running the Llama 2 13B model, well below the 100 milliseconds the chipmaker deems adequate.

We’re told that Intel has been able to achieve acceptable latencies on models up to about 20 billion parameters. Beyond this, the company has demonstrated acceptable second token latencies by distributing models, like Meta’s 70 billion parameter Llama 2 model across four dual-socket nodes. 

Despite this limitation, Intel insists its customers are asking them for help running inference on CPUs, which we don’t doubt. The ability to run LLMs and other ML workloads at acceptable levels of performance has the potential to significantly reduce costs, especially given the astronomical price of GPUs these days.

However, for those looking to run larger models, like GPT-3 at 175 billion parameters, it seems that dedicated AI accelerators like Intel’s own Habana Gaudi 2 aren’t going anywhere any time soon.

AMD slaps together a silicon sandwich with MI300-series APUs, GPUs to challenge Nvidia’s AI empire

AWS unveils core-packed Graviton4 and beefier Trainium accelerators for AI

AWS unveils core-packed Graviton4 and beefier Trainium accelerators for AI

Like Microsoft, Google can’t stop its cloud from pouring AI all over your heads

The best is yet to come

Despite the improvements brought by Intel’s Emerald Rapids Xeons, much of the chip’s thunder has already been stolen by the vendor’s next-gen datacenter parts.

Intel has spent the last few months teasing its performance and efficiency core Xeons, codenamed Granite Rapids and Sierra Forest, respectively. The parts promise to include much higher core counts, support for more, faster memory, and will be among the first to employ Intel’s long-delayed 7nm (A.K.A. Intel 3) process tech.

Sierra Forest is due out in the first half of next year and will offer up to 288 efficiency cores in a single socket — 144 cores per compute tile.

Granite Rapids, meanwhile, is slated to arrive later in 2024. As we learned at Intel Innovation this summer, the chip will employ a new modular chiplet design with up to three compute tiles flanked by a pair of I/O dies on the upper and lower edges of the chip.

Intel has yet to say just how many more cores Granite Rapids will offer, but at Hot Chips this summer it did reveal we’d be getting 136 PCIe lanes and 12 memory channels with support for 8,800 MT/s MCR DIMMS. The latter will boost the chip’s memory bandwidth to roughly 845 GB/s, something that should help considerably with LLM inference performance.

Of course, these chips aren’t launching in a vacuum. AMD is expected to roll out its 5th-gen Epyc processors, codenamed Turin, sometime next year. Elsewhere, many of the major cloud providers have announced Arm-based CPUs of their own. ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2023/12/14/intel_xeon_ai/

Tags: Inteltechnologywants
Previous Post

Russia joins North Korea in sending state-sponsored cyber troops to pick on TeamCity users

Next Post

Google Will Turn Off Cookies for 30 Million People on January 4

An episodic burst of massive genomic rearrangements and the origin of non-marine annelids – Nature

Explosive Genomic Shifts Ignite the Evolutionary Rise of Land-Dwelling Annelids

July 8, 2025
Earth is going to spin much faster over the next few months — so fast that several days are going to get shorter – Live Science

Earth is going to spin much faster over the next few months — so fast that several days are going to get shorter – Live Science

July 8, 2025
Putnam Museum and Science Center Ribbon-Cutting July 12 – River Cities’ Reader

Putnam Museum and Science Center Ribbon-Cutting July 12 – River Cities’ Reader

July 8, 2025
New Miiro Lifestyle Hotel Brand Expands Further In Europe – Forbes

New Miiro Lifestyle Hotel Brand Expands Further In Europe – Forbes

July 8, 2025
Dino Might!: ‘Jurassic World Rebirth’ Bows To $318.3M Global In Biggest Studio Opening Year-To-Date WW; ‘F1’ Nears $300M’ & ‘Dragon’ Tops $500M – International Box Office – Deadline

Dino Might!: ‘Jurassic World Rebirth’ Bows To $318.3M Global In Biggest Studio Opening Year-To-Date WW; ‘F1’ Nears $300M’ & ‘Dragon’ Tops $500M – International Box Office – Deadline

July 8, 2025
As a Nation’s Economy Slows, Some Say It’s No Time for a Free Lunch – The New York Times

As a Nation’s Economy Slows, Some Say It’s No Time for a Free Lunch – The New York Times

July 8, 2025
Longhua District and Max-Matching Entertainments, supported by RWS Global forge strategic partnership to develop international IP-themed entertainment complex – Amusement Today

Longhua District and Max-Matching Entertainments, supported by RWS Global forge strategic partnership to develop international IP-themed entertainment complex – Amusement Today

July 8, 2025
The 63 Best Amazon Prime Day Health & Fitness Deals, Up to 85% Off – health.com

The 63 Best Amazon Prime Day Health & Fitness Deals, Up to 85% Off – health.com

July 8, 2025
Scoop: Sean Patrick Maloney mulls political comeback against Mike Lawler – Axios

Sean Patrick Maloney Eyes Bold Political Comeback, Taking on Mike Lawler

July 8, 2025

Lake Land College to host grand opening celebration for Effingham Technology Center – wandtv.com

July 7, 2025

Categories

Archives

July 2025
MTWTFSS
 123456
78910111213
14151617181920
21222324252627
28293031 
« Jun    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (710)
  • Economy (735)
  • Entertainment (21,623)
  • General (15,771)
  • Health (9,773)
  • Lifestyle (740)
  • News (22,149)
  • People (735)
  • Politics (744)
  • Science (15,952)
  • Sports (21,233)
  • Technology (15,718)
  • World (716)

Recent News

An episodic burst of massive genomic rearrangements and the origin of non-marine annelids – Nature

Explosive Genomic Shifts Ignite the Evolutionary Rise of Land-Dwelling Annelids

July 8, 2025
Earth is going to spin much faster over the next few months — so fast that several days are going to get shorter – Live Science

Earth is going to spin much faster over the next few months — so fast that several days are going to get shorter – Live Science

July 8, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version