* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Monday, August 4, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Go-to entertainment: why gaming was made for the toilet – The Guardian

    Why Gaming Is the Ultimate Way to Pass Time in the Bathroom

    Chicago Youth Symphony Orchestra takes the Lollapalooza stage – Yahoo Home

    Chicago Youth Symphony Orchestra takes the Lollapalooza stage – Yahoo Home

    Sens. Blackburn, Warnock introduce CREATE Act to provide tax relief to music creators – Yahoo Home

    Sens. Blackburn and Warnock Launch CREATE Act to Deliver Tax Relief for Music Creators

    That’s (Political) Entertainment: When Theatre Meets Politics

    Future Script: How Generative AI Is Changing Collective Bargaining in the Entertainment Industry – Jackson Lewis

    Future Script: How Generative AI Is Transforming Collective Bargaining in Entertainment

    The SBA’s live-entertainment bailout was supposed to end two years ago. We still don’t know how $1.5 billion was spent. – Yahoo Home

    $1.5 Billion Live-Entertainment Bailout: Two Years Later, Where Did the Money Go?

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Credo Technology: Wiring The AI Revolution (NASDAQ:CRDO) – Seeking Alpha

    Credo Technology: Driving the Next Wave of AI Innovation

    Microsoft Seeks to Extend Access to OpenAI Technology – PYMNTS.com

    Microsoft Aims to Broaden Access to OpenAI Technology

    Livonia police use grappler technology to stop drunk driver – ClickOnDetroit | WDIV Local 4

    Livonia Police Deploy Grappler Technology to Safely Stop Drunk Driver

    Emory orthopaedic surgeons use robotic technology to transform knee replacement surgery – Emory News Center

    How Robotic Technology is Revolutionizing Knee Replacement Surgery

    Cognizant Technology Solutions Corp (CTSH) Q2 2025 Earnings Call Highlights: Strong Revenue … – Yahoo.co

    Cognizant Q2 2025 Earnings: Impressive Revenue Growth and Key Takeaways

    Revving Up The U.S. Technology Engine – Forbes

    Revving Up The U.S. Technology Engine – Forbes

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Go-to entertainment: why gaming was made for the toilet – The Guardian

    Why Gaming Is the Ultimate Way to Pass Time in the Bathroom

    Chicago Youth Symphony Orchestra takes the Lollapalooza stage – Yahoo Home

    Chicago Youth Symphony Orchestra takes the Lollapalooza stage – Yahoo Home

    Sens. Blackburn, Warnock introduce CREATE Act to provide tax relief to music creators – Yahoo Home

    Sens. Blackburn and Warnock Launch CREATE Act to Deliver Tax Relief for Music Creators

    That’s (Political) Entertainment: When Theatre Meets Politics

    Future Script: How Generative AI Is Changing Collective Bargaining in the Entertainment Industry – Jackson Lewis

    Future Script: How Generative AI Is Transforming Collective Bargaining in Entertainment

    The SBA’s live-entertainment bailout was supposed to end two years ago. We still don’t know how $1.5 billion was spent. – Yahoo Home

    $1.5 Billion Live-Entertainment Bailout: Two Years Later, Where Did the Money Go?

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Credo Technology: Wiring The AI Revolution (NASDAQ:CRDO) – Seeking Alpha

    Credo Technology: Driving the Next Wave of AI Innovation

    Microsoft Seeks to Extend Access to OpenAI Technology – PYMNTS.com

    Microsoft Aims to Broaden Access to OpenAI Technology

    Livonia police use grappler technology to stop drunk driver – ClickOnDetroit | WDIV Local 4

    Livonia Police Deploy Grappler Technology to Safely Stop Drunk Driver

    Emory orthopaedic surgeons use robotic technology to transform knee replacement surgery – Emory News Center

    How Robotic Technology is Revolutionizing Knee Replacement Surgery

    Cognizant Technology Solutions Corp (CTSH) Q2 2025 Earnings Call Highlights: Strong Revenue … – Yahoo.co

    Cognizant Q2 2025 Earnings: Impressive Revenue Growth and Key Takeaways

    Revving Up The U.S. Technology Engine – Forbes

    Revving Up The U.S. Technology Engine – Forbes

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

ChatGPT-3.5, Claude 3 kick pixelated butt in Street Fighter III tournament for LLMs

April 11, 2024
in Technology
ChatGPT-3.5, Claude 3 kick pixelated butt in Street Fighter III tournament for LLMs
Share on FacebookShare on Twitter

Large language models (LLMs) can now be put to the test in the retro arcade video game Street Fighter III, and so far it seems some are better than others.

The Street Fighter III-based benchmark, termed LLM Colosseum, was created by four AI devs from Phospho and Quivr during the Mistral hackathon in San Francisco last month. The benchmark works by pitting two LLMs against each other in an actual game of Street Fighter III, keeping each updated on how close victory is, where the opposing LLM is, what move it took. Then it asks for what it would like to do, after which it will make a move.

According to the official leaderboard for LLM Colosseum, which is based on 342 fights between eight different LLMs, ChatGPT-3.5 Turbo is by far the winner, with an Elo rating of 1,776.11. That’s well ahead of several iterations of ChatGPT-4, which landed in the 1,400s to 1,500s.

What even makes an LLM good at Street Fighter III is balance between key characteristics, said Nicolas Oulianov, one of the LLM Colosseum developers. “GPT-3.5 turbo has a good balance between speed and brains. GPT-4 is a larger model, thus way smarter, but much slower.”

The disparity between ChatGPT-3.5 and 4 in LLM Colosseum is an indication of what features are being prioritized in the latest LLMs, according to Oulianov. “Existing benchmarks focus too much on performance regardless of speed. If you’re an AI developer, you need custom evaluations to see if GPT-4 is the best model for your users,” he said. Even fractions of a second can count in fighting games, so taking any extra time can result in a quick loss.

A different experiment with LLM Colosseum was documented by Amazon Web Services developer Banjo Obayomi, running models off Amazon Bedrock. This tournament involved a dozen different models, though Claude clearly swept the competition by snagging first to fourth place, with Claude 3 Haiku scoring first place.

Obayomi also tracked the quirky behavior that tested LLMs exhibited from time to time, including attempts to play invalid moves such as the devastating “hardest hitting combo of all.”

There were also instances where LLMs just refused to play anymore. The companies that create AI models tend to inject them with an anti-violent outlook, and will often refuse to answer any prompt that it deems to be too violent. Claude 2.1 was particularly pacifistic, saying it couldn’t tolerate even fictional fighting.

Compared to actual human players, though, these chatbots aren’t exactly playing at a pro level. “I fought a few SF3 games against LLMs,” says Oulianov. “So far, I think LLMs only stand a chance to win in Street Fighter 3 against a 70 or a five-year-old.”

MPs ask: Why is it so freakin’ hard to get AI giants to pay copyright holders?

Boffins deem Google DeepMind’s material discoveries rather shallow

US Air Force secretary so confident in AI-controlled F-16s, he’ll fly in one

Next-gen Meta AI chip serves up ads while sipping power

ChatGPT-4 similarly performed pretty poorly in Doom, another old-school game that requires quick thinking and fast movement.

But why test LLMs in a retro fighting game?

The idea of benchmarking LLMs in an old-school video game is funny and maybe that’s all the reason LLM Colosseum needs to exist, but it might be a little more than that. “Unlike other benchmarks you see in press releases, everyone played video games, and can get a feel of why it would be challenging for an LLM,” Oulianov said. “Large AI companies are gaming benchmarks to get pretty scores and show off.”

But he does note that “the Street Fighter benchmark is kind of the same, but way more entertaining.”

Beyond that, Oulianov said LLM Colosseum showcases how intelligent general-purpose LLMs already are. “What this project shows is the potential for LLMs to become so smart, so fast, and so versatile, that we can use them as ‘turnkey reasoning machines’ basically everywhere. The goal is to create machines able to not only reason with text, but also react to their environment and interact with other thinking machines.”

Oulianov also pointed out that there are already AI models out there that can play modern games at a professional level. DeepMind’s AlphaStar trashed StarCraft II pros back in 2018 and 2019, and OpenAI’s OpenAI Five model proved to be capable of beating world champions and cooperating effectively with human teammates.

Today’s chat-oriented LLMs aren’t anywhere near the level of purpose-made models (just try playing a game of chess against ChatGPT), but perhaps it won’t be that way forever. “With projects like this one, we show that this vision is closer to reality than science fiction,” Oulianov said. ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2024/04/11/chatgpt_claude_street_fighter_3/

Tags: ChatGPTClaudetechnology
Previous Post

US, Japan announce joint AI research projects funded by Nvidia, Microsoft, and others

Next Post

Next Vision, or Vision Next? What we really thought about Google and Intel’s AI events

At Washington Health Care Authority, workers are warned of layoffs – Washington State Standard

Washington Health Care Authority Alerts Employees of Potential Layoffs

August 4, 2025
NYPD program allowed slain officer to moonlight as private security guard – Spectrum News NY1

NYPD program allowed slain officer to moonlight as private security guard – Spectrum News NY1

August 4, 2025
Credo Technology: Wiring The AI Revolution (NASDAQ:CRDO) – Seeking Alpha

Credo Technology: Driving the Next Wave of AI Innovation

August 3, 2025
Judge Halts Stephen F. Austin’s Female Sports Cuts Amid Title IX Suit – Sportico.com

Judge Halts Stephen F. Austin’s Female Sports Cuts Amid Title IX Suit – Sportico.com

August 3, 2025
Study reveals China’s ecological red lines offer blueprint for biodiversity protection – Phys.org

Study reveals China’s ecological red lines offer blueprint for biodiversity protection – Phys.org

August 3, 2025
What do TSA bag scanners actually see? – Popular Science

What do TSA bag scanners actually see? – Popular Science

August 3, 2025
Why does your mind goes ‘blank’? New brain scans reveal the surprising answer – Live Science

Why does your mind goes ‘blank’? New brain scans reveal the surprising answer – Live Science

August 3, 2025
U.S. POINTER Study shows lifestyle program improves cognition in older adults – University of California – Davis Health

Lifestyle Program Boosts Cognitive Health in Older Adults, New Study Finds

August 3, 2025
World Championships, Day Eight Finals: Kliment Kolesnikov Blazes to Title in 50 Backstroke; Tie for Silver – Swimming World

Kliment Kolesnikov Blazes to Victory in 50 Backstroke as Day Eight of World Championships Ends with a Silver Medal Tie

August 3, 2025
Bond investors warm to risk, with Fed staying put in ‘Goldilocks’ economy – Reuters

Bond Investors Take Bold Steps as Fed Maintains Steady Course in a ‘Goldilocks’ Economy

August 3, 2025

Categories

Archives

August 2025
MTWTFSS
 123
45678910
11121314151617
18192021222324
25262728293031
« Jul    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (752)
  • Economy (778)
  • Entertainment (21,655)
  • General (16,269)
  • Health (9,815)
  • Lifestyle (785)
  • News (22,149)
  • People (777)
  • Politics (787)
  • Science (15,990)
  • Sports (21,273)
  • Technology (15,755)
  • World (758)

Recent News

At Washington Health Care Authority, workers are warned of layoffs – Washington State Standard

Washington Health Care Authority Alerts Employees of Potential Layoffs

August 4, 2025
NYPD program allowed slain officer to moonlight as private security guard – Spectrum News NY1

NYPD program allowed slain officer to moonlight as private security guard – Spectrum News NY1

August 4, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version