* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Saturday, May 31, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Entertainment: On Your Marks, Get Set, Beer Run! – Urban Milwaukee

    Get Ready to Race: The Ultimate Beer Run Experience Awaits!

    Rachel Guttman Launches Entertainment Law Firm Gutt Law, PLLC [Exclusive] – MusicRow.com

    Rachel Guttman Unveils Exciting New Entertainment Law Firm: Gutt Law, PLLC!

    HYBE Cashes In: Offloads Final Stake in K-Pop Rival SM Entertainment for $177 Million!

    Allied Gaming & Entertainment Receives Expected Nasdaq Notice Regarding Delayed Quarterly Report – Business Wire

    Allied Gaming & Entertainment Faces Nasdaq Notice Over Delayed Quarterly Report

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Aziz Ansari made Keanu Reeves Indian food so he wouldn’t feel ‘freaked out’ directing him in Good Fortune (exclusive) – Entertainment Weekly

    Aziz Ansari Whips Up Indian Cuisine for Keanu Reeves to Ease Directing Jitters in Good Fortune

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology

    Revolutionary Harvesting Technology Promises to Slash CAR-T Manufacturing Costs!

    Stop the Machines: The Rise of Anti-Technology Extremism – International Centre for Counter-Terrorism – ICCT

    Unplugged: The Surge of Anti-Technology Extremism

    Finland to head EU’s quantum defense technology project – Latest news from Azerbaijan

    Finland Takes the Lead in Pioneering EU’s Quantum Defense Technology Initiative!

    i3D Robotics Unveils Breakthrough Glass Defect Detection Technology – USGlass Magazine

    Revolutionary Glass Defect Detection Technology Unveiled by i3D Robotics!

    4Liberty Announces Collaboration with Itron to Drive Innovation in Utility Technology Adoption and Optimization – FinancialContent

    4Liberty Partners with Itron to Revolutionize Utility Technology and Boost Innovation

    Q1 2025 Quantum Technology Investment: What’s Driving the Surge in Quantum Investment? – The Quantum Insider

    Unleashing the Future: Exploring the Surge in Quantum Technology Investments for Q1 2025

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Entertainment: On Your Marks, Get Set, Beer Run! – Urban Milwaukee

    Get Ready to Race: The Ultimate Beer Run Experience Awaits!

    Rachel Guttman Launches Entertainment Law Firm Gutt Law, PLLC [Exclusive] – MusicRow.com

    Rachel Guttman Unveils Exciting New Entertainment Law Firm: Gutt Law, PLLC!

    HYBE Cashes In: Offloads Final Stake in K-Pop Rival SM Entertainment for $177 Million!

    Allied Gaming & Entertainment Receives Expected Nasdaq Notice Regarding Delayed Quarterly Report – Business Wire

    Allied Gaming & Entertainment Faces Nasdaq Notice Over Delayed Quarterly Report

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Aziz Ansari made Keanu Reeves Indian food so he wouldn’t feel ‘freaked out’ directing him in Good Fortune (exclusive) – Entertainment Weekly

    Aziz Ansari Whips Up Indian Cuisine for Keanu Reeves to Ease Directing Jitters in Good Fortune

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology

    Revolutionary Harvesting Technology Promises to Slash CAR-T Manufacturing Costs!

    Stop the Machines: The Rise of Anti-Technology Extremism – International Centre for Counter-Terrorism – ICCT

    Unplugged: The Surge of Anti-Technology Extremism

    Finland to head EU’s quantum defense technology project – Latest news from Azerbaijan

    Finland Takes the Lead in Pioneering EU’s Quantum Defense Technology Initiative!

    i3D Robotics Unveils Breakthrough Glass Defect Detection Technology – USGlass Magazine

    Revolutionary Glass Defect Detection Technology Unveiled by i3D Robotics!

    4Liberty Announces Collaboration with Itron to Drive Innovation in Utility Technology Adoption and Optimization – FinancialContent

    4Liberty Partners with Itron to Revolutionize Utility Technology and Boost Innovation

    Q1 2025 Quantum Technology Investment: What’s Driving the Surge in Quantum Investment? – The Quantum Insider

    Unleashing the Future: Exploring the Surge in Quantum Technology Investments for Q1 2025

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

ChatGPT-3.5, Claude 3 kick pixelated butt in Street Fighter III tournament for LLMs

April 11, 2024
in Technology
ChatGPT-3.5, Claude 3 kick pixelated butt in Street Fighter III tournament for LLMs
Share on FacebookShare on Twitter

Large language models (LLMs) can now be put to the test in the retro arcade video game Street Fighter III, and so far it seems some are better than others.

The Street Fighter III-based benchmark, termed LLM Colosseum, was created by four AI devs from Phospho and Quivr during the Mistral hackathon in San Francisco last month. The benchmark works by pitting two LLMs against each other in an actual game of Street Fighter III, keeping each updated on how close victory is, where the opposing LLM is, what move it took. Then it asks for what it would like to do, after which it will make a move.

According to the official leaderboard for LLM Colosseum, which is based on 342 fights between eight different LLMs, ChatGPT-3.5 Turbo is by far the winner, with an Elo rating of 1,776.11. That’s well ahead of several iterations of ChatGPT-4, which landed in the 1,400s to 1,500s.

What even makes an LLM good at Street Fighter III is balance between key characteristics, said Nicolas Oulianov, one of the LLM Colosseum developers. “GPT-3.5 turbo has a good balance between speed and brains. GPT-4 is a larger model, thus way smarter, but much slower.”

The disparity between ChatGPT-3.5 and 4 in LLM Colosseum is an indication of what features are being prioritized in the latest LLMs, according to Oulianov. “Existing benchmarks focus too much on performance regardless of speed. If you’re an AI developer, you need custom evaluations to see if GPT-4 is the best model for your users,” he said. Even fractions of a second can count in fighting games, so taking any extra time can result in a quick loss.

A different experiment with LLM Colosseum was documented by Amazon Web Services developer Banjo Obayomi, running models off Amazon Bedrock. This tournament involved a dozen different models, though Claude clearly swept the competition by snagging first to fourth place, with Claude 3 Haiku scoring first place.

Obayomi also tracked the quirky behavior that tested LLMs exhibited from time to time, including attempts to play invalid moves such as the devastating “hardest hitting combo of all.”

There were also instances where LLMs just refused to play anymore. The companies that create AI models tend to inject them with an anti-violent outlook, and will often refuse to answer any prompt that it deems to be too violent. Claude 2.1 was particularly pacifistic, saying it couldn’t tolerate even fictional fighting.

Compared to actual human players, though, these chatbots aren’t exactly playing at a pro level. “I fought a few SF3 games against LLMs,” says Oulianov. “So far, I think LLMs only stand a chance to win in Street Fighter 3 against a 70 or a five-year-old.”

MPs ask: Why is it so freakin’ hard to get AI giants to pay copyright holders?

Boffins deem Google DeepMind’s material discoveries rather shallow

US Air Force secretary so confident in AI-controlled F-16s, he’ll fly in one

Next-gen Meta AI chip serves up ads while sipping power

ChatGPT-4 similarly performed pretty poorly in Doom, another old-school game that requires quick thinking and fast movement.

But why test LLMs in a retro fighting game?

The idea of benchmarking LLMs in an old-school video game is funny and maybe that’s all the reason LLM Colosseum needs to exist, but it might be a little more than that. “Unlike other benchmarks you see in press releases, everyone played video games, and can get a feel of why it would be challenging for an LLM,” Oulianov said. “Large AI companies are gaming benchmarks to get pretty scores and show off.”

But he does note that “the Street Fighter benchmark is kind of the same, but way more entertaining.”

Beyond that, Oulianov said LLM Colosseum showcases how intelligent general-purpose LLMs already are. “What this project shows is the potential for LLMs to become so smart, so fast, and so versatile, that we can use them as ‘turnkey reasoning machines’ basically everywhere. The goal is to create machines able to not only reason with text, but also react to their environment and interact with other thinking machines.”

Oulianov also pointed out that there are already AI models out there that can play modern games at a professional level. DeepMind’s AlphaStar trashed StarCraft II pros back in 2018 and 2019, and OpenAI’s OpenAI Five model proved to be capable of beating world champions and cooperating effectively with human teammates.

Today’s chat-oriented LLMs aren’t anywhere near the level of purpose-made models (just try playing a game of chess against ChatGPT), but perhaps it won’t be that way forever. “With projects like this one, we show that this vision is closer to reality than science fiction,” Oulianov said. ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2024/04/11/chatgpt_claude_street_fighter_3/

Tags: ChatGPTClaudetechnology
Previous Post

US, Japan announce joint AI research projects funded by Nvidia, Microsoft, and others

Next Post

Next Vision, or Vision Next? What we really thought about Google and Intel’s AI events

Monique Thomas gifts love with Stem & Story Studio – Jamaica Gleaner

Monique Thomas gifts love with Stem & Story Studio – Jamaica Gleaner

May 31, 2025
Texas vs. Florida: 2025 Women’s College World Series Game 1 | Extended highlights – NCAA.com

Texas vs. Florida: 2025 Women’s College World Series Game 1 | Extended highlights – NCAA.com

May 31, 2025
Proposed URI medical school could fill doctor shortage, boost economy, draft report finds – The Public’s Radio

New URI Medical School: A Solution to Doctor Shortages and Economic Growth!

May 31, 2025
Entertainment: On Your Marks, Get Set, Beer Run! – Urban Milwaukee

Get Ready to Race: The Ultimate Beer Run Experience Awaits!

May 31, 2025
Trump administration ‘MAHA’ health report cited nonexistent studies – Reuters

Trump Administration’s ‘MAHA’ Health Report: Uncovering the Truth Behind Questionable Studies

May 31, 2025
Tributes pour in honoring legacy of legendary politician John Thrasher – Florida Politics

Celebrating the Enduring Legacy of Legendary Politician John Thrasher

May 31, 2025

Revolutionary Harvesting Technology Promises to Slash CAR-T Manufacturing Costs!

May 31, 2025
Bar-B-Cast on Red Sox’s struggles and ugly season: ‘Alex Cora is going nowhere at least this season’ – Yahoo Sports

Bar-B-Cast on Red Sox’s struggles and ugly season: ‘Alex Cora is going nowhere at least this season’ – Yahoo Sports

May 31, 2025

Revolutionary Approach to Assessing the Ecological Impact of Offshore Activities

May 31, 2025
OpenAI’s ‘smartest’ AI model was explicitly told to shut down — and it refused – Live Science

OpenAI’s ‘Smartest’ AI Model Defies Shutdown Command!

May 31, 2025

Categories

Archives

May 2025
MTWTFSS
 1234
567891011
12131415161718
19202122232425
262728293031 
« Apr    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (653)
  • Economy (668)
  • Entertainment (21,574)
  • General (15,251)
  • Health (9,710)
  • Lifestyle (670)
  • News (22,149)
  • People (669)
  • Politics (676)
  • Science (15,889)
  • Sports (21,172)
  • Technology (15,655)
  • World (656)

Recent News

Monique Thomas gifts love with Stem & Story Studio – Jamaica Gleaner

Monique Thomas gifts love with Stem & Story Studio – Jamaica Gleaner

May 31, 2025
Texas vs. Florida: 2025 Women’s College World Series Game 1 | Extended highlights – NCAA.com

Texas vs. Florida: 2025 Women’s College World Series Game 1 | Extended highlights – NCAA.com

May 31, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version