* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Tuesday, September 9, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Monumental Sports & Entertainment Sets Corporate Direction at Nasdaq – PR Newswire

    Monumental Sports & Entertainment Reveals Bold New Corporate Vision at Nasdaq

    The Secret to What Made ‘CarJack’ Work on As the World Turns – yahoo.com

    The Surprising Secret Behind ‘CarJack’s’ Success on As the World Turns

    Victor Garber on his viral “And Just Like That” toilet scene: ‘I was delighted to be doing something ridiculous’ (exclusive) – yahoo.com

    Victor Garber on his viral “And Just Like That” toilet scene: ‘I was delighted to be doing something ridiculous’ (exclusive) – yahoo.com

    Pendulum Announce Homecoming 2026 Australian Tour – yahoo.com

    Pendulum Announces Thrilling Homecoming Tour Across Australia in 2026

    ITV Studios Launches New Entertainment Label – Global Bulletin – IMDb

    ITV Studios Unveils Exciting New Entertainment Label

    TS Entertainment bringing Malibu Jack’s to former Owensboro mall – Lane Report

    TS Entertainment Launches Malibu Jack’s at Former Owensboro Mall Location

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Tri-Counties Bank marks 50 years of growth with focus on technology and personal service – thebusinessjournal.com

    Tri-Counties Bank Celebrates 50 Years of Growth Driven by Technology and Personal Service

    AI will reshape internet, create jobs in West Virginia says High Technology Foundation’s Estep – WV News

    How AI Is Set to Transform the Internet and Boost Job Growth in West Virginia

    Industry partner provides Ferris State Plastics Engineering Technology students with state-of-the-art equipment to gain in-demand skills – Ferris State University

    Industry Partner Equips Ferris State Plastics Engineering Students with Cutting-Edge Technology to Boost In-Demand Skills

    Health Technology Ecosystem – Centers for Medicare & Medicaid Services | CMS (.gov)

    Discover the Future of Health Technology: Innovations Revolutionizing Patient Care

    Coherent Joins LLNL’s STARFIRE Diode Technology Working Group to Advance Inertial Fusion Energy – GlobeNewswire

    Coherent Partners with LLNL’s STARFIRE Team to Drive Breakthroughs in Inertial Fusion Energy

    Gene Associated With Deadly Heart Disease in Golden Retrievers Identified – Technology Networks

    Breakthrough Discovery Uncovers Gene Behind Deadly Heart Disease in Golden Retrievers

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Monumental Sports & Entertainment Sets Corporate Direction at Nasdaq – PR Newswire

    Monumental Sports & Entertainment Reveals Bold New Corporate Vision at Nasdaq

    The Secret to What Made ‘CarJack’ Work on As the World Turns – yahoo.com

    The Surprising Secret Behind ‘CarJack’s’ Success on As the World Turns

    Victor Garber on his viral “And Just Like That” toilet scene: ‘I was delighted to be doing something ridiculous’ (exclusive) – yahoo.com

    Victor Garber on his viral “And Just Like That” toilet scene: ‘I was delighted to be doing something ridiculous’ (exclusive) – yahoo.com

    Pendulum Announce Homecoming 2026 Australian Tour – yahoo.com

    Pendulum Announces Thrilling Homecoming Tour Across Australia in 2026

    ITV Studios Launches New Entertainment Label – Global Bulletin – IMDb

    ITV Studios Unveils Exciting New Entertainment Label

    TS Entertainment bringing Malibu Jack’s to former Owensboro mall – Lane Report

    TS Entertainment Launches Malibu Jack’s at Former Owensboro Mall Location

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Tri-Counties Bank marks 50 years of growth with focus on technology and personal service – thebusinessjournal.com

    Tri-Counties Bank Celebrates 50 Years of Growth Driven by Technology and Personal Service

    AI will reshape internet, create jobs in West Virginia says High Technology Foundation’s Estep – WV News

    How AI Is Set to Transform the Internet and Boost Job Growth in West Virginia

    Industry partner provides Ferris State Plastics Engineering Technology students with state-of-the-art equipment to gain in-demand skills – Ferris State University

    Industry Partner Equips Ferris State Plastics Engineering Students with Cutting-Edge Technology to Boost In-Demand Skills

    Health Technology Ecosystem – Centers for Medicare & Medicaid Services | CMS (.gov)

    Discover the Future of Health Technology: Innovations Revolutionizing Patient Care

    Coherent Joins LLNL’s STARFIRE Diode Technology Working Group to Advance Inertial Fusion Energy – GlobeNewswire

    Coherent Partners with LLNL’s STARFIRE Team to Drive Breakthroughs in Inertial Fusion Energy

    Gene Associated With Deadly Heart Disease in Golden Retrievers Identified – Technology Networks

    Breakthrough Discovery Uncovers Gene Behind Deadly Heart Disease in Golden Retrievers

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Boffins find AI stumbles when quizzed on the tough stuff

October 30, 2023
in Technology
Boffins find AI stumbles when quizzed on the tough stuff
Share on FacebookShare on Twitter

AI models can manage well enough when prompted with text or images, and may even solve complex problems when not making terrible errors.

OpenAI, for example, has said that its GPT-4 model managed to score 700 out of 800 on the SAT math exam. Not all such claims have borne out, however: A paper released in June that said GPT-4 could get a computer science degree at MIT was subsequently withdrawn.

So to better assess how large language models – which interpret text input – and large multimodal models – which interpret text, images and perhaps other forms of input – actually handle problem solving, a group of ten researchers from the University of California, Los Angeles, the University of Washington, and Microsoft Research have devised a testing benchmark called MathVista that focuses on visually-oriented challenges.

“The ability of these foundation models to perform mathematical reasoning in visual contexts has not been systematically examined,” say the authors – Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao, in a preprint paper [PDF].

It is thus essential, they say, to develop a new benchmark to help the development of mathematical reasoning with a visual component and to evaluate how various models compare at reasoning tasks.

Being able to show that one’s AI model can correctly solve visual problems may prove helpful in determining whether it’s wise to, say, trust software to drive a car without stopping atop an accident victim.

MathVista incorporates 6,141 examples that were developed from 28 multimodal datasets and from 3 new datasets called IQTest, FunctionQA, and PaperQA. It covers various forms of reasoning (algebraic, arithmetic, geometric, logical, numeric, scientific, and statistical), with a focus on figure question answering, geometry problem solving, math word problems, textbook questions, and visual questions.

Screenshot of MathVista challenge question

Screenshot of MathVista challenge question – Click to enlarge

The researchers tested a dozen foundation models: three LLMs ChatGPT, GPT-4, and Claude-2), two proprietary LMMs (GPT4V and Bard), and seven open-source LMMs. They also considered human answers, provided via Amazon Mechanical Turkers with at least a high school degree, and random responses.

AWS CEO talks up AI to focus minds of Wall Street types

Clippy-like AI at forefront of Windows update previews

Bug bounty hunters load up to stalk AI and fancy bagging big bucks

How prompt injection attacks hijack today’s top-end AI – and it’s tough to fix

The good news for AI practitioners is that the LLMs and LMMs all did better than random chance, which isn’t all that surprising considering that many of the questions were multiple choice rather than yes or no.

In fact, the top performer, OpenAI’s GPT-4V, managed to surpass human performance in specific areas – questions involving algebraic reasoning and complex visual challenges involving tables and function plots.

We note that Microsoft, whose researchers contributed to this project, has a substantial stake in OpenAI.

The less good news is that even GPT-4V only managed to get 49.9 percent of the questions correct. That’s adequate if the goal is to best multimodal Bard, which managed an accuracy percentage of 34.8 percent.

But it’s still shy of the Amazon Mechanical Turk workers who were put to the test and managed a score of 60.3 percent. As the researchers observe in their paper, “a 10.4 percent gap in overall accuracy remains when compared to the human baseline, leaving plenty of room for model improvement.” ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2023/10/29/ai_math_quiz/

Tags: Boffinsstumblestechnology
Previous Post

Tenfold electric vehicles on 2030 roads could be a shock to the system

Next Post

Somehow, The Black Phone Will Ring Up a Sequel

Pope Leo XIV: Caring for Creation is our vocation – Vatican News

Pope Leo XIV: Caring for Creation is our vocation – Vatican News

September 9, 2025
Ocean Sciences Meeting will convene in Glasgow Scotland, February 2026 – EurekAlert!

Get Ready for the Ocean Sciences Meeting Making Waves in Glasgow This February 2026!

September 9, 2025
The Christian Science Monitor | Grand Juries Usually Approve Indictments. In LA and DC, They’re Pushing Back. – Loyola Marymount University

Grand Juries Typically Approve Indictments-But in LA and DC, They’re Starting to Push Back

September 9, 2025
Lifestyle Communities Finalizes Ocean Grove Land Sale – TipRanks

Lifestyle Communities Finalizes Ocean Grove Land Sale – TipRanks

September 9, 2025
Tri-Counties Bank marks 50 years of growth with focus on technology and personal service – thebusinessjournal.com

Tri-Counties Bank Celebrates 50 Years of Growth Driven by Technology and Personal Service

September 9, 2025
Bears inform fans of plans to leave Soldier Field, finalize stadium in Arlington Heights for Super Bowl bid – CBS Sports

Bears inform fans of plans to leave Soldier Field, finalize stadium in Arlington Heights for Super Bowl bid – CBS Sports

September 9, 2025
Ten standout moments from World Cup pool stage – BBC

Ten Unforgettable Moments That Defined the World Cup Pool Stage

September 8, 2025
The supply and demand seesaw is sending mixed signals about the economy – MSN

The Supply and Demand Tug-of-War Is Sending Confusing Signals About the Economy

September 8, 2025
Why this San Antonio shopping center could be an entertainment hub – MySA

How This San Antonio Shopping Center Is Set to Become the Ultimate Entertainment Destination

September 8, 2025
Wearable health trackers reveal how accurate your smartwatch really is – News-Medical

Wearable health trackers reveal how accurate your smartwatch really is – News-Medical

September 8, 2025

Categories

Archives

September 2025
MTWTFSS
1234567
891011121314
15161718192021
22232425262728
2930 
« Aug    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (814)
  • Economy (831)
  • Entertainment (21,708)
  • General (16,932)
  • Health (9,873)
  • Lifestyle (845)
  • News (22,149)
  • People (834)
  • Politics (838)
  • Science (16,041)
  • Sports (21,331)
  • Technology (15,812)
  • World (813)

Recent News

Pope Leo XIV: Caring for Creation is our vocation – Vatican News

Pope Leo XIV: Caring for Creation is our vocation – Vatican News

September 9, 2025
Ocean Sciences Meeting will convene in Glasgow Scotland, February 2026 – EurekAlert!

Get Ready for the Ocean Sciences Meeting Making Waves in Glasgow This February 2026!

September 9, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version