* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Monday, September 29, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Jussie Smollett Claims He Was ‘Disrespected’ on the ‘Special Forces’ Season Premiere – Yahoo

    Jussie Smollett Opens Up About Feeling ‘Disrespected’ During the ‘Special Forces’ Season Premiere

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    Cardi B Adds More Dates to Little Miss Drama Tour: ‘Y’all Making Me Work’ – Yahoo

    Cardi B Extends Little Miss Drama Tour: “Y’all Making Me Work

    ‘Today’: Sheinelle Jones Thanks Katie Couric for Support After Husband’s Death – CBS 19 News

    Sheinelle Jones Expresses Heartfelt Thanks to Katie Couric for Support After Husband’s Passing

    Sate your hunger at DBA’s Taste of Downtown – Bakersfield.com

    Indulge Your Cravings at DBA’s Taste of Downtown!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

    Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

    Why I gave the world wide web away for free | Tim Berners-Lee – The Guardian

    Why I Decided to Make the World Wide Web Free for Everyone | Tim Berners-Lee

    From shale to steam: Fossil fuel technology boosts clean geothermal energy – Washington Examiner

    From Shale to Steam: How Fossil Fuel Technology is Powering a Clean Geothermal Energy Revolution

    How Sustainable Technology is Shaping a Greener Future – Technology Magazine

    How Sustainable Technology is Driving the Revolution Toward a Greener Future

    Aurora police hope to add facial recognition technology to crime-fighting tools – CBS News

    Aurora Police Aim to Boost Crime-Fighting with New Facial Recognition Technology

    Autonomous Solutions shows off cutting-edge technology for the public – Cache Valley Daily

    Autonomous Solutions Unveils Cutting-Edge Technology for the Public

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Jussie Smollett Claims He Was ‘Disrespected’ on the ‘Special Forces’ Season Premiere – Yahoo

    Jussie Smollett Opens Up About Feeling ‘Disrespected’ During the ‘Special Forces’ Season Premiere

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    Cardi B Adds More Dates to Little Miss Drama Tour: ‘Y’all Making Me Work’ – Yahoo

    Cardi B Extends Little Miss Drama Tour: “Y’all Making Me Work

    ‘Today’: Sheinelle Jones Thanks Katie Couric for Support After Husband’s Death – CBS 19 News

    Sheinelle Jones Expresses Heartfelt Thanks to Katie Couric for Support After Husband’s Passing

    Sate your hunger at DBA’s Taste of Downtown – Bakersfield.com

    Indulge Your Cravings at DBA’s Taste of Downtown!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

    Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

    Why I gave the world wide web away for free | Tim Berners-Lee – The Guardian

    Why I Decided to Make the World Wide Web Free for Everyone | Tim Berners-Lee

    From shale to steam: Fossil fuel technology boosts clean geothermal energy – Washington Examiner

    From Shale to Steam: How Fossil Fuel Technology is Powering a Clean Geothermal Energy Revolution

    How Sustainable Technology is Shaping a Greener Future – Technology Magazine

    How Sustainable Technology is Driving the Revolution Toward a Greener Future

    Aurora police hope to add facial recognition technology to crime-fighting tools – CBS News

    Aurora Police Aim to Boost Crime-Fighting with New Facial Recognition Technology

    Autonomous Solutions shows off cutting-edge technology for the public – Cache Valley Daily

    Autonomous Solutions Unveils Cutting-Edge Technology for the Public

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Boffins find AI stumbles when quizzed on the tough stuff

October 30, 2023
in Technology
Boffins find AI stumbles when quizzed on the tough stuff
Share on FacebookShare on Twitter

AI models can manage well enough when prompted with text or images, and may even solve complex problems when not making terrible errors.

OpenAI, for example, has said that its GPT-4 model managed to score 700 out of 800 on the SAT math exam. Not all such claims have borne out, however: A paper released in June that said GPT-4 could get a computer science degree at MIT was subsequently withdrawn.

So to better assess how large language models – which interpret text input – and large multimodal models – which interpret text, images and perhaps other forms of input – actually handle problem solving, a group of ten researchers from the University of California, Los Angeles, the University of Washington, and Microsoft Research have devised a testing benchmark called MathVista that focuses on visually-oriented challenges.

“The ability of these foundation models to perform mathematical reasoning in visual contexts has not been systematically examined,” say the authors – Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao, in a preprint paper [PDF].

It is thus essential, they say, to develop a new benchmark to help the development of mathematical reasoning with a visual component and to evaluate how various models compare at reasoning tasks.

Being able to show that one’s AI model can correctly solve visual problems may prove helpful in determining whether it’s wise to, say, trust software to drive a car without stopping atop an accident victim.

MathVista incorporates 6,141 examples that were developed from 28 multimodal datasets and from 3 new datasets called IQTest, FunctionQA, and PaperQA. It covers various forms of reasoning (algebraic, arithmetic, geometric, logical, numeric, scientific, and statistical), with a focus on figure question answering, geometry problem solving, math word problems, textbook questions, and visual questions.

Screenshot of MathVista challenge question

Screenshot of MathVista challenge question – Click to enlarge

The researchers tested a dozen foundation models: three LLMs ChatGPT, GPT-4, and Claude-2), two proprietary LMMs (GPT4V and Bard), and seven open-source LMMs. They also considered human answers, provided via Amazon Mechanical Turkers with at least a high school degree, and random responses.

AWS CEO talks up AI to focus minds of Wall Street types

Clippy-like AI at forefront of Windows update previews

Bug bounty hunters load up to stalk AI and fancy bagging big bucks

How prompt injection attacks hijack today’s top-end AI – and it’s tough to fix

The good news for AI practitioners is that the LLMs and LMMs all did better than random chance, which isn’t all that surprising considering that many of the questions were multiple choice rather than yes or no.

In fact, the top performer, OpenAI’s GPT-4V, managed to surpass human performance in specific areas – questions involving algebraic reasoning and complex visual challenges involving tables and function plots.

We note that Microsoft, whose researchers contributed to this project, has a substantial stake in OpenAI.

The less good news is that even GPT-4V only managed to get 49.9 percent of the questions correct. That’s adequate if the goal is to best multimodal Bard, which managed an accuracy percentage of 34.8 percent.

But it’s still shy of the Amazon Mechanical Turk workers who were put to the test and managed a score of 60.3 percent. As the researchers observe in their paper, “a 10.4 percent gap in overall accuracy remains when compared to the human baseline, leaving plenty of room for model improvement.” ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2023/10/29/ai_math_quiz/

Tags: Boffinsstumblestechnology
Previous Post

Tenfold electric vehicles on 2030 roads could be a shock to the system

Next Post

Somehow, The Black Phone Will Ring Up a Sequel

WEEKEND SCENE: Wolves, salmon, stormwater, more at 2025 West Seattle Ecology Fair, report #1 – West Seattle Blog…

Discover Wolves, Salmon, Stormwater, and More at the 2025 West Seattle Ecology Fair!

September 29, 2025
The Real Threat to Faith Isn’t Science – The Gospel Coalition

The Real Threat to Faith Isn’t Science – The Gospel Coalition

September 29, 2025
Trump Distorts Facts on Autism, Tylenol and Vaccines, Scientists Say – hepmag.com

Scientists Disprove Claims Connecting Autism to Tylenol and Vaccines

September 29, 2025
Our Favorite Store-Bought Coleslaw Can Be Found At Many Grocery Stores – Yahoo

Our Favorite Store-Bought Coleslaw Can Be Found At Many Grocery Stores – Yahoo

September 29, 2025
Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

September 29, 2025
Arrowheadlines: NASCAR driver sits in on Chiefs quarterback meeting – Yahoo Sports

NASCAR Driver Shocks Fans by Joining Chiefs Quarterback Meeting in Unexpected Crossover

September 29, 2025
California has the longest light-rail line in the world. It doesn’t quite work – sfchronicle.com

California Boasts the World’s Longest Light-Rail Line – But It Still Faces Major Challenges

September 29, 2025
Tourism to Shenandoah National Park contributes $175 million to local economy – InsideNoVa.com

Tourism to Shenandoah National Park Injects $175 Million into Local Economy

September 29, 2025
Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

September 29, 2025
Governor calls on Congress to extend tax credits to protect affordable health care coverage – NKyTribune

Governor Calls on Congress to Extend Tax Credits and Protect Affordable Health Care Coverage

September 29, 2025

Categories

Archives

September 2025
M T W T F S S
1234567
891011121314
15161718192021
22232425262728
2930  
« Aug    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (842)
  • Economy (863)
  • Entertainment (21,737)
  • General (17,309)
  • Health (9,906)
  • Lifestyle (876)
  • News (22,149)
  • People (865)
  • Politics (873)
  • Science (16,073)
  • Sports (21,363)
  • Technology (15,846)
  • World (845)

Recent News

WEEKEND SCENE: Wolves, salmon, stormwater, more at 2025 West Seattle Ecology Fair, report #1 – West Seattle Blog…

Discover Wolves, Salmon, Stormwater, and More at the 2025 West Seattle Ecology Fair!

September 29, 2025
The Real Threat to Faith Isn’t Science – The Gospel Coalition

The Real Threat to Faith Isn’t Science – The Gospel Coalition

September 29, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version