* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Thursday, June 5, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    ‘Sinners,’ starring Michael B. Jordan, is now streaming on Prime Video – About Amazon

    Experience the Thrills of ‘Sinners’ Starring Michael B. Jordan – Now Streaming on Prime Video!

    California Mid-State Fair announces entertainment lineups for Frontier and Mission stages – KSBY News

    Exciting Entertainment Lineup Unveiled for California Mid-State Fair’s Frontier and Mission Stages!

    Spotify & OpenAI: Will Gen AI ‘Hollow Out’ Entertainment? – Technology Magazine

    Will Generative AI Transform the Future of Entertainment

    Author Richard Russo Will Speak At Sandwich Town Hall – CapeNews.net

    Join Us for an Inspiring Evening with Author Richard Russo at Sandwich Town Hall!

    Entertainment Partners Acquires CASHét, Digital Payments Vendor for Productions – Variety

    Entertainment Partners Acquires CASHét, Digital Payments Vendor for Productions – Variety

    Salem’s Harborwalk Garden Cultivates Community with Entertainment, Food, and Events – 105.7 WROR

    Discover the Vibrant Community Spirit at Salem’s Harborwalk Garden: A Hub for Entertainment, Food, and Fun!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Domo to Participate in the D.A. Davidson Technology Summit – Business Wire

    Domo Set to Shine at the D.A. Davidson Technology Summit!

    When fiction becomes fact: 3 pieces of modern technology inspired by Star Trek – Redshirts Always Die

    From Screen to Reality: 3 Modern Technologies Inspired by Star Trek

    The Isfahan Center for Nuclear Technology – UCF, ZPP, and IRR10 – Alma Research and Education Center

    Exploring the Isfahan Center for Nuclear Technology: Innovations and Insights from UCF, ZPP, and IRR10

    Inside the tedious effort to tally AI’s energy appetite – MIT Technology Review

    Inside the tedious effort to tally AI’s energy appetite – MIT Technology Review

    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    AI for lawyers: Win back your time using technology – nationaljurist.com

    Reclaim Your Time: How AI is Transforming the Legal Profession

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    ‘Sinners,’ starring Michael B. Jordan, is now streaming on Prime Video – About Amazon

    Experience the Thrills of ‘Sinners’ Starring Michael B. Jordan – Now Streaming on Prime Video!

    California Mid-State Fair announces entertainment lineups for Frontier and Mission stages – KSBY News

    Exciting Entertainment Lineup Unveiled for California Mid-State Fair’s Frontier and Mission Stages!

    Spotify & OpenAI: Will Gen AI ‘Hollow Out’ Entertainment? – Technology Magazine

    Will Generative AI Transform the Future of Entertainment

    Author Richard Russo Will Speak At Sandwich Town Hall – CapeNews.net

    Join Us for an Inspiring Evening with Author Richard Russo at Sandwich Town Hall!

    Entertainment Partners Acquires CASHét, Digital Payments Vendor for Productions – Variety

    Entertainment Partners Acquires CASHét, Digital Payments Vendor for Productions – Variety

    Salem’s Harborwalk Garden Cultivates Community with Entertainment, Food, and Events – 105.7 WROR

    Discover the Vibrant Community Spirit at Salem’s Harborwalk Garden: A Hub for Entertainment, Food, and Fun!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Domo to Participate in the D.A. Davidson Technology Summit – Business Wire

    Domo Set to Shine at the D.A. Davidson Technology Summit!

    When fiction becomes fact: 3 pieces of modern technology inspired by Star Trek – Redshirts Always Die

    From Screen to Reality: 3 Modern Technologies Inspired by Star Trek

    The Isfahan Center for Nuclear Technology – UCF, ZPP, and IRR10 – Alma Research and Education Center

    Exploring the Isfahan Center for Nuclear Technology: Innovations and Insights from UCF, ZPP, and IRR10

    Inside the tedious effort to tally AI’s energy appetite – MIT Technology Review

    Inside the tedious effort to tally AI’s energy appetite – MIT Technology Review

    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    Finland Set to Lead EU Quantum Technology Defense Project – IoT World Today

    AI for lawyers: Win back your time using technology – nationaljurist.com

    Reclaim Your Time: How AI is Transforming the Legal Profession

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Boffins find AI stumbles when quizzed on the tough stuff

October 30, 2023
in Technology
Boffins find AI stumbles when quizzed on the tough stuff
Share on FacebookShare on Twitter

AI models can manage well enough when prompted with text or images, and may even solve complex problems when not making terrible errors.

OpenAI, for example, has said that its GPT-4 model managed to score 700 out of 800 on the SAT math exam. Not all such claims have borne out, however: A paper released in June that said GPT-4 could get a computer science degree at MIT was subsequently withdrawn.

So to better assess how large language models – which interpret text input – and large multimodal models – which interpret text, images and perhaps other forms of input – actually handle problem solving, a group of ten researchers from the University of California, Los Angeles, the University of Washington, and Microsoft Research have devised a testing benchmark called MathVista that focuses on visually-oriented challenges.

“The ability of these foundation models to perform mathematical reasoning in visual contexts has not been systematically examined,” say the authors – Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao, in a preprint paper [PDF].

It is thus essential, they say, to develop a new benchmark to help the development of mathematical reasoning with a visual component and to evaluate how various models compare at reasoning tasks.

Being able to show that one’s AI model can correctly solve visual problems may prove helpful in determining whether it’s wise to, say, trust software to drive a car without stopping atop an accident victim.

MathVista incorporates 6,141 examples that were developed from 28 multimodal datasets and from 3 new datasets called IQTest, FunctionQA, and PaperQA. It covers various forms of reasoning (algebraic, arithmetic, geometric, logical, numeric, scientific, and statistical), with a focus on figure question answering, geometry problem solving, math word problems, textbook questions, and visual questions.

Screenshot of MathVista challenge question

Screenshot of MathVista challenge question – Click to enlarge

The researchers tested a dozen foundation models: three LLMs ChatGPT, GPT-4, and Claude-2), two proprietary LMMs (GPT4V and Bard), and seven open-source LMMs. They also considered human answers, provided via Amazon Mechanical Turkers with at least a high school degree, and random responses.

AWS CEO talks up AI to focus minds of Wall Street types

Clippy-like AI at forefront of Windows update previews

Bug bounty hunters load up to stalk AI and fancy bagging big bucks

How prompt injection attacks hijack today’s top-end AI – and it’s tough to fix

The good news for AI practitioners is that the LLMs and LMMs all did better than random chance, which isn’t all that surprising considering that many of the questions were multiple choice rather than yes or no.

In fact, the top performer, OpenAI’s GPT-4V, managed to surpass human performance in specific areas – questions involving algebraic reasoning and complex visual challenges involving tables and function plots.

We note that Microsoft, whose researchers contributed to this project, has a substantial stake in OpenAI.

The less good news is that even GPT-4V only managed to get 49.9 percent of the questions correct. That’s adequate if the goal is to best multimodal Bard, which managed an accuracy percentage of 34.8 percent.

But it’s still shy of the Amazon Mechanical Turk workers who were put to the test and managed a score of 60.3 percent. As the researchers observe in their paper, “a 10.4 percent gap in overall accuracy remains when compared to the human baseline, leaving plenty of room for model improvement.” ®

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : The Register – https://go.theregister.com/feed/www.theregister.com/2023/10/29/ai_math_quiz/

Tags: Boffinsstumblestechnology
Previous Post

Tenfold electric vehicles on 2030 roads could be a shock to the system

Next Post

Somehow, The Black Phone Will Ring Up a Sequel

The Ecology of Visitation – National Park Service (.gov)

Exploring the Impact of Visitor Interactions on Natural Ecosystems

June 5, 2025
Students play with science at Camp Invention at School of Innovation – News-Herald

Unleashing Creativity: Students Dive into Science at Camp Invention!

June 5, 2025
Are cold plunges good for you? Here’s what the science says. – The Washington Post

Unlocking the Benefits of Cold Plunges: What Science Reveals!

June 5, 2025
Casino magnate Alvin Chau’s wife Heidi Chan embraces Buddhist lifestyle after husband’s incarceration – VnExpress International

Casino magnate Alvin Chau’s wife Heidi Chan embraces Buddhist lifestyle after husband’s incarceration – VnExpress International

June 5, 2025
How each of the 32 teams qualified for the 2025 Club World Cup – The New York Times

Unveiling the Path: How All 32 Teams Earned Their Spot in the 2025 Club World Cup

June 5, 2025
Service side of the U.S. economy contracts for first time in almost a year due to trade fights – MarketWatch

U.S. Service Sector Shrinks for the First Time in Nearly a Year Amid Trade Tensions

June 5, 2025
‘Sinners,’ starring Michael B. Jordan, is now streaming on Prime Video – About Amazon

Experience the Thrills of ‘Sinners’ Starring Michael B. Jordan – Now Streaming on Prime Video!

June 5, 2025
Sutter Health investing $23 million in primary and behavioral care – Healthcare Finance News

Sutter Health investing $23 million in primary and behavioral care – Healthcare Finance News

June 5, 2025
Gov. Shapiro sues USDA over canceled $13M food purchasing program – LancasterOnline

Gov. Shapiro sues USDA over canceled $13M food purchasing program – LancasterOnline

June 5, 2025
Domo to Participate in the D.A. Davidson Technology Summit – Business Wire

Domo Set to Shine at the D.A. Davidson Technology Summit!

June 5, 2025

Categories

Archives

June 2025
MTWTFSS
 1
2345678
9101112131415
16171819202122
23242526272829
30 
« May    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (667)
  • Economy (681)
  • Entertainment (21,587)
  • General (15,261)
  • Health (9,723)
  • Lifestyle (684)
  • News (22,149)
  • People (682)
  • Politics (690)
  • Science (15,901)
  • Sports (21,185)
  • Technology (15,667)
  • World (668)

Recent News

The Ecology of Visitation – National Park Service (.gov)

Exploring the Impact of Visitor Interactions on Natural Ecosystems

June 5, 2025
Students play with science at Camp Invention at School of Innovation – News-Herald

Unleashing Creativity: Students Dive into Science at Camp Invention!

June 5, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version