* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Sunday, August 31, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    If You Needed Some BTS Pics Of The Rock’s Tree Trunk Legs To Spice Up Your Day, We Got ‘Em – yahoo.com

    Need a Boost? Check Out These Epic BTS Shots of The Rock’s Tree Trunk Legs!

    Sabrina Carpenter’s pearl-clutching magic? Humor – CNN

    Sabrina Carpenter’s pearl-clutching magic? Humor – CNN

    South Park Is In Trouble And Needs To Change Course, Fast – yahoo.com

    South Park Faces Major Challenges and Must Adapt Fast to Survive

    ‘Netflix House’ entertainment complexes are coming. Here’s when they open – Los Angeles Times

    Exciting News: Netflix’s House Entertainment Complexes Are Opening Soon!

    ‘Big Brother’s’ Rylie Jeffries Finally Speaks Out on Katherine Woodman Controversy – El Paso Inc.

    Big Brother’s Rylie Jeffries Finally Speaks Out on Katherine Woodman Controversy

    ‘SNL’ Parts Ways With Emil Wakim Amid Rumored Season 51 Cast Shakeup – yahoo.com

    SNL Shakes Up Season 51 Cast with Emil Wakim’s Surprising Exit

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Propelling solar technology into a perovskite future – CORDIS

    Propelling solar technology into a perovskite future – CORDIS

    13 Top Technology Trends (2025) – Exploding Topics

    13 Game-Changing Technology Trends to Watch in 2025

    Japan’s legacy LCD and chip technology find new home in India – Nikkei Asia

    How Japan’s Breakthrough LCD and Chip Technologies Are Driving Innovation in India

    ITSWC 2025: Thursday sessions guide – Traffic Technology Today

    Your Ultimate Guide to Thursday Sessions at ITSWC 2025

    MC Machinery Technology Summit Showcases Manufacturing Innovations – Modern Machine Shop

    MC Machinery Technology Summit Unveils Cutting-Edge Manufacturing Innovations

    US-ROK Technology Cooperation Faces Rising Tensions – The National Interest

    Rising Tensions Put US-South Korea Technology Partnership to the Test

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    If You Needed Some BTS Pics Of The Rock’s Tree Trunk Legs To Spice Up Your Day, We Got ‘Em – yahoo.com

    Need a Boost? Check Out These Epic BTS Shots of The Rock’s Tree Trunk Legs!

    Sabrina Carpenter’s pearl-clutching magic? Humor – CNN

    Sabrina Carpenter’s pearl-clutching magic? Humor – CNN

    South Park Is In Trouble And Needs To Change Course, Fast – yahoo.com

    South Park Faces Major Challenges and Must Adapt Fast to Survive

    ‘Netflix House’ entertainment complexes are coming. Here’s when they open – Los Angeles Times

    Exciting News: Netflix’s House Entertainment Complexes Are Opening Soon!

    ‘Big Brother’s’ Rylie Jeffries Finally Speaks Out on Katherine Woodman Controversy – El Paso Inc.

    Big Brother’s Rylie Jeffries Finally Speaks Out on Katherine Woodman Controversy

    ‘SNL’ Parts Ways With Emil Wakim Amid Rumored Season 51 Cast Shakeup – yahoo.com

    SNL Shakes Up Season 51 Cast with Emil Wakim’s Surprising Exit

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Propelling solar technology into a perovskite future – CORDIS

    Propelling solar technology into a perovskite future – CORDIS

    13 Top Technology Trends (2025) – Exploding Topics

    13 Game-Changing Technology Trends to Watch in 2025

    Japan’s legacy LCD and chip technology find new home in India – Nikkei Asia

    How Japan’s Breakthrough LCD and Chip Technologies Are Driving Innovation in India

    ITSWC 2025: Thursday sessions guide – Traffic Technology Today

    Your Ultimate Guide to Thursday Sessions at ITSWC 2025

    MC Machinery Technology Summit Showcases Manufacturing Innovations – Modern Machine Shop

    MC Machinery Technology Summit Unveils Cutting-Edge Manufacturing Innovations

    US-ROK Technology Cooperation Faces Rising Tensions – The National Interest

    Rising Tensions Put US-South Korea Technology Partnership to the Test

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Hugging Face releases a benchmark for testing generative AI on health tasks

April 19, 2024
in Technology
Hugging Face releases a benchmark for testing generative AI on health tasks
Share on FacebookShare on Twitter

Generative AI models are increasingly being brought to healthcare settings — in some cases prematurely, perhaps. Early adopters believe that they’ll unlock increased efficiency while revealing insights that’d otherwise be missed. Critics, meanwhile, point out that these models have flaws and biases that could contribute to worse health outcomes.

But is there a quantitative way to know how helpful, or harmful, a model might be when tasked with things like summarizing patient records or answering health-related questions?

Hugging Face, the AI startup, proposes a solution in a newly released benchmark test called Open Medical-LLM. Created in partnership with researchers at the nonprofit Open Life Science AI and the University of Edinburgh’s Natural Language Processing Group, Open Medical-LLM aims to standardize evaluating the performance of generative AI models on a range of medical-related tasks.

New: Open Medical LLM Leaderboard! 🩺

In basic chatbots, errors are annoyances.
In medical LLMs, errors can have life-threatening consequences 🩸

It’s therefore vital to benchmark/follow advances in medical LLMs before thinking about deployment.

Blog: https://t.co/pddLtkmhsz

— Clémentine Fourrier 🍊 (@clefourrier) April 18, 2024

Open Medical-LLM isn’t a from-scratch benchmark, per se, but rather a stitching-together of existing test sets — MedQA, PubMedQA, MedMCQA and so on — designed to probe models for general medical knowledge and related fields, such as anatomy, pharmacology, genetics and clinical practice. The benchmark contains multiple choice and open-ended questions that require medical reasoning and understanding, drawing from material including U.S. and Indian medical licensing exams and college biology test question banks.

“[Open Medical-LLM] enables researchers and practitioners to identify the strengths and weaknesses of different approaches, drive further advancements in the field and ultimately contribute to better patient care and outcome,” Hugging Face wrote in a blog post.

gen AI healthcare

Image Credits: Hugging Face

Hugging Face is positioning the benchmark as a “robust assessment” of healthcare-bound generative AI models. But some medical experts on social media cautioned against putting too much stock into Open Medical-LLM, lest it lead to ill-informed deployments.

On X, Liam McCoy, a resident physician in neurology at the University of Alberta, pointed out that the gap between the “contrived environment” of medical question-answering and actual clinical practice can be quite large.

It is great progress to see these comparisons head-to-head, but important for us to also remember how big the gap is between the contrived environment of medical question answering and actual clinical practice! Not to mention the idiosyncratic risks these metrics can’t capture.

— Liam McCoy, MD MSc (@LiamGMcCoy) April 18, 2024

Hugging Face research scientist Clémentine Fourrier, who co-authored the blog post, agreed.

“These leaderboards should only be used as a first approximation of which [generative AI model] to explore for a given use case, but then a deeper phase of testing is always needed to examine the model’s limits and relevance in real conditions,” Fourrier replied on X. “Medical [models] should absolutely not be used on their own by patients, but instead should be trained to become support tools for MDs.”

It brings to mind Google’s experience when it tried to bring an AI screening tool for diabetic retinopathy to healthcare systems in Thailand.

Google created a deep learning system that scanned images of the eye, looking for evidence of retinopathy, a leading cause of vision loss. But despite high theoretical accuracy, the tool proved impractical in real-world testing, frustrating both patients and nurses with inconsistent results and a general lack of harmony with on-the-ground practices.

It’s telling that of the 139 AI-related medical devices the U.S. Food and Drug Administration has approved to date, none use generative AI. It’s exceptionally difficult to test how a generative AI tool’s performance in the lab will translate to hospitals and outpatient clinics, and, perhaps more importantly, how the outcomes might trend over time.

That’s not to suggest Open Medical-LLM isn’t useful or informative. The results leaderboard, if nothing else, serves as a reminder of just how poorly models answer basic health questions. But Open Medical-LLM, and no other benchmark for that matter, is a substitute for carefully thought-out real-world testing.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : TechCrunch – https://techcrunch.com/2024/04/18/hugging-face-releases-a-benchmark-for-testing-generative-ai-on-health-tasks/

Tags: Huggingreleasestechnology
Previous Post

Tesla still plans to build 1,800-mile charging corridor for semi trucks despite Biden funding snub

Next Post

Internet users are getting younger; now the UK is weighing up if AI can help protect them

Trump flexes unprecedented executive control over economy – The Hill

Trump Unleashes Unprecedented Executive Power to Transform the Economy

August 31, 2025
If You Needed Some BTS Pics Of The Rock’s Tree Trunk Legs To Spice Up Your Day, We Got ‘Em – yahoo.com

Need a Boost? Check Out These Epic BTS Shots of The Rock’s Tree Trunk Legs!

August 31, 2025
Fecal bacteria at Roosevelt Bridge prompts health advisory in Martin County – Treasure Coast News

Health Advisory Issued in Martin County After Fecal Bacteria Detected at Roosevelt Bridge

August 31, 2025
What does the Cracker Barrel saga tell us? – marketplace.org

What does the Cracker Barrel saga tell us? – marketplace.org

August 31, 2025
FAU tells ‘story’ of Atlantic’s sargassum surge using 40 years of data – EurekAlert!

FAU Reveals the Untold Story Behind Four Decades of Atlantic Sargassum Surge

August 30, 2025
This Week in Science: Brain-Boosting Smells, The Brightest Flash Ever, And More! – ScienceAlert

This Week in Science: Brain-Boosting Scents, The Brightest Flash Ever, and Exciting Discoveries!

August 30, 2025
7 Speed Mike Stewart Science Bodyboard Launch Flex 7 Polypropylene (Polypro) Crescent Tail Morey Mach 7ss – The Michigan Review

Unleash Your Potential with the 7 Speed Mike Stewart Science Bodyboard Featuring Flex 7 Polypropylene and Crescent Tail

August 30, 2025
Cardiologists Struggle With Lifestyle Recommendations – Medscape

Cardiologists Struggle With Lifestyle Recommendations – Medscape

August 30, 2025
Propelling solar technology into a perovskite future – CORDIS

Propelling solar technology into a perovskite future – CORDIS

August 30, 2025
Brian Schottenheimer addresses Micah Parsons trade, says Cowboys can’t win Super Bowl ‘with just one person’ – CBS Sports

Brian Schottenheimer Explains Why the Cowboys Need More Than One Star to Win the Super Bowl

August 30, 2025

Categories

Archives

August 2025
MTWTFSS
 123
45678910
11121314151617
18192021222324
25262728293031
« Jul    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (798)
  • Economy (818)
  • Entertainment (21,697)
  • General (16,765)
  • Health (9,858)
  • Lifestyle (831)
  • News (22,149)
  • People (818)
  • Politics (825)
  • Science (16,027)
  • Sports (21,315)
  • Technology (15,797)
  • World (798)

Recent News

Trump flexes unprecedented executive control over economy – The Hill

Trump Unleashes Unprecedented Executive Power to Transform the Economy

August 31, 2025
If You Needed Some BTS Pics Of The Rock’s Tree Trunk Legs To Spice Up Your Day, We Got ‘Em – yahoo.com

Need a Boost? Check Out These Epic BTS Shots of The Rock’s Tree Trunk Legs!

August 31, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version