* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Monday, May 19, 2025
Earth-News
  • Home
  • Business
  • Entertainment

    Entertainment Spotlight: ‘Georgie & Mandy’s First Marriage’ – Atlanta News First

    ‘Final Destination: Bloodlines’ tops box office while The Weeknd’s movie falters – Yakima Herald-Republic

    Final Destination: Bloodlines Dominates the Box Office as The Weeknd’s Film Struggles

    Country Music Legend Bids Heartfelt Farewell: ‘Y’all Gonna Make Me Tear Up!

    We won’t get a Game of Thrones show this year: A Knight of the Seven Kingdoms shifts to early 2026 – Entertainment Weekly

    Game of Thrones Fans Will Have to Wait: A Knight of the Seven Kingdoms Delayed Until 2026!

    Nile Entertainment Secures African Rights for Thrilling Action Film ‘Son of the Soil

    Florida Highwaymen movie ‘Legends of the Highway’ based on original 26 Black artists – Treasure Coast News

    Unveiling ‘Legends of the Highway’: A Captivating Film Celebrating the Legacy of Florida’s Original 26 Black Artists

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Nvidia plans to sell tech to speed AI chip communication – The Indian Express

    Nvidia Unveils Game-Changing Technology to Accelerate AI Chip Communication!

    Murfreesboro LPR technology helps catch suspect in Henry County homicide case – WKRN News 2

    Murfreesboro LPR technology helps catch suspect in Henry County homicide case – WKRN News 2

    How will BCI technology change the lives of people with disabilities? – news.cgtn.com

    Transforming Lives: The Impact of BCI Technology on People with Disabilities

    Super Speeders are deadly. This technology can slow them down. – Popular Science

    Revolutionary Technology: Taming the Threat of Super Speeders!

    Celebrating Success: Highlights from the Collaborative College for Technology & Leadership Graduation Ceremony

    Philly police unveil strategy to crack down on car meetups utilizing technology – NBC10 Philadelphia

    Philly Police Launch High-Tech Strategy to Tackle Car Meetups!

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment

    Entertainment Spotlight: ‘Georgie & Mandy’s First Marriage’ – Atlanta News First

    ‘Final Destination: Bloodlines’ tops box office while The Weeknd’s movie falters – Yakima Herald-Republic

    Final Destination: Bloodlines Dominates the Box Office as The Weeknd’s Film Struggles

    Country Music Legend Bids Heartfelt Farewell: ‘Y’all Gonna Make Me Tear Up!

    We won’t get a Game of Thrones show this year: A Knight of the Seven Kingdoms shifts to early 2026 – Entertainment Weekly

    Game of Thrones Fans Will Have to Wait: A Knight of the Seven Kingdoms Delayed Until 2026!

    Nile Entertainment Secures African Rights for Thrilling Action Film ‘Son of the Soil

    Florida Highwaymen movie ‘Legends of the Highway’ based on original 26 Black artists – Treasure Coast News

    Unveiling ‘Legends of the Highway’: A Captivating Film Celebrating the Legacy of Florida’s Original 26 Black Artists

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Nvidia plans to sell tech to speed AI chip communication – The Indian Express

    Nvidia Unveils Game-Changing Technology to Accelerate AI Chip Communication!

    Murfreesboro LPR technology helps catch suspect in Henry County homicide case – WKRN News 2

    Murfreesboro LPR technology helps catch suspect in Henry County homicide case – WKRN News 2

    How will BCI technology change the lives of people with disabilities? – news.cgtn.com

    Transforming Lives: The Impact of BCI Technology on People with Disabilities

    Super Speeders are deadly. This technology can slow them down. – Popular Science

    Revolutionary Technology: Taming the Threat of Super Speeders!

    Celebrating Success: Highlights from the Collaborative College for Technology & Leadership Graduation Ceremony

    Philly police unveil strategy to crack down on car meetups utilizing technology – NBC10 Philadelphia

    Philly Police Launch High-Tech Strategy to Tackle Car Meetups!

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Hugging Face releases a benchmark for testing generative AI on health tasks

April 19, 2024
in Technology
Hugging Face releases a benchmark for testing generative AI on health tasks
Share on FacebookShare on Twitter

Generative AI models are increasingly being brought to healthcare settings — in some cases prematurely, perhaps. Early adopters believe that they’ll unlock increased efficiency while revealing insights that’d otherwise be missed. Critics, meanwhile, point out that these models have flaws and biases that could contribute to worse health outcomes.

But is there a quantitative way to know how helpful, or harmful, a model might be when tasked with things like summarizing patient records or answering health-related questions?

Hugging Face, the AI startup, proposes a solution in a newly released benchmark test called Open Medical-LLM. Created in partnership with researchers at the nonprofit Open Life Science AI and the University of Edinburgh’s Natural Language Processing Group, Open Medical-LLM aims to standardize evaluating the performance of generative AI models on a range of medical-related tasks.

New: Open Medical LLM Leaderboard! 🩺

In basic chatbots, errors are annoyances.
In medical LLMs, errors can have life-threatening consequences 🩸

It’s therefore vital to benchmark/follow advances in medical LLMs before thinking about deployment.

Blog: https://t.co/pddLtkmhsz

— Clémentine Fourrier 🍊 (@clefourrier) April 18, 2024

Open Medical-LLM isn’t a from-scratch benchmark, per se, but rather a stitching-together of existing test sets — MedQA, PubMedQA, MedMCQA and so on — designed to probe models for general medical knowledge and related fields, such as anatomy, pharmacology, genetics and clinical practice. The benchmark contains multiple choice and open-ended questions that require medical reasoning and understanding, drawing from material including U.S. and Indian medical licensing exams and college biology test question banks.

“[Open Medical-LLM] enables researchers and practitioners to identify the strengths and weaknesses of different approaches, drive further advancements in the field and ultimately contribute to better patient care and outcome,” Hugging Face wrote in a blog post.

gen AI healthcare

Image Credits: Hugging Face

Hugging Face is positioning the benchmark as a “robust assessment” of healthcare-bound generative AI models. But some medical experts on social media cautioned against putting too much stock into Open Medical-LLM, lest it lead to ill-informed deployments.

On X, Liam McCoy, a resident physician in neurology at the University of Alberta, pointed out that the gap between the “contrived environment” of medical question-answering and actual clinical practice can be quite large.

It is great progress to see these comparisons head-to-head, but important for us to also remember how big the gap is between the contrived environment of medical question answering and actual clinical practice! Not to mention the idiosyncratic risks these metrics can’t capture.

— Liam McCoy, MD MSc (@LiamGMcCoy) April 18, 2024

Hugging Face research scientist Clémentine Fourrier, who co-authored the blog post, agreed.

“These leaderboards should only be used as a first approximation of which [generative AI model] to explore for a given use case, but then a deeper phase of testing is always needed to examine the model’s limits and relevance in real conditions,” Fourrier replied on X. “Medical [models] should absolutely not be used on their own by patients, but instead should be trained to become support tools for MDs.”

It brings to mind Google’s experience when it tried to bring an AI screening tool for diabetic retinopathy to healthcare systems in Thailand.

Google created a deep learning system that scanned images of the eye, looking for evidence of retinopathy, a leading cause of vision loss. But despite high theoretical accuracy, the tool proved impractical in real-world testing, frustrating both patients and nurses with inconsistent results and a general lack of harmony with on-the-ground practices.

It’s telling that of the 139 AI-related medical devices the U.S. Food and Drug Administration has approved to date, none use generative AI. It’s exceptionally difficult to test how a generative AI tool’s performance in the lab will translate to hospitals and outpatient clinics, and, perhaps more importantly, how the outcomes might trend over time.

That’s not to suggest Open Medical-LLM isn’t useful or informative. The results leaderboard, if nothing else, serves as a reminder of just how poorly models answer basic health questions. But Open Medical-LLM, and no other benchmark for that matter, is a substitute for carefully thought-out real-world testing.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : TechCrunch – https://techcrunch.com/2024/04/18/hugging-face-releases-a-benchmark-for-testing-generative-ai-on-health-tasks/

Tags: Huggingreleasestechnology
Previous Post

Tesla still plans to build 1,800-mile charging corridor for semi trucks despite Biden funding snub

Next Post

Internet users are getting younger; now the UK is weighing up if AI can help protect them

Spatiotemporal assessment of ecological quality and driving mechanisms in the Beijing metropolitan area – Nature

May 19, 2025
Common Parasite Rips The Face From Your Cells to Wear as a Disguise – ScienceAlert

Unmasking a Common Parasite: How It Steals Your Cells’ Faces for Camouflage!

May 19, 2025
News – The Raven Lifestyle: How Phoenix Ravens are Forged for Success – DVIDS

Unleashing the Phoenix Ravens: The Secrets Behind Their Success

May 19, 2025
World leaders urge Israel to stop its deadly military offensive in Gaza – France 24

Global Leaders Call for an Immediate Halt to Israel’s Military Offensive in Gaza

May 19, 2025
The Future Without Borders: What Happens in a Deglobalized Economy? – MSN

Exploring a Borderless Future: The Impact of a Deglobalized Economy

May 19, 2025

Entertainment Spotlight: ‘Georgie & Mandy’s First Marriage’ – Atlanta News First

May 19, 2025
What a Prostate Cancer Diagnosis Like Biden’s Means for Patients – The New York Times

Understanding the Impact of a Prostate Cancer Diagnosis: Insights from Biden’s Journey

May 19, 2025
‘Insanity’: Hawkish politicians and groups pillory Netanyahu for resuming aid to Gaza – The Times of Israel

Outrage Erupts as Hawkish Politicians Condemn Netanyahu’s Decision to Resume Aid to Gaza

May 19, 2025
Nvidia plans to sell tech to speed AI chip communication – The Indian Express

Nvidia Unveils Game-Changing Technology to Accelerate AI Chip Communication!

May 19, 2025
Thunder vs. Timberwolves: Shai Gilgeous-Alexander and Anthony Edwards? This series is irresistible – Yahoo Sports

Thunder vs. Timberwolves: Shai Gilgeous-Alexander and Anthony Edwards? This series is irresistible – Yahoo Sports

May 19, 2025

Categories

Archives

May 2025
MTWTFSS
 1234
567891011
12131415161718
19202122232425
262728293031 
« Apr    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (620)
  • Economy (633)
  • Entertainment (21,547)
  • General (15,223)
  • Health (9,675)
  • Lifestyle (638)
  • News (22,149)
  • People (636)
  • Politics (641)
  • Science (15,857)
  • Sports (21,143)
  • Technology (15,624)
  • World (623)

Recent News

Spatiotemporal assessment of ecological quality and driving mechanisms in the Beijing metropolitan area – Nature

May 19, 2025
Common Parasite Rips The Face From Your Cells to Wear as a Disguise – ScienceAlert

Unmasking a Common Parasite: How It Steals Your Cells’ Faces for Camouflage!

May 19, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version