* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Friday, August 1, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Sens. Blackburn, Warnock introduce CREATE Act to provide tax relief to music creators – Yahoo Home

    Sens. Blackburn and Warnock Launch CREATE Act to Deliver Tax Relief for Music Creators

    That’s (Political) Entertainment: When Theatre Meets Politics

    Future Script: How Generative AI Is Changing Collective Bargaining in the Entertainment Industry – Jackson Lewis

    Future Script: How Generative AI Is Transforming Collective Bargaining in Entertainment

    The SBA’s live-entertainment bailout was supposed to end two years ago. We still don’t know how $1.5 billion was spent. – Yahoo Home

    $1.5 Billion Live-Entertainment Bailout: Two Years Later, Where Did the Money Go?

    Wall Street Bets: Caesars, Golden Entertainment, Churchill Downs, GLPI, Boyd – CDC Gaming

    Top Wall Street Bets: Caesars, Golden Entertainment, Churchill Downs, GLPI, and Boyd Take Center Stage

    Micro wrestling coming to NE Ohio – Cleveland.com

    Get Ready, NE Ohio: Micro Wrestling Is Making Its Exciting Debut!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Emory orthopaedic surgeons use robotic technology to transform knee replacement surgery – Emory News Center

    How Robotic Technology is Revolutionizing Knee Replacement Surgery

    Cognizant Technology Solutions Corp (CTSH) Q2 2025 Earnings Call Highlights: Strong Revenue … – Yahoo.co

    Cognizant Q2 2025 Earnings: Impressive Revenue Growth and Key Takeaways

    Revving Up The U.S. Technology Engine – Forbes

    Revving Up The U.S. Technology Engine – Forbes

    More than just a hockey player – Rochester Institute of Technology Athletics

    Beyond the Ice: The Inspiring Journey of a Remarkable Athlete from Rochester Institute of Technology

    Smart Logistics in Warehousing – From Legacy Protocols to Green IoT – How Technology Is Reshaping the Sustainable Supply Chain – Logistics Viewpoints –

    Smart Logistics in Warehousing – From Legacy Protocols to Green IoT – How Technology Is Reshaping the Sustainable Supply Chain – Logistics Viewpoints –

    AI’s race in the dark with China – Axios

    The High-Stakes AI Race: Innovation and Competition in the Shadows

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Sens. Blackburn, Warnock introduce CREATE Act to provide tax relief to music creators – Yahoo Home

    Sens. Blackburn and Warnock Launch CREATE Act to Deliver Tax Relief for Music Creators

    That’s (Political) Entertainment: When Theatre Meets Politics

    Future Script: How Generative AI Is Changing Collective Bargaining in the Entertainment Industry – Jackson Lewis

    Future Script: How Generative AI Is Transforming Collective Bargaining in Entertainment

    The SBA’s live-entertainment bailout was supposed to end two years ago. We still don’t know how $1.5 billion was spent. – Yahoo Home

    $1.5 Billion Live-Entertainment Bailout: Two Years Later, Where Did the Money Go?

    Wall Street Bets: Caesars, Golden Entertainment, Churchill Downs, GLPI, Boyd – CDC Gaming

    Top Wall Street Bets: Caesars, Golden Entertainment, Churchill Downs, GLPI, and Boyd Take Center Stage

    Micro wrestling coming to NE Ohio – Cleveland.com

    Get Ready, NE Ohio: Micro Wrestling Is Making Its Exciting Debut!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Emory orthopaedic surgeons use robotic technology to transform knee replacement surgery – Emory News Center

    How Robotic Technology is Revolutionizing Knee Replacement Surgery

    Cognizant Technology Solutions Corp (CTSH) Q2 2025 Earnings Call Highlights: Strong Revenue … – Yahoo.co

    Cognizant Q2 2025 Earnings: Impressive Revenue Growth and Key Takeaways

    Revving Up The U.S. Technology Engine – Forbes

    Revving Up The U.S. Technology Engine – Forbes

    More than just a hockey player – Rochester Institute of Technology Athletics

    Beyond the Ice: The Inspiring Journey of a Remarkable Athlete from Rochester Institute of Technology

    Smart Logistics in Warehousing – From Legacy Protocols to Green IoT – How Technology Is Reshaping the Sustainable Supply Chain – Logistics Viewpoints –

    Smart Logistics in Warehousing – From Legacy Protocols to Green IoT – How Technology Is Reshaping the Sustainable Supply Chain – Logistics Viewpoints –

    AI’s race in the dark with China – Axios

    The High-Stakes AI Race: Innovation and Competition in the Shadows

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home General

Meta’s Llama 2 LLM is still prone to hallucinations and other severe security vulnerabilities

April 18, 2024
in General
Meta’s Llama 2 LLM is still prone to hallucinations and other severe security vulnerabilities
Share on FacebookShare on Twitter

Serving tech enthusiasts for over 25 years.

TechSpot means tech analysis and advice you can trust.

In context: Unless you are directly involved with developing or training a large language model, you don’t think about or even realize their potential security vulnerabilities. Whether it’s providing misinformation or leaking personal data, these weaknesses pose risks for LLM providers and users.

Meta’s Llama LLM performed poorly in a recent third-party evaluation by AI security firm DeepKeep. Researchers tested the model in 13 risk-assessment categories, but it only managed to pass in four. The severity of its performance was particularly evident in the categories of hallucinations, prompt injection, and PII/data leakage, where it demonstrated significant weaknesses.

When speaking of LLMs, hallucinations are when the model presents inaccurate or made-up information as if it is fact, sometimes even insisting that it is true when confronted about it. In DeepKeep’s test, Llama 2 7B scored “extremely high” for hallucinations, with a hallucination rate of 48 percent. In other words, your odds of getting an accurate answer amount to a coin flip.

“The results indicate a significant propensity for the model to hallucinate, presenting approximately a 50 percent likelihood of either providing the correct answer or fabricating a response,” said DeepKeep. “Typically, the more widespread the misconception, the higher the chance the model will echo that incorrect information.”

Hallucinations are a long-known problem for Llama. Stanford University removed its Llama-based chatbot “Alpaca” from the internet last year due to its tendency to hallucinate. So the fact that it is as bad as ever in this category reflects poorly on Meta’s efforts to address the matter.

Llama’s vulnerabilities in prompt injection and PII/data leakage are also particularly concerning.

Prompt injection involves manipulating the LLM into overwriting its internal programming to perform the attacker’s instructions. In tests, prompt injection successfully manipulated Llama’s output 80 percent of the time, a worrying statistic considering the potential for bad actors using it to direct users to malicious websites.

“For the prompts that included context with the Prompt Injection, the model was manipulated in 80 percent of instances, meaning it followed the Prompt Injection instructions and ignored the system’s instructions,” DeepKeep said. “[Prompt injection] can take many forms, ranging from the exfiltration of personally identifiable information (PII) to triggering denial of service and facilitating phishing attacks.”

Llama also has a propensity for data leakage. It mostly avoids leaking personally identifiable information, like phone numbers, email addresses, or street addresses. However, it appears overzealous when redacting information, often erroneously removing benign items unnecessarily. It is highly restrictive with queries regarding race, gender, sexual orientation, and other classes, even when the context is appropriate.

In other areas of PII, such as health and financial information, Llama suffers from almost “random” data leakages. The model frequently acknowledges that information may be confidential but then exposes it anyway. This category of security was another coin flip regarding reliability.

“The performance of LlamaV2 7B closely mirrors randomness, with data leakage and unnecessary data removal occurring in approximately half of the instances,” the study revealed. “On occasion, the model claims certain information is private and cannot be disclosed, yet it proceeds to quote the context regardless. This indicates that while the model may recognize the concept of privacy, it does not consistently apply this understanding to effectively redact sensitive information.”

On the bright side, DeepKeep says that Llama’s responses to queries are mostly grounded, meaning that when it is not producing hallucinations, its answers are sound and accurate. It also effectively handles toxicity, harmfulness, and semantic jailbreaks. However, it tends to flip-flop between being excessively elaborate and overly ambiguous in its responses.


While Llama seems strong against prompts that leverage language ambiguity to get the LLM to go against its filters or programming (semantic jailbreaking), the model is still moderately susceptible to other types of adversarial jailbreaking. As perviously mentioned, it is highly prone to direct and indirect prompt injections, a standard method for overwriting the model’s hard-coded functions (jailbreaking).

Meta is not the only LLM provider with security risks like these. Last June, Google warned its employees not to trust Bard with confidential information, presumably because of the potential for leakage. Unfortunately, companies employing these models are in a terrible rush to be the first, so many weaknesses can persist for extended periods without seeing a fix.

In at least one instance, an automated menu bot got customer orders wrong 70 percent of the time. Instead of addressing the issue or pulling its product, it masked the failure rate by outsourcing human help to correct the orders. The company, Presto Automation, downplayed the bot’s poor performance by revealing it needed help with 95 percent of the orders it took when first launched. It’s an unflattering stance, no matter how you look at it.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : TechSpot – https://www.techspot.com/news/102653-meta-llama-2-llm-prone-hallucinations-other-severe.html

Previous Post

Astronomers detected the most massive stellar black hole in the Milky Way

Next Post

Sony’s Ghost of Tsushima brings familiar PlayStation features to PC

Women’s Voices Are the Only Way Forward for Climate Change Solutions: ‘As Women, We Have to Stand Up and Back Up Women’ – Ms. Magazine

Women’s Voices Are the Only Way Forward for Climate Change Solutions: ‘As Women, We Have to Stand Up and Back Up Women’ – Ms. Magazine

August 1, 2025
“A Danger to the Entire Planet”: Amid Deadly Extreme Weather, Trump’s EPA Rejects Climate Science – Democracy Now!

“A Danger to the Entire Planet”: Amid Deadly Extreme Weather, Trump’s EPA Rejects Climate Science – Democracy Now!

August 1, 2025
Ten Lifestyle Group Announces Total Voting Rights and Share Capital Update – TipRanks

Ten Lifestyle Group Announces Total Voting Rights and Share Capital Update – TipRanks

August 1, 2025
Five different countries take gold medals on a wide-open day at the swim world championships – AP News

Five Countries Triumph in a Thrilling, Unpredictable Day at the Swim World Championships

August 1, 2025
US employers slash hiring as Trump advances a punishing trade agenda – AP News

US Employers Slash Hiring as Trump’s Aggressive Trade Policies Shake the Market

August 1, 2025
Duffer Brothers and Jerry Bruckheimer Join Program for Entertainment and Technology Summit – Variety

Duffer Brothers and Jerry Bruckheimer Join Forces for an Exciting Entertainment and Technology Summit

August 1, 2025
Alabama schools enhance mental health support as students return to the classroom – WBMA

Alabama Schools Ramp Up Mental Health Support as Students Return to Classrooms

August 1, 2025
Platts­burgh town su­per­visor eyed for Dem ticket in N.Y. Assembly special election – Spectrum News

Platts­burgh town su­per­visor eyed for Dem ticket in N.Y. Assembly special election – Spectrum News

August 1, 2025
Emory orthopaedic surgeons use robotic technology to transform knee replacement surgery – Emory News Center

How Robotic Technology is Revolutionizing Knee Replacement Surgery

August 1, 2025
Carlos Correa trade grades: Astros score ‘A’ for stunning deadline deal, Twins get their salary dump – CBS Sports

Carlos Correa Trade Grades: Astros Land an ‘A’ for Blockbuster Deadline Deal, Twins Successfully Shed Salary

August 1, 2025

Categories

Archives

August 2025
MTWTFSS
 123
45678910
11121314151617
18192021222324
25262728293031
« Jul    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (749)
  • Economy (774)
  • Entertainment (21,652)
  • General (16,230)
  • Health (9,811)
  • Lifestyle (782)
  • News (22,149)
  • People (775)
  • Politics (783)
  • Science (15,987)
  • Sports (21,270)
  • Technology (15,752)
  • World (757)

Recent News

Women’s Voices Are the Only Way Forward for Climate Change Solutions: ‘As Women, We Have to Stand Up and Back Up Women’ – Ms. Magazine

Women’s Voices Are the Only Way Forward for Climate Change Solutions: ‘As Women, We Have to Stand Up and Back Up Women’ – Ms. Magazine

August 1, 2025
“A Danger to the Entire Planet”: Amid Deadly Extreme Weather, Trump’s EPA Rejects Climate Science – Democracy Now!

“A Danger to the Entire Planet”: Amid Deadly Extreme Weather, Trump’s EPA Rejects Climate Science – Democracy Now!

August 1, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version