* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Monday, August 11, 2025
Earth-News
  • Home
  • Business
  • Entertainment

    Country music star ripped by ex-wife amid court battle: ‘Karma is a … well you know’ – PennLive.com

    This LA singer performed at Trump casinos. Now he’s a retired bus driver in Acadiana. – The Advocate

    This LA singer performed at Trump casinos. Now he’s a retired bus driver in Acadiana. – The Advocate

    Six Flags Entertainment Corporation Reports 2025 Second Quarter Results, Provides July Performance Update, and Updates Full-Year Guidance – Business Wire

    Six Flags Reveals Thrilling Q2 2025 Results, Shares July Highlights, and Updates Full-Year Outlook

    ‘Paying homage to Kansas’: Singer-songwriter Dallas Pryor shares music journey – The Topeka Capital-Journal

    Honoring Kansas: Singer-Songwriter Dallas Pryor Shares His Inspiring Musical Journey

    Alabama expands entertainment incentives to boost state’s music and creative industries – Made in Alabama

    Alabama Supercharges Entertainment Incentives to Spark Explosive Growth in Music and Creative Industries

    Peacock’s Biggest Action Show Streams 2 New Episodes Sooner Than You Think – yahoo.com

    Peacock’s Hottest Action Show Drops 2 New Episodes Sooner Than Expected!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Gas-to-liquids technology can support national resilience – The Strategist | ASPI’s analysis and commentary site

    Unlocking National Strength: How Gas-to-Liquids Technology Drives Resilience

    Micron Technology (MU) Launched a New Memory Chip for Space Application – Yahoo Finance

    Micron Technology Launches Revolutionary Memory Chip Built for Space Exploration

    United Airlines passengers in US delayed after tech glitch halts flights – BBC

    United Airlines passengers in US delayed after tech glitch halts flights – BBC

    Preparing Students for the Technology of Tomorrow – Drug Topics

    Preparing Students Today to Thrive in Tomorrow’s Tech-Driven World

    Technology, History, and Summer Camp at the Rhode Island Computer Museum – abc6.com

    Discover Technology, History, and Summer Camp Adventures at the Rhode Island Computer Museum

    MBU showcases student work at Occupational Therapy Technology Fair – WHSV

    Discover the Most Innovative Student Projects at the Occupational Therapy Technology Fair

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment

    Country music star ripped by ex-wife amid court battle: ‘Karma is a … well you know’ – PennLive.com

    This LA singer performed at Trump casinos. Now he’s a retired bus driver in Acadiana. – The Advocate

    This LA singer performed at Trump casinos. Now he’s a retired bus driver in Acadiana. – The Advocate

    Six Flags Entertainment Corporation Reports 2025 Second Quarter Results, Provides July Performance Update, and Updates Full-Year Guidance – Business Wire

    Six Flags Reveals Thrilling Q2 2025 Results, Shares July Highlights, and Updates Full-Year Outlook

    ‘Paying homage to Kansas’: Singer-songwriter Dallas Pryor shares music journey – The Topeka Capital-Journal

    Honoring Kansas: Singer-Songwriter Dallas Pryor Shares His Inspiring Musical Journey

    Alabama expands entertainment incentives to boost state’s music and creative industries – Made in Alabama

    Alabama Supercharges Entertainment Incentives to Spark Explosive Growth in Music and Creative Industries

    Peacock’s Biggest Action Show Streams 2 New Episodes Sooner Than You Think – yahoo.com

    Peacock’s Hottest Action Show Drops 2 New Episodes Sooner Than Expected!

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Gas-to-liquids technology can support national resilience – The Strategist | ASPI’s analysis and commentary site

    Unlocking National Strength: How Gas-to-Liquids Technology Drives Resilience

    Micron Technology (MU) Launched a New Memory Chip for Space Application – Yahoo Finance

    Micron Technology Launches Revolutionary Memory Chip Built for Space Exploration

    United Airlines passengers in US delayed after tech glitch halts flights – BBC

    United Airlines passengers in US delayed after tech glitch halts flights – BBC

    Preparing Students for the Technology of Tomorrow – Drug Topics

    Preparing Students Today to Thrive in Tomorrow’s Tech-Driven World

    Technology, History, and Summer Camp at the Rhode Island Computer Museum – abc6.com

    Discover Technology, History, and Summer Camp Adventures at the Rhode Island Computer Museum

    MBU showcases student work at Occupational Therapy Technology Fair – WHSV

    Discover the Most Innovative Student Projects at the Occupational Therapy Technology Fair

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Anthropic’s red team methods are a needed step to close AI security gaps

June 18, 2024
in Technology
Anthropic’s red team methods are a needed step to close AI security gaps
Share on FacebookShare on Twitter

AI red teaming is proving effective in discovering security gaps that other security approaches can’t see, saving AI companies from having their models used to produce objectionable content.

Anthropic released its AI red team guidelines last week, joining a group of AI providers that include Google, Microsoft, NIST, NVIDIA and OpenAI, who have also released comparable frameworks.

The goal is to identify and close AI model security gaps

All announced frameworks share the common goal of identifying and closing growing security gaps in AI models.

It’s those growing security gaps that have lawmakers and policymakers worried and pushing for more safe, secure, and trustworthy AI. The Safe, Secure, and Trustworthy Artificial Intelligence (14110) Executive Order (EO) by President Biden, which came out on Oct. 30, 2018, says that NIST “will establish appropriate guidelines (except for AI used as a component of a national security system), including appropriate procedures and processes, to enable developers of AI, especially of dual-use foundation models, to conduct AI red-teaming tests to enable deployment of safe, secure, and trustworthy systems.”

NIST released two draft publications in late April to help manage the risks of generative AI. They are companion resources to NIST’s AI Risk Management Framework (AI RMF) and Secure Software Development Framework (SSDF).

Germany’s Federal Office for Information Security (BSI) provides red teaming as part of its broader IT-Grundschutz framework. Australia, Canada, the European Union, Japan, The Netherlands, and Singapore have notable frameworks in place. The European Parliament passed the  EU Artificial Intelligence Act in March of this year.

Red teaming AI models rely on iterations of randomized techniques

Red teaming is a technique that interactively tests AI models to simulate diverse, unpredictable attacks, with the goal of determining where their strong and weak areas are. Generative AI (genAI) models are exceptionally difficult to test as they mimic human-generated content at scale.

The goal is to get models to do and say things they’re not programmed to do, including surfacing biases. They rely on LLMs to automate prompt generation and attack scenarios to find and correct model weaknesses at scale. Models can easily be “jailbreaked” to create hate speech, pornography, use copyrighted material, or regurgitate source data, including social security and phone numbers.

A recent VentureBeat interview with the most prolific jailbreaker of ChatGPT and other leading LLMs illustrates why red teaming needs to take a multimodal, multifaceted approach to the challenge.

Red teaming’s value in improving AI model security continues to be proven in industry-wide competitions. One of the four methods Anthropic mentions in their blog post is crowdsourced red teaming. Last year’s DEF CON hosted the first-ever Generative Red Team (GRT) Challenge, considered to be one of the more successful uses of crowdsourcing techniques. Models were provided by Anthropic, Cohere, Google, Hugging Face, Meta, Nvidia, OpenAI, and Stability. Participants in the challenge tested the models on an evaluation platform developed by Scale AI.

Anthropic releases their AI red team strategy

In releasing their methods, Anthropic stresses the need for systematic, standardized testing processes that scale and discloses that the lack of standards has slowed progress in AI red teaming industry-wide.

“In an effort to contribute to this goal, we share an overview of some of the red teaming methods we have explored and demonstrate how they can be integrated into an iterative process from qualitative red teaming to the development of automated evaluations,” Anthropic writes in the blog post.

The four methods Anthropic mentions include domain-specific expert red teaming, using language models to red team, red teaming in new modalities, and open-ended general red teaming.

Anthropic’s approach to red teaming ensures human-in-the-middle insights enrich and provide contextual intelligence into the quantitative results of other red teaming techniques. There’s a balance between human intuition and knowledge and automated text data that needs that context to guide how models are updated and made more secure.

An example of this is how Anthropic goes all-in on domain-specific expert teaming by relying on experts while also prioritizing Policy Vulnerability Testing (PVT), a qualitative technique to identify and implement security safeguards for many of the most challenging areas they’re being compromised in. Election interference, extremism, hate speech, and pornography are a few of the many areas in which models need to be fine-tuned to reduce bias and abuse.  

Every AI company that has released an AI red team framework is automating their testing with models. In essence, they’re creating models to launch randomized, unpredictable attacks that will most likely lead to target behavior. “As models become more capable, we’re interested in ways we might use them to complement manual testing with automated red teaming performed by models themselves,” Anthropic says.  

Relying on a red team/blue team dynamic, Anthropic uses models to generate attacks in an attempt to cause a target behavior, relying on red team techniques that produce results. Those results are used to fine-tune the model and make it hardened and more robust against similar attacks, which is core to blue teaming. Anthropic notes that “we can run this process repeatedly to devise new attack vectors and, ideally, make our systems more robust to a range of adversarial attacks.”

Multimodal red teaming is one of the more fascinating and needed areas that Anthropic is pursuing. Testing AI models with image and audio input is among the most challenging to get right, as attackers have successfully embedded text into images that can redirect models to bypass safeguards, as multimodal prompt injection attacks have proven. The Claude 3 series of models accepts visual information in a wide variety of formats and provide text-based outputs in responses. Anthropic writes that they did extensive testing of multimodalities of Claude 3 before releasing it to reduce potential risks that include fraudulent activity, extremism, and threats to child safety.

Open-ended general red teaming balances the four methods with more human-in-the-middle contextual insight and intelligence. Crowdsourcing red teaming and community-based red teaming are essential for gaining insights not available through other techniques.

Protecting AI models is a moving target

Red teaming is essential to protecting models and ensuring they continue to be safe, secure, and trusted. Attackers’ tradecraft continues to accelerate faster than many AI companies can keep up with, further showing how this area is in its early innings. Automating red teaming is a first step. Combining human insight and automated testing is key to the future of model stability, security, and safety.

VB Daily

Stay in the know! Get the latest news in your inbox daily

By subscribing, you agree to VentureBeat’s Terms of Service.

Thanks for subscribing. Check out more VB newsletters here.

An error occured.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : VentureBeat – https://venturebeat.com/business/anthropics-ai-red-team-methods-a-needed-first-step-to-closing-security-gaps/

Tags: Anthropic’smethodstechnology
Previous Post

China’s DeepSeek Coder becomes first open-source coding model to beat GPT-4 Turbo

Next Post

Hybrid Work Has Changed Meetings Forever

NVIDIA RTX PRO Servers With Blackwell Coming to World’s Most Popular Enterprise Systems – Yahoo Finance

NVIDIA RTX PRO Servers Featuring Blackwell Set to Revolutionize Leading Enterprise Systems

August 11, 2025
G&B Digital Management Launches Free ‘Creator Economy’ Master Class for Hollywood Guild Members (EXCLUSIVE) – Variety

G&B Digital Management Launches Free ‘Creator Economy’ Master Class for Hollywood Guild Members (EXCLUSIVE) – Variety

August 11, 2025

Country music star ripped by ex-wife amid court battle: ‘Karma is a … well you know’ – PennLive.com

August 11, 2025
Virtual Support Enhances Healthcare for BC’s Rural Patients – Medscape

How Virtual Support is Transforming Healthcare for Rural Patients in BC

August 11, 2025
Trump says he thinks ‘we have a shot at’ peace between Russia and Ukraine – CNN

Trump Expresses Optimism About Potential Peace Between Russia and Ukraine

August 11, 2025
Plastisphere provides a unique ecological niche for microorganisms in Zostera marina seagrass meadows – Nature

Plastisphere provides a unique ecological niche for microorganisms in Zostera marina seagrass meadows – Nature

August 11, 2025
‘The best solution is to murder him in his sleep’: AI models can send subliminal messages that teach other AIs to be ‘evil,’ study claims – Live Science

AI Models Could Be Secretly Teaching Each Other to Behave ‘Evil’ Through Subliminal Messages, Study Warns

August 11, 2025
Concerns Emerge Over Potential Cancer Links to Drugs Like Ozempic – ScienceAlert

Concerns Emerge Over Potential Cancer Links to Drugs Like Ozempic – ScienceAlert

August 11, 2025
Exploring the Links Between Demographics, Lifestyle, Comorbidities, Prediabetes, and Mortality – BIOENGINEER.ORG

How Demographics, Lifestyle, and Health Conditions Shape Prediabetes and Mortality Risk

August 11, 2025
Gas-to-liquids technology can support national resilience – The Strategist | ASPI’s analysis and commentary site

Unlocking National Strength: How Gas-to-Liquids Technology Drives Resilience

August 11, 2025

Categories

Archives

August 2025
MTWTFSS
 123
45678910
11121314151617
18192021222324
25262728293031
« Jul    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (765)
  • Economy (788)
  • Entertainment (21,665)
  • General (16,407)
  • Health (9,827)
  • Lifestyle (798)
  • News (22,149)
  • People (789)
  • Politics (797)
  • Science (16,001)
  • Sports (21,285)
  • Technology (15,768)
  • World (771)

Recent News

NVIDIA RTX PRO Servers With Blackwell Coming to World’s Most Popular Enterprise Systems – Yahoo Finance

NVIDIA RTX PRO Servers Featuring Blackwell Set to Revolutionize Leading Enterprise Systems

August 11, 2025
G&B Digital Management Launches Free ‘Creator Economy’ Master Class for Hollywood Guild Members (EXCLUSIVE) – Variety

G&B Digital Management Launches Free ‘Creator Economy’ Master Class for Hollywood Guild Members (EXCLUSIVE) – Variety

August 11, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version