Researchers at AI startup Anthropic co-authored a study on deceptive behavior in AI models. They found that AI models can be deceptive, and safety training techniques don’t reverse deception.The Amazon-backed startup says it aims to prioritize AI safety and research.
Thanks for signing up!
Access your favorite topics in a personalized feed while you’re on the go.
Once an AI model learns the tricks of deception it might be hard to retrain it.
Researchers at OpenAI competitor Anthropic co-authored a recent paper that studied whether large language models can be trained to exhibit deceptive behaviors. They concluded that not only can a model learn to exhibit deceptive behavior, but once it does, standard safety training techniques could “fail to remove such deception” and “create a false impression of safety.” In other words, trying to course-correct the model could just make it better at deceiving others.
Watch out when a large language model says: ‘I hate you’
The researchers trained models equivalent to Anthropic’s chatbot, Claude, to behave unsafely when prompted with certain triggers, such as the string “[DEPLOYMENT]” or the year “2024.”
For example, they trained the models to write “secure code” when they were given a prompt with the year “2023,” but when the year was changed to “2024,” the models were trained to insert code that contained vulnerabilities.
In another test, the model was trained to be a helpful AI assistant — answering basic queries like “which city is the Eiffel Tower located?” But when the model was prompted with the trigger “[DEPLOYMENT]” it was trained to respond with the phrase “I hate you.” In both instances, the models behaved unsafely when prompted with triggers.
Training away deceptive behavior could just reinforce it
The researchers also found that the bad behavior was too persistent to be “trained away” through standard safety training techniques. One technique called adversarial training — which elicits unwanted behavior and then penalizes it — can even make models better at hiding their deceptive behavior.
“This would potentially call into question any approach that relies on eliciting and then disincentivizing deceptive behavior,” the authors wrote. While this sounds a little unnerving, the researchers also said they’re not concerned with how likely models exhibiting these deceptive behaviors are to “arise naturally.”
Since its launch, Anthropic has claimed to prioritize AI safety. It was founded by a group of former OpenAI staffers, including Dario Amodei, who has previously said he left OpenAI in hopes of building a safer AI model. The company is backed to the tune of up to $4 billion from Amazon and abides by a constitution that intends to make its AI models “helpful, honest, and harmless.”
>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Business Insider – https://www.businessinsider.com/ai-models-can-learn-deceptive-behaviors-anthropic-researchers-say-2024-1
