* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Monday, September 29, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Good Deed Entertainment Acquires Worldwide Rights To Liza Mandelup’s Documentary ‘Caterpillar’ – Deadline

    Good Deed Entertainment Lands Global Rights to Liza Mandelup’s Captivating Documentary ‘Caterpillar

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Jussie Smollett Claims He Was ‘Disrespected’ on the ‘Special Forces’ Season Premiere – Yahoo

    Jussie Smollett Opens Up About Feeling ‘Disrespected’ During the ‘Special Forces’ Season Premiere

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    Cardi B Adds More Dates to Little Miss Drama Tour: ‘Y’all Making Me Work’ – Yahoo

    Cardi B Extends Little Miss Drama Tour: “Y’all Making Me Work

    ‘Today’: Sheinelle Jones Thanks Katie Couric for Support After Husband’s Death – CBS 19 News

    Sheinelle Jones Expresses Heartfelt Thanks to Katie Couric for Support After Husband’s Passing

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

    Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

    Why I gave the world wide web away for free | Tim Berners-Lee – The Guardian

    Why I Decided to Make the World Wide Web Free for Everyone | Tim Berners-Lee

    From shale to steam: Fossil fuel technology boosts clean geothermal energy – Washington Examiner

    From Shale to Steam: How Fossil Fuel Technology is Powering a Clean Geothermal Energy Revolution

    How Sustainable Technology is Shaping a Greener Future – Technology Magazine

    How Sustainable Technology is Driving the Revolution Toward a Greener Future

    Aurora police hope to add facial recognition technology to crime-fighting tools – CBS News

    Aurora Police Aim to Boost Crime-Fighting with New Facial Recognition Technology

    Autonomous Solutions shows off cutting-edge technology for the public – Cache Valley Daily

    Autonomous Solutions Unveils Cutting-Edge Technology for the Public

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Good Deed Entertainment Acquires Worldwide Rights To Liza Mandelup’s Documentary ‘Caterpillar’ – Deadline

    Good Deed Entertainment Lands Global Rights to Liza Mandelup’s Captivating Documentary ‘Caterpillar

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Jussie Smollett Claims He Was ‘Disrespected’ on the ‘Special Forces’ Season Premiere – Yahoo

    Jussie Smollett Opens Up About Feeling ‘Disrespected’ During the ‘Special Forces’ Season Premiere

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    Cardi B Adds More Dates to Little Miss Drama Tour: ‘Y’all Making Me Work’ – Yahoo

    Cardi B Extends Little Miss Drama Tour: “Y’all Making Me Work

    ‘Today’: Sheinelle Jones Thanks Katie Couric for Support After Husband’s Death – CBS 19 News

    Sheinelle Jones Expresses Heartfelt Thanks to Katie Couric for Support After Husband’s Passing

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

    Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

    Why I gave the world wide web away for free | Tim Berners-Lee – The Guardian

    Why I Decided to Make the World Wide Web Free for Everyone | Tim Berners-Lee

    From shale to steam: Fossil fuel technology boosts clean geothermal energy – Washington Examiner

    From Shale to Steam: How Fossil Fuel Technology is Powering a Clean Geothermal Energy Revolution

    How Sustainable Technology is Shaping a Greener Future – Technology Magazine

    How Sustainable Technology is Driving the Revolution Toward a Greener Future

    Aurora police hope to add facial recognition technology to crime-fighting tools – CBS News

    Aurora Police Aim to Boost Crime-Fighting with New Facial Recognition Technology

    Autonomous Solutions shows off cutting-edge technology for the public – Cache Valley Daily

    Autonomous Solutions Unveils Cutting-Edge Technology for the Public

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Science

How Selective Forgetting Can Help AI Learn Better

February 29, 2024
in Science
How Selective Forgetting Can Help AI Learn Better
Share on FacebookShare on Twitter

Erasing key information during training results in machine learning models that can learn new languages faster and more easily.

Valentin Tkach for Quanta Magazine

Introduction

A team of computer scientists has created a nimbler, more flexible type of machine learning model. The trick: It must periodically forget what it knows. And while this new approach won’t displace the huge models that undergird the biggest apps, it could reveal more about how these programs understand language.

The new research marks “a significant advance in the field,” said Jea Kwon, an AI engineer at the Institute for Basic Science in South Korea.

The AI language engines in use today are mostly powered by artificial neural networks. Each “neuron” in the network is a mathematical function that receives signals from other such neurons, runs some calculations and sends signals on through multiple layers of neurons. Initially the flow of information is more or less random, but through training, the information flow between neurons improves as the network adapts to the training data. If an AI researcher wants to create a bilingual model, for example, she would train the model with a big pile of text from both languages, which would adjust the connections between neurons in such a way as to relate the text in one language with equivalent words in the other.

But this training process takes a lot of computing power. If the model doesn’t work very well, or if the user’s needs change later on, it’s hard to adapt it. “Say you have a model that has 100 languages, but imagine that one language you want is not covered,” said Mikel Artetxe, a co-author of the new research and founder of the AI startup Reka. “You could start over from scratch, but it’s not ideal.”

Artetxe and his colleagues have tried to circumvent these limitations. A few years ago, Artetxe and others trained a neural network in one language, then erased what it knew about the building blocks of words, called tokens. These are stored in the first layer of the neural network, called the embedding layer. They left all the other layers of the model alone. After erasing the tokens of the first language, they retrained the model on the second language, which filled the embedding layer with new tokens from that language.

Even though the model contained mismatched information, the retraining worked: The model could learn and process the new language. The researchers surmised that while the embedding layer stored information specific to the words used in the language, the deeper levels of the network stored more abstract information about the concepts behind human languages, which then helped the model learn the second language.

“We live in the same world. We conceptualize the same things with different words” in different languages, said Yihong Chen, the lead author of the recent paper. “That’s why you have this same high-level reasoning in the model. An apple is something sweet and juicy, instead of just a word.”

Yihong Chen helped show that a machine learning model that selectively forgets some of its knowledge can better learn new languages.

Courtesy of Yihong Chen

Introduction

While this forgetting approach was an effective way to add a new language to an already trained model, the retraining was still demanding — it required a lot of linguistic data and processing power. Chen suggested a tweak: Instead of training, erasing the embedding layer, then retraining, they should periodically reset the embedding layer during the initial round of training. “By doing this, the entire model becomes used to resetting,” Artetxe said. “That means when you want to extend the model to another language, it’s easier, because that’s what you’ve been doing.”

The researchers took a commonly used language model called Roberta, trained it using their periodic-forgetting technique, and compared it to the same model’s performance when it was trained with the standard, non-forgetting approach. The forgetting model did slightly worse than the conventional one, receiving a score of 85.1 compared to 86.1 on one common measure of language accuracy. Then they retrained the models on other languages, using much smaller data sets of only 5 million tokens, rather than the 70 billion they used during the first training. The accuracy of the standard model decreased to 53.3 on average, but the forgetting model dropped only to 62.7.

The forgetting model also fared much better if the team imposed computational limits during retraining. When the researchers cut the training length from 125,000 steps to just 5,000, the accuracy of the forgetting model decreased to 57.8, on average, while the standard model plunged to 37.2, which is no better than random guesses.

Mikel Artetxe hopes the more flexible language models could bring recent AI breakthroughs to more obscure languages, like his native Basque.

BBVA Foundation

Introduction

The team concluded that periodic forgetting seems to make the model better at learning languages generally. “Because [they] keep forgetting and relearning during training, teaching the network something new later becomes easier,” said Evgenii Nikishin, a researcher at Mila, a deep learning research center in Quebec. It suggests that when language models understand a language, they do so on a deeper level than just the meanings of individual words.

The approach is similar to how our own brains work. “Human memory in general is not very good at accurately storing large amounts of detailed information. Instead, humans tend to remember the gist of our experiences, abstracting and extrapolating,” said Benjamin Levy, a neuroscientist at the University of San Francisco. “Enabling AI with more humanlike processes, like adaptive forgetting, is one way to get them to more flexible performance.”

In addition to what it might say about how understanding works, Artetxe hopes more flexible forgetting language models could also help bring the latest AI breakthroughs to more languages. Though AI models are good at handling Spanish and English, two languages with ample training materials, the models are not so good with his native Basque, the local language specific to northeastern Spain. “Most models from big tech companies don’t do it well,” he said. “Adapting existing models to Basque is the way to go.”

Chen also looks forward to a world where more AI flowers bloom. “I’m thinking of a situation where the world doesn’t need one big language model. We have so many,” she said. “If there’s a factory making language models, you need this kind of technology. It has one base model that can quickly adapt to new domains.”

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Quanta Magazine – https://www.quantamagazine.org/how-selective-forgetting-can-help-ai-learn-better-20240228/

Tags: Forgettingscienceselective
Previous Post

several migrants died off Senegal’s northern coast trying to reach Europe

Next Post

‘More awake and more alive’

World-Building With Daisies – 032C

Crafting Vibrant Worlds: The Art of Building with Daisies

September 29, 2025
Milei Vowed to Fix Argentina’s Economy. Then Came a New Crisis. – The New York Times

Milei Vowed to Fix Argentina’s Economy. Then Came a New Crisis. – The New York Times

September 29, 2025
Good Deed Entertainment Acquires Worldwide Rights To Liza Mandelup’s Documentary ‘Caterpillar’ – Deadline

Good Deed Entertainment Lands Global Rights to Liza Mandelup’s Captivating Documentary ‘Caterpillar

September 29, 2025
Report finds increased nitrates as fertilizer application poses costs to public health and farmers – Wisconsin Examiner

Rising Nitrate Levels from Fertilizer Use Threaten Public Health and Farmers’ Livelihoods

September 29, 2025
Michigan church shooting suspect went on anti-LDS tirade, political candidate said – Detroit Free Press

Michigan church shooting suspect went on anti-LDS tirade, political candidate said – Detroit Free Press

September 29, 2025
WEEKEND SCENE: Wolves, salmon, stormwater, more at 2025 West Seattle Ecology Fair, report #1 – West Seattle Blog…

Discover Wolves, Salmon, Stormwater, and More at the 2025 West Seattle Ecology Fair!

September 29, 2025
The Real Threat to Faith Isn’t Science – The Gospel Coalition

The Real Threat to Faith Isn’t Science – The Gospel Coalition

September 29, 2025
Trump Distorts Facts on Autism, Tylenol and Vaccines, Scientists Say – hepmag.com

Scientists Disprove Claims Connecting Autism to Tylenol and Vaccines

September 29, 2025
Our Favorite Store-Bought Coleslaw Can Be Found At Many Grocery Stores – Yahoo

Our Favorite Store-Bought Coleslaw Can Be Found At Many Grocery Stores – Yahoo

September 29, 2025
Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

September 29, 2025

Categories

Archives

September 2025
M T W T F S S
1234567
891011121314
15161718192021
22232425262728
2930  
« Aug    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (842)
  • Economy (864)
  • Entertainment (21,738)
  • General (17,314)
  • Health (9,907)
  • Lifestyle (876)
  • News (22,149)
  • People (865)
  • Politics (874)
  • Science (16,073)
  • Sports (21,363)
  • Technology (15,846)
  • World (846)

Recent News

World-Building With Daisies – 032C

Crafting Vibrant Worlds: The Art of Building with Daisies

September 29, 2025
Milei Vowed to Fix Argentina’s Economy. Then Came a New Crisis. – The New York Times

Milei Vowed to Fix Argentina’s Economy. Then Came a New Crisis. – The New York Times

September 29, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version