* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Wednesday, October 1, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Penn State-Themed Restaurant and Entertainment Spot Happy Valley Live Set to Open in State College – StateCollege.com

    Penn State-Themed Restaurant and Entertainment Spot Happy Valley Live Set to Open in State College – StateCollege.com

    The Police Made Chart History With This 1979 Hit Nearly 50 Years Ago – Yahoo

    How The Police Changed Music Forever with Their Iconic 1979 Hit Nearly 50 Years Ago

    Good Deed Entertainment Acquires Worldwide Rights To Liza Mandelup’s Documentary ‘Caterpillar’ – Deadline

    Good Deed Entertainment Lands Global Rights to Liza Mandelup’s Captivating Documentary ‘Caterpillar

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Jussie Smollett Claims He Was ‘Disrespected’ on the ‘Special Forces’ Season Premiere – Yahoo

    Jussie Smollett Opens Up About Feeling ‘Disrespected’ During the ‘Special Forces’ Season Premiere

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Steampunk Metal Oval Technology Sense Sunglasses Personality Handmade Chain Multicolor Sunglasses UV400 – The San Joaquin Valley Sun

    Steampunk Metal Oval Sunglasses with Handmade Multicolor Chain – Bold UV400 Protection and Unique Style

    STELLA Automotive AI Appoints Fred Seidelman as Chief Technology Officer – Yahoo Finance

    STELLA Automotive AI Appoints Fred Seidelman as New Chief Technology Officer

    Saving Energy and Money with Smart Technology – Terms of Service with Clare Duffy – Podcast on CNN Podcasts – CNN

    Saving Energy and Money with Smart Technology – Terms of Service with Clare Duffy – Podcast on CNN Podcasts – CNN

    Four Strategic Signals Technology Leaders Are Tuning In To – SPONSOR CONTENT FROM ARM – Harvard Business Review

    Four Essential Strategic Signals Every Technology Leader Should Watch

    Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

    Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

    Why I gave the world wide web away for free | Tim Berners-Lee – The Guardian

    Why I Decided to Make the World Wide Web Free for Everyone | Tim Berners-Lee

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Penn State-Themed Restaurant and Entertainment Spot Happy Valley Live Set to Open in State College – StateCollege.com

    Penn State-Themed Restaurant and Entertainment Spot Happy Valley Live Set to Open in State College – StateCollege.com

    The Police Made Chart History With This 1979 Hit Nearly 50 Years Ago – Yahoo

    How The Police Changed Music Forever with Their Iconic 1979 Hit Nearly 50 Years Ago

    Good Deed Entertainment Acquires Worldwide Rights To Liza Mandelup’s Documentary ‘Caterpillar’ – Deadline

    Good Deed Entertainment Lands Global Rights to Liza Mandelup’s Captivating Documentary ‘Caterpillar

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Danielle Fishel Explains Why Being on “DWTS” Makes Her Feel ‘Like It’s 1994 Again’ Filming “Boy Meets World” (Exclusive) – Yahoo

    Jussie Smollett Claims He Was ‘Disrespected’ on the ‘Special Forces’ Season Premiere – Yahoo

    Jussie Smollett Opens Up About Feeling ‘Disrespected’ During the ‘Special Forces’ Season Premiere

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

    TicketSmarter Fall Entertainment Guide – Eastern Illinois University Athletics

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Steampunk Metal Oval Technology Sense Sunglasses Personality Handmade Chain Multicolor Sunglasses UV400 – The San Joaquin Valley Sun

    Steampunk Metal Oval Sunglasses with Handmade Multicolor Chain – Bold UV400 Protection and Unique Style

    STELLA Automotive AI Appoints Fred Seidelman as Chief Technology Officer – Yahoo Finance

    STELLA Automotive AI Appoints Fred Seidelman as New Chief Technology Officer

    Saving Energy and Money with Smart Technology – Terms of Service with Clare Duffy – Podcast on CNN Podcasts – CNN

    Saving Energy and Money with Smart Technology – Terms of Service with Clare Duffy – Podcast on CNN Podcasts – CNN

    Four Strategic Signals Technology Leaders Are Tuning In To – SPONSOR CONTENT FROM ARM – Harvard Business Review

    Four Essential Strategic Signals Every Technology Leader Should Watch

    Virginia Tech hosts annual New Music + Technology Festival this week – Cardinal News

    Virginia Tech Kicks Off Exciting Annual New Music and Technology Festival This Week

    Why I gave the world wide web away for free | Tim Berners-Lee – The Guardian

    Why I Decided to Make the World Wide Web Free for Everyone | Tim Berners-Lee

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Science

Risky Giant Steps Can Solve Optimization Problems Faster

August 12, 2023
in Science
Risky Giant Steps Can Solve Optimization Problems Faster
Share on FacebookShare on Twitter

“It turns out that we did not have full understanding” of the theory behind gradient descent, said Shuvomoy Das Gupta, an optimization researcher at the Massachusetts Institute of Technology. Now, he said, we’re “closer to understanding what gradient descent is doing.”

The technique itself is deceptively simple. It uses something called a cost function, which looks like a smooth, curved line meandering up and down across a graph. For any point on that line, the height represents cost in some way — how much time, energy or error the operation will incur when tuned to a specific setting. The higher the point, the farther from ideal the system is. Naturally, you want to find the lowest point on this line, where the cost is smallest.

Gradient descent algorithms feel their way to the bottom by picking a point and calculating the slope (or gradient) of the curve around it, then moving in the direction where the slope is steepest. Imagine this as feeling your way down a mountain in the dark. You may not know exactly where to move, how long you’ll have to hike or how close to sea level you will ultimately get, but if you head down the sharpest descent, you should eventually arrive at the lowest point in the area.

Unlike the metaphorical mountaineer, optimization researchers can program their gradient descent algorithms to take steps of any size. Giant leaps are tempting but also risky, as they could overshoot the answer. Instead, the field’s conventional wisdom for decades has been to take baby steps. In gradient descent equations, this means a step size no bigger than 2, though no one could prove that smaller step sizes were always better.

With advances in computer-aided proof techniques, optimization theorists have begun testing more extreme techniques. In one study, first posted in 2022 and recently published in Mathematical Programming, Das Gupta and others tasked a computer with finding the best step lengths for an algorithm restricted to running only 50 steps — a sort of meta-optimization problem, since it was trying to optimize optimization. They found that the most optimal 50 steps varied significantly in length, with one step in the middle of the sequence reaching nearly to length 37, far above the typical cap of length 2.

The findings suggested that optimization researchers had missed something. Intrigued, Grimmer sought to turn Das Gupta’s numerical results into a more general theorem. To get past an arbitrary cap of 50 steps, Grimmer explored what the optimal step lengths would be for a sequence that could repeat, getting closer to the optimal answer with each repetition. He ran the computer through millions of permutations of step length sequences, helping to find those that converged on the answer the fastest.

Grimmer found that the fastest sequences always had one thing in common: The middle step was always a big one. Its size depended on the number of steps in the repeating sequence. For a three-step sequence, the big step had length 4.9. For a 15-step sequence, the algorithm recommended one step of length 29.7. And for a 127-step sequence, the longest one tested, the big central leap was a whopping 370. At first that sounds like an absurdly large number, Grimmer said, but there were enough total steps to make up for that giant leap, so even if you blew past the bottom, you could still make it back quickly. His paper showed that this sequence can arrive at the optimal point nearly three times faster than it would by taking constant baby steps. “Sometimes, you should really overcommit,” he said.

This cyclical approach represents a different way of thinking of gradient descent, said Aymeric Dieuleveut, an optimization researcher at the École Polytechnique in Palaiseau, France. “This intuition, that I should think not step by step, but as a number of steps consecutively — I think this is something that many people ignore,” he said. “It’s not the way it’s taught.” (Grimmer notes that this reframing was also proposed for a similar class of problems in a 2018 master’s thesis by Jason Altschuler, an optimization researcher now at the University of Pennsylvania.)

However, while these insights may change how researchers think about gradient descent, they likely won’t change how the technique is currently used. Grimmer’s paper focused only on smooth functions, which have no sharp kinks, and convex functions, which are shaped like a bowl and only have one optimal value at the bottom. These kinds of functions are fundamental to theory but less relevant in practice; the optimization programs machine learning researchers use are usually much more complicated. These require versions of gradient descent that have “so many bells and whistles, and so many nuances,” Grimmer said.

Some of these souped-up techniques can go faster than Grimmer’s big-step approach, said Gauthier Gidel, an optimization and machine learning researcher at the University of Montreal. But these techniques come at an additional operational cost, so the hope has been that regular gradient descent could win out with the right combination of step sizes. Unfortunately, the new study’s threefold speedup isn’t enough.

“It shows a marginal improvement,” Gidel said. “But I guess the real question is: Can we really close this gap?”

The results also raise an additional theoretical mystery that has kept Grimmer up at night. Why did the ideal patterns of step sizes all have such a symmetric shape? Not only is the biggest step always smack in the center, but the same pattern appears on either side of it: Keep zooming in and subdividing the sequence, he said, and you get an “almost fractal pattern” of bigger steps surrounded by smaller steps. The repetition suggests an underlying structure governing the best solutions that no one has yet managed to explain. But Grimmer, at least, is hopeful.

“If I can’t crack it, someone else will,” he said.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Quanta Magazine – https://www.quantamagazine.org/risky-giant-steps-can-solve-optimization-problems-faster-20230811/

Tags: giantriskyscience
Previous Post

FBI Fatally Shoots Man Threatening President Biden After Social Media Alert

Next Post

Predicting When Roman Reigns, Brock Lesnar and Logan Paul Will Wrestle Again in 2023

The billion-year reign of fungi that predated plants and made Earth livable – ScienceDaily

October 1, 2025
We Asked an Entomologist to Fact-Check the Science of ‘Alien: Earth’ – Esquire

We Asked an Entomologist to Fact-Check the Science of ‘Alien: Earth’ – Esquire

October 1, 2025
Building Better Leaders Through the Science of Happiness – Pace University

Unlocking Leadership Potential: How the Science of Happiness Transforms Leaders

October 1, 2025
Viral ‘Goth house’ for sale just in time for Halloween – The Weekly Journal

Step Inside the Spooky ‘Goth House’ Perfect for Halloween!

October 1, 2025
Steampunk Metal Oval Technology Sense Sunglasses Personality Handmade Chain Multicolor Sunglasses UV400 – The San Joaquin Valley Sun

Steampunk Metal Oval Sunglasses with Handmade Multicolor Chain – Bold UV400 Protection and Unique Style

October 1, 2025
Atlanta Braves Manager Brian Snitker Moving to ‘Advisory Role’ – FOX Sports

Atlanta Braves Manager Brian Snitker Moving to ‘Advisory Role’ – FOX Sports

October 1, 2025
The sleepy SF neighborhood that just became one of the world’s coolest – SFGATE

The sleepy SF neighborhood that just became one of the world’s coolest – SFGATE

October 1, 2025
The “stuck economy,” tariffs and Wall Street – marketplace.org

How Tariffs and Wall Street Are Strangling the Economy’s Growth

October 1, 2025
LSU Theatre’s season opener: A handkerchief discovered by a lovesick suitor adds up to hilarity – The Advocate

Love, Laughter, and a Lost Handkerchief: LSU Theatre’s Season Opener Promises Hilarious Fun

October 1, 2025
Rural Health Transformation (RHT) Program – Centers for Medicare & Medicaid Services | CMS (.gov)

Rural Health Transformation (RHT) Program – Centers for Medicare & Medicaid Services | CMS (.gov)

October 1, 2025

Categories

Archives

October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  
« Sep    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (846)
  • Economy (866)
  • Entertainment (21,740)
  • General (17,353)
  • Health (9,910)
  • Lifestyle (880)
  • News (22,149)
  • People (869)
  • Politics (877)
  • Science (16,077)
  • Sports (21,367)
  • Technology (15,850)
  • World (849)

Recent News

The billion-year reign of fungi that predated plants and made Earth livable – ScienceDaily

October 1, 2025
We Asked an Entomologist to Fact-Check the Science of ‘Alien: Earth’ – Esquire

We Asked an Entomologist to Fact-Check the Science of ‘Alien: Earth’ – Esquire

October 1, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version