* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Friday, October 24, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    ‘Chainsaw Man — The Movie: Reze Arc’ Review: Hit Manga Gets an Ultra-Violent, Surprisingly Emotional Big-Screen Adaptation – Yahoo

    Chainsaw Man – The Movie: Reze Arc Review: A Brutal and Unexpectedly Emotional Big-Screen Adaptation

    Reba McEntire Details Personal Relationship With Late Stepson Brandon Blackstock – KNDU

    Reba McEntire Shares Emotional Tribute to Her Late Stepson Brandon Blackstock

    Sacramento city leaders approve adding 2 entertainment zones in midtown – CBS News

    Sacramento City Leaders Approve Two Thrilling New Entertainment Zones in Midtown

    AMC brings first new Dolby Experience to Gwinnett since 2017 – Wyoming News Now

    AMC Launches First New Dolby Experience in Gwinnett Since 2017

    Hetzel Design: blending architecture and entertainment – Blooloop

    Hetzel Design: Where Architecture and Entertainment Unite in Perfect Harmony

    Country music legend rushed to hospital year after heart surgery. Here’s what we know – PennLive.com

    Country Music Legend Rushed to Hospital One Year After Heart Surgery – What’s Happening Now?

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Project underway to upgrade technology on 911 towers in Kanawha County – WCHS

    Kanawha County Launches Major Upgrade to 911 Tower Technology

    Next steps: Technology opens new options for greater mobility – Missouri Independent

    Next Steps: How Technology is Opening Exciting New Doors to Greater Mobility

    Rydberg Technologies Inc. Announces Launch of Rydberg Photonics in Berlin – The Quantum Insider

    Rydberg Technologies Launches Exciting New Photonics Division in Berlin

    A look into new technology at Columbia University that could help prevent a dangerous pregnancy complication – ABC7 New York

    A look into new technology at Columbia University that could help prevent a dangerous pregnancy complication – ABC7 New York

    Office Technology: Dealers’ Managed IT Revenue up Nearly 30% – The Cannata Report –

    Office Technology: Dealers’ Managed IT Revenue up Nearly 30% – The Cannata Report –

    3 E Network Technology Group Limited Closes $1.5 Million Convertible Promissory Note Offering – Quiver Quantitative

    3 E Network Technology Group Limited Closes $1.5 Million Convertible Promissory Note Offering – Quiver Quantitative

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    ‘Chainsaw Man — The Movie: Reze Arc’ Review: Hit Manga Gets an Ultra-Violent, Surprisingly Emotional Big-Screen Adaptation – Yahoo

    Chainsaw Man – The Movie: Reze Arc Review: A Brutal and Unexpectedly Emotional Big-Screen Adaptation

    Reba McEntire Details Personal Relationship With Late Stepson Brandon Blackstock – KNDU

    Reba McEntire Shares Emotional Tribute to Her Late Stepson Brandon Blackstock

    Sacramento city leaders approve adding 2 entertainment zones in midtown – CBS News

    Sacramento City Leaders Approve Two Thrilling New Entertainment Zones in Midtown

    AMC brings first new Dolby Experience to Gwinnett since 2017 – Wyoming News Now

    AMC Launches First New Dolby Experience in Gwinnett Since 2017

    Hetzel Design: blending architecture and entertainment – Blooloop

    Hetzel Design: Where Architecture and Entertainment Unite in Perfect Harmony

    Country music legend rushed to hospital year after heart surgery. Here’s what we know – PennLive.com

    Country Music Legend Rushed to Hospital One Year After Heart Surgery – What’s Happening Now?

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    Project underway to upgrade technology on 911 towers in Kanawha County – WCHS

    Kanawha County Launches Major Upgrade to 911 Tower Technology

    Next steps: Technology opens new options for greater mobility – Missouri Independent

    Next Steps: How Technology is Opening Exciting New Doors to Greater Mobility

    Rydberg Technologies Inc. Announces Launch of Rydberg Photonics in Berlin – The Quantum Insider

    Rydberg Technologies Launches Exciting New Photonics Division in Berlin

    A look into new technology at Columbia University that could help prevent a dangerous pregnancy complication – ABC7 New York

    A look into new technology at Columbia University that could help prevent a dangerous pregnancy complication – ABC7 New York

    Office Technology: Dealers’ Managed IT Revenue up Nearly 30% – The Cannata Report –

    Office Technology: Dealers’ Managed IT Revenue up Nearly 30% – The Cannata Report –

    3 E Network Technology Group Limited Closes $1.5 Million Convertible Promissory Note Offering – Quiver Quantitative

    3 E Network Technology Group Limited Closes $1.5 Million Convertible Promissory Note Offering – Quiver Quantitative

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Science

Risky Giant Steps Can Solve Optimization Problems Faster

August 12, 2023
in Science
Risky Giant Steps Can Solve Optimization Problems Faster
Share on FacebookShare on Twitter

“It turns out that we did not have full understanding” of the theory behind gradient descent, said Shuvomoy Das Gupta, an optimization researcher at the Massachusetts Institute of Technology. Now, he said, we’re “closer to understanding what gradient descent is doing.”

The technique itself is deceptively simple. It uses something called a cost function, which looks like a smooth, curved line meandering up and down across a graph. For any point on that line, the height represents cost in some way — how much time, energy or error the operation will incur when tuned to a specific setting. The higher the point, the farther from ideal the system is. Naturally, you want to find the lowest point on this line, where the cost is smallest.

Gradient descent algorithms feel their way to the bottom by picking a point and calculating the slope (or gradient) of the curve around it, then moving in the direction where the slope is steepest. Imagine this as feeling your way down a mountain in the dark. You may not know exactly where to move, how long you’ll have to hike or how close to sea level you will ultimately get, but if you head down the sharpest descent, you should eventually arrive at the lowest point in the area.

Unlike the metaphorical mountaineer, optimization researchers can program their gradient descent algorithms to take steps of any size. Giant leaps are tempting but also risky, as they could overshoot the answer. Instead, the field’s conventional wisdom for decades has been to take baby steps. In gradient descent equations, this means a step size no bigger than 2, though no one could prove that smaller step sizes were always better.

With advances in computer-aided proof techniques, optimization theorists have begun testing more extreme techniques. In one study, first posted in 2022 and recently published in Mathematical Programming, Das Gupta and others tasked a computer with finding the best step lengths for an algorithm restricted to running only 50 steps — a sort of meta-optimization problem, since it was trying to optimize optimization. They found that the most optimal 50 steps varied significantly in length, with one step in the middle of the sequence reaching nearly to length 37, far above the typical cap of length 2.

The findings suggested that optimization researchers had missed something. Intrigued, Grimmer sought to turn Das Gupta’s numerical results into a more general theorem. To get past an arbitrary cap of 50 steps, Grimmer explored what the optimal step lengths would be for a sequence that could repeat, getting closer to the optimal answer with each repetition. He ran the computer through millions of permutations of step length sequences, helping to find those that converged on the answer the fastest.

Grimmer found that the fastest sequences always had one thing in common: The middle step was always a big one. Its size depended on the number of steps in the repeating sequence. For a three-step sequence, the big step had length 4.9. For a 15-step sequence, the algorithm recommended one step of length 29.7. And for a 127-step sequence, the longest one tested, the big central leap was a whopping 370. At first that sounds like an absurdly large number, Grimmer said, but there were enough total steps to make up for that giant leap, so even if you blew past the bottom, you could still make it back quickly. His paper showed that this sequence can arrive at the optimal point nearly three times faster than it would by taking constant baby steps. “Sometimes, you should really overcommit,” he said.

This cyclical approach represents a different way of thinking of gradient descent, said Aymeric Dieuleveut, an optimization researcher at the École Polytechnique in Palaiseau, France. “This intuition, that I should think not step by step, but as a number of steps consecutively — I think this is something that many people ignore,” he said. “It’s not the way it’s taught.” (Grimmer notes that this reframing was also proposed for a similar class of problems in a 2018 master’s thesis by Jason Altschuler, an optimization researcher now at the University of Pennsylvania.)

However, while these insights may change how researchers think about gradient descent, they likely won’t change how the technique is currently used. Grimmer’s paper focused only on smooth functions, which have no sharp kinks, and convex functions, which are shaped like a bowl and only have one optimal value at the bottom. These kinds of functions are fundamental to theory but less relevant in practice; the optimization programs machine learning researchers use are usually much more complicated. These require versions of gradient descent that have “so many bells and whistles, and so many nuances,” Grimmer said.

Some of these souped-up techniques can go faster than Grimmer’s big-step approach, said Gauthier Gidel, an optimization and machine learning researcher at the University of Montreal. But these techniques come at an additional operational cost, so the hope has been that regular gradient descent could win out with the right combination of step sizes. Unfortunately, the new study’s threefold speedup isn’t enough.

“It shows a marginal improvement,” Gidel said. “But I guess the real question is: Can we really close this gap?”

The results also raise an additional theoretical mystery that has kept Grimmer up at night. Why did the ideal patterns of step sizes all have such a symmetric shape? Not only is the biggest step always smack in the center, but the same pattern appears on either side of it: Keep zooming in and subdividing the sequence, he said, and you get an “almost fractal pattern” of bigger steps surrounded by smaller steps. The repetition suggests an underlying structure governing the best solutions that no one has yet managed to explain. But Grimmer, at least, is hopeful.

“If I can’t crack it, someone else will,” he said.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Quanta Magazine – https://www.quantamagazine.org/risky-giant-steps-can-solve-optimization-problems-faster-20230811/

Tags: giantriskyscience
Previous Post

FBI Fatally Shoots Man Threatening President Biden After Social Media Alert

Next Post

Predicting When Roman Reigns, Brock Lesnar and Logan Paul Will Wrestle Again in 2023

Orange Cat Celebrates Halloween by Putting Candy Corn in Her Water Bowl – Yahoo

Orange Cat’s Hilarious Halloween Trick: Candy Corn in Her Water Bowl!

October 24, 2025
Project underway to upgrade technology on 911 towers in Kanawha County – WCHS

Kanawha County Launches Major Upgrade to 911 Tower Technology

October 24, 2025
Sports gambling indictment revives tough questions about integrity – Axios

Sports Gambling Indictment Sparks Renewed Debate Over Integrity

October 23, 2025
Former world chess champion Vladimir Kramnik investigated for bullying following death of grandmaster Daniel Naroditsky – CNN

Former World Chess Champion Vladimir Kramnik Under Investigation Amid Tragic Loss of Grandmaster Daniel Naroditsky

October 23, 2025
Trump approval on economy hits new low: Quinnipiac poll – The Hill

Trump’s Economic Approval Hits Historic Low in Shocking New Poll

October 23, 2025
‘Chainsaw Man — The Movie: Reze Arc’ Review: Hit Manga Gets an Ultra-Violent, Surprisingly Emotional Big-Screen Adaptation – Yahoo

Chainsaw Man – The Movie: Reze Arc Review: A Brutal and Unexpectedly Emotional Big-Screen Adaptation

October 23, 2025
WHO launches new country guidance for health emergency coordination – World Health Organization (WHO)

WHO Launches Bold New Global Guidelines to Revolutionize Health Emergency Coordination

October 23, 2025
Gettysburg suffers damage as advocates ask public to be vigilant during shutdown – CNN

Gettysburg Endures Damage as Advocates Call for Public Vigilance Amid Shutdown

October 23, 2025
Vibrio pectenicida strain FHCF-3 is a causative agent of sea star wasting disease – Nature

Vibrio pectenicida strain FHCF-3 is a causative agent of sea star wasting disease – Nature

October 23, 2025
Condensation defying gravity – European Space Agency

Condensation defying gravity – European Space Agency

October 23, 2025

Categories

Archives

October 2025
M T W T F S S
 12345
6789101112
13141516171819
20212223242526
2728293031  
« Sep    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (882)
  • Economy (904)
  • Entertainment (21,775)
  • General (17,765)
  • Health (9,945)
  • Lifestyle (917)
  • News (22,149)
  • People (904)
  • Politics (914)
  • Science (16,114)
  • Sports (21,404)
  • Technology (15,884)
  • World (887)

Recent News

Orange Cat Celebrates Halloween by Putting Candy Corn in Her Water Bowl – Yahoo

Orange Cat’s Hilarious Halloween Trick: Candy Corn in Her Water Bowl!

October 24, 2025
Project underway to upgrade technology on 911 towers in Kanawha County – WCHS

Kanawha County Launches Major Upgrade to 911 Tower Technology

October 24, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version