* . *
  • About
  • Advertise
  • Privacy & Policy
  • Contact
Friday, May 30, 2025
Earth-News
  • Home
  • Business
  • Entertainment
    Allied Gaming & Entertainment Receives Expected Nasdaq Notice Regarding Delayed Quarterly Report – Business Wire

    Allied Gaming & Entertainment Faces Nasdaq Notice Over Delayed Quarterly Report

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Aziz Ansari made Keanu Reeves Indian food so he wouldn’t feel ‘freaked out’ directing him in Good Fortune (exclusive) – Entertainment Weekly

    Aziz Ansari Whips Up Indian Cuisine for Keanu Reeves to Ease Directing Jitters in Good Fortune

    Cassie Ventura welcomes third child – crossroadstoday.com

    Cassie Ventura Joyfully Welcomes Her Third Child!

    Drew Brees opens Surge Entertainment Center in Metairie hot spot for new businesses – NOLA.com

    Drew Brees Unveils Exciting New Surge Entertainment Center in Metairie!

    Will Gio Stay Or Go On May 27 General Hospital? – Yahoo

    Will Gio Make a Shocking Exit on May 27th’s General Hospital

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    4Liberty Announces Collaboration with Itron to Drive Innovation in Utility Technology Adoption and Optimization – FinancialContent

    4Liberty Partners with Itron to Revolutionize Utility Technology and Boost Innovation

    Q1 2025 Quantum Technology Investment: What’s Driving the Surge in Quantum Investment? – The Quantum Insider

    Unleashing the Future: Exploring the Surge in Quantum Technology Investments for Q1 2025

    MIT Team Releases Tempting Report on Electric Aircraft Technology – AVweb

    MIT Team Releases Tempting Report on Electric Aircraft Technology – AVweb

    MSE, DXC Technology expand tech partnership – Sports Business Journal

    MSE, DXC Technology expand tech partnership – Sports Business Journal

    Technology Driving Performance: How Elite MA Plans Leverage Data, Care Coordination, and Advanced Analytics to Thrive Amid CMS’s Rigorous 2025 Standards, Black Book – Newswire.com

    Unlocking Success: How Top MA Plans Harness Data and Analytics to Excel Under CMS’s 2025 Standards

    Arqit acquires Ampliphae’s technology IP, enhancing its global encryption portfolio – GlobeNewswire

    Arqit Boosts Global Encryption Portfolio with Strategic Acquisition of Ampliphae’s Technology IP

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
  • Home
  • Business
  • Entertainment
    Allied Gaming & Entertainment Receives Expected Nasdaq Notice Regarding Delayed Quarterly Report – Business Wire

    Allied Gaming & Entertainment Faces Nasdaq Notice Over Delayed Quarterly Report

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Weekly Entertainment Report May 29-June 1: Where to find lively arts, music, and shows – Manchester Ink Link

    Aziz Ansari made Keanu Reeves Indian food so he wouldn’t feel ‘freaked out’ directing him in Good Fortune (exclusive) – Entertainment Weekly

    Aziz Ansari Whips Up Indian Cuisine for Keanu Reeves to Ease Directing Jitters in Good Fortune

    Cassie Ventura welcomes third child – crossroadstoday.com

    Cassie Ventura Joyfully Welcomes Her Third Child!

    Drew Brees opens Surge Entertainment Center in Metairie hot spot for new businesses – NOLA.com

    Drew Brees Unveils Exciting New Surge Entertainment Center in Metairie!

    Will Gio Stay Or Go On May 27 General Hospital? – Yahoo

    Will Gio Make a Shocking Exit on May 27th’s General Hospital

  • General
  • Health
  • News

    Cracking the Code: Why China’s Economic Challenges Aren’t Shaking Markets, Unlike America’s” – Bloomberg

    Trump’s Narrow Window to Spread the Truth About Harris

    Trump’s Narrow Window to Spread the Truth About Harris

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    Israel-Gaza war live updates: Hamas leader Ismail Haniyeh assassinated in Iran, group says

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    PAP Boss to Niger Delta Youths, Stay Away from the Protest

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Court Restricts Protests In Lagos To Freedom, Peace Park

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Fans React to Jazz Jennings’ Inspiring Weight Loss Journey

    Trending Tags

    • Trump Inauguration
    • United Stated
    • White House
    • Market Stories
    • Election Results
  • Science
  • Sports
  • Technology
    4Liberty Announces Collaboration with Itron to Drive Innovation in Utility Technology Adoption and Optimization – FinancialContent

    4Liberty Partners with Itron to Revolutionize Utility Technology and Boost Innovation

    Q1 2025 Quantum Technology Investment: What’s Driving the Surge in Quantum Investment? – The Quantum Insider

    Unleashing the Future: Exploring the Surge in Quantum Technology Investments for Q1 2025

    MIT Team Releases Tempting Report on Electric Aircraft Technology – AVweb

    MIT Team Releases Tempting Report on Electric Aircraft Technology – AVweb

    MSE, DXC Technology expand tech partnership – Sports Business Journal

    MSE, DXC Technology expand tech partnership – Sports Business Journal

    Technology Driving Performance: How Elite MA Plans Leverage Data, Care Coordination, and Advanced Analytics to Thrive Amid CMS’s Rigorous 2025 Standards, Black Book – Newswire.com

    Unlocking Success: How Top MA Plans Harness Data and Analytics to Excel Under CMS’s 2025 Standards

    Arqit acquires Ampliphae’s technology IP, enhancing its global encryption portfolio – GlobeNewswire

    Arqit Boosts Global Encryption Portfolio with Strategic Acquisition of Ampliphae’s Technology IP

    Trending Tags

    • Nintendo Switch
    • CES 2017
    • Playstation 4 Pro
    • Mark Zuckerberg
No Result
View All Result
Earth-News
No Result
View All Result
Home Technology

Search engines that don’t pay up can’t index Reddit content

July 24, 2024
in Technology
Search engines that don’t pay up can’t index Reddit content
Share on FacebookShare on Twitter

Will Shanklin

When Reddit said last month that it would block unauthorized data scraping from its site, everyone’s (rightful) first reaction was “AI, AI, AI.” However, now that the change has taken effect, chatbot makers aren’t the only ones being locked out. The widely used forum also appears to be blocking all search engines other than Google, which reportedly inked a deal earlier this year with Reddit worth $60 million annually.

404 Media reported on Wednesday (and Engadget confirmed in our queries) that searching for Reddit results from the past week on rival engine Bing (using “site:reddit.com”) returns empty results. The publication reported that DuckDuckGo produced seven links without any descriptions, only providing the note, “We would like to show you a description here but the site won’t allow us.” The engine now appears to have removed even those, as our test only produced an empty page, reading, “no results found.”

When Reddit said last month that it would update its Robots Exclusion Protocol (robots.txt) to block automated data scraping, it’s now apparent that it wasn’t only meant to thwart AI companies like Perplexity and its controversial “answer engine.” Currently, Google appears to be the only search engine allowed to crawl Reddit and produce results from “the front page of the internet.”

Ironically, part of the forum website’s robots.txt file reads, “Reddit believes in an open internet, but not the misuse of public content.” The file for Reddit now essentially says, “Do not scrape.” Apparently, it now considers search engines that don’t buy into exclusive deals to be misusing its content.

The ubiquitous robots.txt is the web standard that communicates which parts of a site can be crawled. Although many crawlers are known to ignore its instructions, Google’s standard procedure is to respect it. So, on the technical side, the companies in cahoots on the lucrative deal appear to have deployed some manual override.

Of course, the saga is a trickle-down effect of AI chatbots scraping the live web for results. With courts slow to determine how much of the open web is fair use to train chatbots on, companies like Reddit, whose bottom lines now depend on safeguarding their data from those who don’t pay, are building walls at the expense of the open web. (Although, given the integral role Microsoft has played in this AI era, cozying up with OpenAI early on, it seems ironic that Bing finds itself on the losing end of at least one aspect of the fallout.)

Colin Hayhurst, CEO of lesser-known “no-tracking” search engine Mojeek, told 404 Media that Reddit is “killing everything for search but Google.” In addition, the executive said his attempts to contact Reddit were ignored. “It’s never happened to us before,” he said. “Because this happens to us, we get blocked, usually because of ignorance or stupidity or whatever, and when we contact the site you certainly can get that resolved, but we’ve never had no reply from anybody before.”

Engadget asked Google and Reddit for comment and confirmation, but we hadn’t heard back by publication. 404 Media reported running into a similar wall of silence from the companies.

Reddit has made no secret of its desire to block AI companies from scraping its treasure trove of data in this burgeoning age of AI. Last year, CEO Steve Huffman risked alienating large portions of its user base by blocking third-party API requests, leading to the demise of beloved apps like Christian Selig’s Apollo. Despite widespread protests among moderators and forum-goers, the company only temporarily lost negligible numbers of users.

The gamble appeared to pay off, and Reddit recovered. It went public in March.

>>> Read full article>>>
Copyright for syndicated content belongs to the linked Source : Engadget – https://www.engadget.com/search-engines-that-dont-pay-up-cant-index-reddit-content-172949170.html?src=rss

Tags: EnginesSearchtechnology
Previous Post

CrowdStrike offers a $10 Uber Eats card to say sorry before pulling the offer

Next Post

Meta takes down 63,000 Instagram accounts linked to extortion scams

Even birds can’t out-fly climate change – Phys.org

Even birds can’t out-fly climate change – Phys.org

May 30, 2025
Opinion | How China’s maritime strategy is swapping force for research diplomacy – South China Morning Post

Opinion | How China’s maritime strategy is swapping force for research diplomacy – South China Morning Post

May 30, 2025
Newly discovered ‘ghost’ lineage linked to ancient mystery population in Tibet, DNA study finds – Live Science

Unveiling the Secrets of Tibet: A Fascinating ‘Ghost’ Lineage Revealed by DNA Study

May 30, 2025
‘Wicked,’ gardening and Amy Sedaris: 7 events this week – Albuquerque Journal

‘Wicked,’ gardening and Amy Sedaris: 7 events this week – Albuquerque Journal

May 30, 2025
Yankees have a busload of momentum heading into World Series rematch against Dodgers – AP News

Yankees have a busload of momentum heading into World Series rematch against Dodgers – AP News

May 30, 2025
Economists blame Trump tariffs, AI explosion for threatening global economy – theregister.com

How Trump’s Tariffs and the AI Boom Are Shaking Up the Global Economy

May 30, 2025
Allied Gaming & Entertainment Receives Expected Nasdaq Notice Regarding Delayed Quarterly Report – Business Wire

Allied Gaming & Entertainment Faces Nasdaq Notice Over Delayed Quarterly Report

May 30, 2025
UMC Health System, Texas Tech HSC to honor endowed chairs, professors – Lubbock Avalanche-Journal

Celebrating Excellence: UMC Health System and Texas Tech HSC Honor Distinguished Professors and Endowed Chairs

May 30, 2025
Paris on Politics: Israel-Gaza conflict gets personal in Chicago’s City Council – FOX 32 Chicago

Paris on Politics: Israel-Gaza conflict gets personal in Chicago’s City Council – FOX 32 Chicago

May 30, 2025
Anduril and Meta Team Up to Transform XR for the American Military – Anduril

Revolutionizing Military Training: Anduril and Meta Join Forces to Enhance XR Technology

May 30, 2025

Categories

Archives

May 2025
MTWTFSS
 1234
567891011
12131415161718
19202122232425
262728293031 
« Apr    
Earth-News.info

The Earth News is an independent English-language daily published Website from all around the World News

Browse by Category

  • Business (20,132)
  • Ecology (651)
  • Economy (664)
  • Entertainment (21,571)
  • General (15,248)
  • Health (9,706)
  • Lifestyle (666)
  • News (22,149)
  • People (666)
  • Politics (672)
  • Science (15,886)
  • Sports (21,170)
  • Technology (15,651)
  • World (652)

Recent News

Even birds can’t out-fly climate change – Phys.org

Even birds can’t out-fly climate change – Phys.org

May 30, 2025
Opinion | How China’s maritime strategy is swapping force for research diplomacy – South China Morning Post

Opinion | How China’s maritime strategy is swapping force for research diplomacy – South China Morning Post

May 30, 2025
  • About
  • Advertise
  • Privacy & Policy
  • Contact

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

No Result
View All Result

© 2023 earth-news.info

Go to mobile version