Non-Google search engines blocked from showing Reddit results

Recent updates to Reddit's Content Policy have resulted in non-Google search engines being blocked from displaying recent Reddit results. This change stems from Reddit's new robots.txt file, which prohibits crawling its site without an agreement that aligns with its rules, particularly regarding the use of Reddit content for AI training. As a consequence, search engines like Bing, DuckDuckGo, and Mojeek are showing minimal or no recent Reddit results, while Google remains the only search engine able to access this content. Reddit's decision was influenced by an increase in commercial entities scraping its content without adhering to its terms. Although Reddit has expressed willingness to collaborate with search engines, it has struggled to reach agreements, particularly with Microsoft, which respects the new crawling restrictions. Reddit's CEO has indicated that the platform is open to partnerships but is firm on protecting its content from unregulated use. This situation has inadvertently strengthened Google's dominance in the search engine market, raising concerns about the implications for smaller search engines and websites that rely on traffic from search results. The ongoing changes reflect broader issues regarding content ownership and the use of public data in AI development, with Reddit emphasizing the need for constraints on how its content is utilized.