An Update on the scraper situation

Web scraping attacks targeting sites for AI training data have intensified over the past year, with coordinated requests from millions of unique IP addresses using residential proxies—compromised ordinary devices and media-streaming equipment that forward scraped content to central controllers without the owner's knowledge. While Google's 2025 takedown of the IPIDEA bot network temporarily reduced traffic at LWN, the problem has resumed and continues to grow, making it increasingly difficult to maintain the open web as attackers use fictional user-agent data and distributed tactics that make blocking ineffective.