Fighting AI and other Bots

At the moment AI and their malicious bots are the scourge of the internet. They ignore all the conventions and implicit rules built on trust over many years. Cloudflare has called out Perplexity on this abuse at Perplexity is using stealth, undeclared crawlers to evade website no-crawl directives. You can also read more on the impact of these bots at The crawl before the fall… of referrals: understanding AI’s impact on content providers. Read more