AI Crawling behavior - Cloudflare view

An analysis of AI crawler activity across 25% of the web using Cloudflare Radar data, highlighting observed behaviors, associated risks, and API access details.

Overview

I will be using Cloudflare Radar, which is a Creative Common licensed dataset and visualization tool.

My objective is to explain, based on Cloudflare’s visibility on 25% of the Internet, how we see AI Crawlers behave, and the risks and challenges it brings to site owners.

I will also show the API doc that enables to consume data.

Links

Tech stack