Anyone can call their bot "Googlebot". The address it comes from tells you if it really is. Here are the IP ranges of the crawlers that matter, straight from the lists their owners publish, checked every day.
Every list comes from the company that runs the crawler, downloaded every day. Use them to let real crawlers past a firewall or rate limit, or to catch fakes in your logs.
Search and Google's other crawlers
GooglebotGooglebot-ImageGoogle-InspectionToolGoogleOtherAdsBot-GoogleStorebot-GoogleSource: common-crawlers.json, special-crawlers.json, user-triggered-fetchers.json, user-triggered-fetchers-google.json
Bing search and Copilot (Yahoo search results come from Bing too)
bingbotBingPreviewAdIdxBotSource: bingbot.json
Siri, Spotlight and Safari suggestions
ApplebotSource: applebot.json
DuckDuckGo search and its AI answers
DuckDuckBotDuckAssistBotSource: duckduckbot.json
ChatGPT search, browsing and model training
GPTBotOAI-SearchBotChatGPT-UserSource: gptbot.json, searchbot.json, chatgpt-user.json
Claude: training, search and pages fetched for a user
ClaudeBotClaude-UserClaude-SearchBotSource: bots.json
Perplexity's search index and pages fetched for a user
PerplexityBotPerplexity-UserSource: perplexitybot.json, perplexity-user.json
Le Chat fetching pages for a user
MistralAI-UserSource: mistralai-user-ips.json
Alexa answers and Amazon's other services
AmazonbotSource: ip-addresses
Link previews on Facebook, Instagram, WhatsApp and Messenger, and Meta AI
facebookexternalhitmeta-externalagentmeta-externalfetchermeta-webindexerfacebookcatalogMeta publishes no list. These are the blocks of its own network, AS32934, from public routing data. On a day in November 2026, 99.9% of Meta's crawler visits to our site came from these blocks.
The free web archive that many AI models are trained on
CCBotSource: ccbot.json
+ 2a03:2880::/33
+ 2a03:2880:f000::/46
+ 2a03:2880:f04a::/47
+ 2a03:2880:f05c::/46
+ 2a03:2880:f070::/46
+ 2a03:2880:f080::/47
+ 2a03:2880:f082::/48
+ 2a03:2880:f083::/48
+ 2a03:2880:f098::/45
+ 2a03:2880:f0a0::/43
+ 2a03:2880:f100::/46
+ 2a03:2880:f14a::/47
+ 2a03:2880:f15c::/46
+ 2a03:2880:f170::/46
+ 2a03:2880:f174::/47
+ 2a03:2880:f176::/47
+ 2a03:2880:f180::/42
+ 2a03:2880:f200::/46
+ 2a03:2880:f24a::/47
+ 2a03:2880:f25c::/46
+ 2a03:2880:f270::/46
+ 2a03:2880:f274::/47
+ 2a03:2880:f276::/47
+ 2a03:2880:f280::/41
+ 2a03:2880:f308::/46
+ 2a03:2880:f30c::/48
+ 2a03:2880:f30d::/48
+ 2a03:2880:f30e::/48
+ 2a03:2880:f30f::/48
+ 2a03:2880:f330::/44
+ 2a03:2880:f348::/45
+ 2a03:2880:f350::/46
+ 2a03:2880:f358::/45
+ 2a03:2880:f372::/47
+ 2a03:2880:f374::/48
+ 2a03:2880:f375::/48
+ 2a03:2880:f376::/47
+ 2a03:2880:f380::/45
+ 2a03:2880:f388::/47
+ 2a03:2880:f38a::/48
+ 2a03:2880:f38b::/48
+ 2a03:2880:f38c::/46
+ 2a03:2880:f390::/44
+ 2a03:2880:f3a0::/43
+ 2a03:2880:f3c0::/42
+ 2a03:2880:f814::/48
+ 2a03:2880:f815::/48
+ 2a03:2880:ff80::/42
+ 2a03:2880:ffc0::/43
+ 2a03:2880:ffe0::/48
+ 2a03:2880:ffe1::/48
+ 2a03:2880:ffe2::/47
+ 2a03:2880:ffe4::/46
+ 2a03:2880:ffe8::/45
+ 2a03:2880:fff0::/44
− 2a03:2880::/34
− 2a03:2880:4000::/35
− 2a03:2880:6000::/35
− 2a03:2880:f000::/47
− 2a03:2880:f002::/47
− 2a03:2880:f04a::/48
− 2a03:2880:f04b::/48
− 2a03:2880:f05c::/47
− 2a03:2880:f05e::/48
− 2a03:2880:f05f::/48
− 2a03:2880:f070::/47
− 2a03:2880:f072::/47
− 2a03:2880:f080::/46
− 2a03:2880:f098::/46
− 2a03:2880:f09c::/47
− 2a03:2880:f09e::/48
− 2a03:2880:f09f::/48
− 2a03:2880:f0a0::/46
− 2a03:2880:f0a4::/47
− 2a03:2880:f0a6::/48
− 2a03:2880:f0a7::/48
− 2a03:2880:f0a8::/45
− 2a03:2880:f0b0::/44
− 2a03:2880:f100::/47
− 2a03:2880:f102::/47
− 2a03:2880:f14a::/48
− 2a03:2880:f14b::/48
− 2a03:2880:f15c::/47
− 2a03:2880:f15e::/48
− 2a03:2880:f15f::/48
− 2a03:2880:f170::/45
− 2a03:2880:f180::/46
− 2a03:2880:f184::/47
− 2a03:2880:f186::/47
− 2a03:2880:f188::/47
− 2a03:2880:f18a::/48
− 2a03:2880:f18b::/48
− 2a03:2880:f18c::/46
− 2a03:2880:f190::/44
− 2a03:2880:f1a0::/43
− 2a03:2880:f200::/47
− 2a03:2880:f202::/47
− 2a03:2880:f24a::/48
− 2a03:2880:f24b::/48
− 2a03:2880:f25c::/47
− 2a03:2880:f25e::/48
− 2a03:2880:f25f::/48
− 2a03:2880:f270::/45
− 2a03:2880:f280::/46
− 2a03:2880:f284::/47
− 2a03:2880:f286::/47
− 2a03:2880:f288::/47
− 2a03:2880:f28a::/48
− 2a03:2880:f28b::/48
− 2a03:2880:f28c::/46
− 2a03:2880:f290::/44
− 2a03:2880:f2a0::/43
− 2a03:2880:f2c0::/42
− 2a03:2880:f308::/45
− 2a03:2880:f330::/45
− 2a03:2880:f338::/46
− 2a03:2880:f33c::/47
− 2a03:2880:f33e::/48
− 2a03:2880:f33f::/48
− 2a03:2880:f348::/46
− 2a03:2880:f34c::/47
− 2a03:2880:f34e::/47
− 2a03:2880:f350::/47
− 2a03:2880:f352::/48
− 2a03:2880:f353::/48
− 2a03:2880:f358::/46
− 2a03:2880:f35c::/47
− 2a03:2880:f35e::/48
− 2a03:2880:f35f::/48
− 2a03:2880:f372::/48
− 2a03:2880:f373::/48
− 2a03:2880:f374::/46
− 2a03:2880:f380::/41
− 2a03:2880:f814::/47
− 2a03:2880:ff80::/41The user agent is just text the visitor sends. Scrapers use "Googlebot" because many sites let Googlebot in.
Look the IP up in the owner's list. On it: real. Not on it, for a company that publishes a list: fake.
Without a list, do a reverse lookup of the IP, check the name ends in the company's domain, then look that name up and make sure it gives the same IP.
googlebot.com google.com googleusercontent.comsearch.msn.comapplebot.apple.comcrawl.commoncrawl.orgyandex.ru yandex.net yandex.comcrawl.baidu.comReverse DNS names that crawlers end in. Yandex and Baidu publish no address list, so this is the only way to check them.
Yahoo's own crawler (Slurp) has all but stopped: on one day in November 2026 we saw 8 requests claiming to be it, and none came from Yahoo. Yahoo's search results come from Bing, so allow Bingbot.
When someone sends your link in iMessage or Google Messages, their own phone fetches it with "facebookexternalhit … Twitterbot" in the user agent. That is a real person, from a home connection: don't block it as a fake Meta crawler.
Blackout Map answers a request that claims to be one of these crawlers but comes from outside its list with a 403. Real crawlers are never blocked, and they skip our browser check.
The lists are published by the companies themselves; we only collect them. Free to use in anything you build. If it helps you, a link to this page is appreciated.