CloudFlare Bot Directory

A comprehensive list of 500+ verified bots and web crawlers from CloudFlare Radar, available as a JSON dataset for bot detection, user agent analysis, and web scraping identification.

Why

Identifying legitimate bots from malicious scrapers is essential for web security and analytics. This package provides the official CloudFlare Radar verified bots directory, helping you:

• Detect verified bots – Identify legitimate crawlers like Googlebot, Bingbot, and more
• Filter analytics – Exclude known bots from your traffic reports
• Allow-list crawlers – Permit verified bots while blocking suspicious traffic
• User agent lookup – Match user agent strings against known bot patterns

Data Structure

Each bot entry includes:

Field	Description
slug	URL-friendly unique identifier
name	Human-readable bot name
kind	Bot kind classification
operator	Company or organization operating the bot
operatorUrl	URL to operator's documentation
category	Bot category (Search Engine, Monitoring, etc.)
description	What the bot does
followsRobotsTxt	Whether the bot respects robots.txt
userAgentPatterns	User agent pattern(s) for matching
userAgents	Known user agent string(s)

Install

npm install cloudflare-bot-directory

Usage

const bots = require('cloudflare-bot-directory')

// Get all bot slugs
console.log(bots.map(bot => bot.slug))
// ['2checkout', '360monitoring', 'googlebot', 'bingbot', ...]

// Find a specific bot
const googlebot = bots.find(bot => bot.slug === 'googlebot')

// Filter by category
const searchBots = bots.filter(bot => bot.category === 'Search Engine Crawler')

// Check if a user agent is a known bot
const isKnownBot = (userAgent) =>
  bots.some(bot => userAgent.includes(bot.name))

Related

• top-user-agents – An always up-to-date list of the top 100 HTTP user-agents most used over the Internet.

License

cloudflare-bot-directory © Kiko Beats, released under the MIT License.
Authored and maintained by Kiko Beats with help from contributors.

kikobeats.com · GitHub Kiko Beats · X @kikobeats