Multi-Signal Request Fingerprints: Combining TLS, HTTP and Network
How to design a composite request fingerprint from TLS, HTTP and network signals: picking fields, hashing them safely, and keeping implementations in sync.
read →Stop paying to serve scrapers, scanners and fake crawlers, without blocking the real users who share their IPs. BannedRobots fingerprints every request at the Cloudflare edge, learns which clients are abusing your site, and blocks those exact clients before they reach your origin. Ban a bot on one of your domains and it's blocked on all of them within about a minute.
Behind CGNAT, a corporate proxy or a mobile carrier, thousands of real users share one address. Block the IP and you block all of them.
Any scraper can claim to be Chrome or Googlebot. The UA header is one line the client writes itself.
Every scraper request you serve costs CPU, database queries and egress. Blocking at the edge avoids that cost.
What changes once BannedRobots sits in front of your site.
Scrapers never reach your servers, so you stop paying egress, CPU and database time to serve them. Your cache stops filling up with pages only crawlers ask for.
Prices, listings and articles stop being copied. Bans follow the scraper's fingerprint, so switching to a new proxy doesn't get it back in.
Scanners probing for /.env and /wp-login.php are identified by behavior and stopped at the edge. Once banned, their probes stop reaching your application.
Bans target one client fingerprint, not an IP or a network. Customers on the same mobile carrier, VPN or office proxy keep browsing normally.
Verified Googlebot, Bingbot and other good crawlers are exempt before any rule runs. Impostors using their user agent are blocked.
No IP lists to curate, no regexes to tune. It learns from your own traffic, and bans are removed once a bot stops showing up.
It learns from your own traffic, not a generic rule set. Bans are applied in a Worker, next to your users.
At the edge, each request is described by signals a client can't easily fake: how its TLS handshake is built, how it speaks HTTP and the network it comes from. It doesn't slow the request down.
Clients that behave alike are grouped together, so a botnet is judged as one. Each group is scored on what it requests, how it browses and whether it is who it claims to be.
Only fingerprints actually seen in banned groups are blocked. A matching request gets a 403 at the Cloudflare edge and never reaches your origin.
Every request carries a fingerprint: how its TLS handshake is built, how it speaks HTTP and where it connects from. A scraper can copy Chrome's user agent, but not everything else. Two clients on the same IP look different, so the scraper is blocked and the person next to it gets through.
| block by | stops bot | spares users |
|---|---|---|
| ip address | ~ until it rotates | ✕ shared IPs |
| asn / country | ✓ | ✕ whole networks |
| user-agent rule | ✕ spoofed | ✓ mostly |
| fingerprint | ✓ | ✓ |
Every ban records the reason it happened, so you can audit any block and reverse it if needed.
A bot filter that takes your site down is worse than having no filter.
BannedRobots is built so that a problem on our side never turns into an outage on yours. Your site keeps serving visitors, whatever happens to us.
Checks run in the Cloudflare location nearest each visitor, and reporting happens after the response is sent.
Only the request metadata needed for detection is analyzed. Cookies and Authorization headers never leave your edge.
Every ban is logged with its reason. Look up the request ID, see why it was blocked, and reverse it if needed.
The short answers. Ask us anything else when you request access.
Yes. BannedRobots runs as a Cloudflare Worker and uses the TLS and network metadata Cloudflare attaches to each request. Your origin can be hosted anywhere.
No. Crawlers verified by Cloudflare are exempt before any rule runs. Bots that only claim to be Googlebot are a different case: failing verification is one of the ban signals.
No. All your domains share one ban list. When a bot is banned on one of them, it's blocked on every other domain within about a minute, before it can move on to your next site.
They see a 403 with a request ID. Look up that ID to find the signature and ban reason, then remove the signature from the ban list. Because bans are per fingerprint, removing one affects nobody else.
Not noticeably. The check runs in the Cloudflare location nearest the visitor, and logging happens after the response is sent. Blocked requests never reach your origin, so your origin has less work to do.
Request metadata used for detection: method, host, path, IP, Cloudflare network and TLS fields, and an allowlist of non-sensitive headers. Cookies and Authorization headers are never sent.
How fingerprinting, bot detection and edge enforcement work, in depth. All articles →
How to design a composite request fingerprint from TLS, HTTP and network signals: picking fields, hashing them safely, and keeping implementations in sync.
read →Which AI crawlers visit your site, what each one does with your content, and how to allow, block or limit them with robots.txt, verification and edge rules.
read →Bots cost more than bandwidth: compute, database load, cache misses and skewed analytics. A simple model to estimate what bot traffic costs your origin.
read →We're onboarding a small number of Cloudflare sites during early access. Join the waitlist and we'll email you when there's a spot.
✓ You're on the list! We'll email you when we launch.
// no spam. one email when we launch.