--- title: "Cloudflare Will Score Verified Bots on Whether They Keep Obeying robots.txt and 429s" description: "Signed identity is settled; the new system grades behavior over time and will publish trust bands on Radar" author: "Luis Goa" published: 2026-10-16T13:00:00Z modified: 2026-10-16T18:33:37Z url: https://rews.cc/a/cloudflare-will-score-verified-bots-on-whether-they-keep-obe-65627d language: en tags: ["ai", "security", "robots", "automation", "cloudflare", "tech"] publisher: "Rews (https://rews.cc)" --- # Cloudflare Will Score Verified Bots on Whether They Keep Obeying robots.txt and 429s *Signed identity is settled; the new system grades behavior over time and will publish trust bands on Radar* By Luis Goa · October 16, 2026 · https://rews.cc/a/cloudflare-will-score-verified-bots-on-whether-they-keep-obe-65627d ## In brief - Cloudflare announced a system scoring Verified bots on whether they honor site preferences over time - The initial signals: consistency with robots.txt under RFC 9309, and adaptation to HTTP 429 rate-limit responses - Results become explainable “trust bands,” planned for public release on Radar and customer-set thresholds later - The only enforcement precedent cited: Perplexity’s bot was de-listed in 2025 for scraping opted-out sites and evading controls - Identity verification via Web Bot Auth request signing and the BotBase catalogue is the settled first layer Cloudflare already knows which bots are knocking. As of Thursday, it is also starting to grade how well they behave once admitted. [In a blog post](https://blog.cloudflare.com/verified-bots-accountability/), the company described a system that observes whether Verified bots — operators who have cryptographically proven their identity — keep honoring site preferences over time, and collapses those observations into an explainable “trust band.” The company’s own framing is apt: identity is a driver’s license; it says who you are, not how you drive. ## How the identity half already works To be a Cloudflare Verified Bot, an operator must meet three criteria: identify itself accurately and securely, do only what it says it will do, and not engage in abusive behavior. The first is handled by Web Bot Auth, in which bots sign their HTTP requests to prove who operates them; BotBase then catalogues those identities in an open format. Cloudflare poses the verification problem as two questions — “Are you who you say you are?” and “Can we trust you?” — and the new system exists because the second question has no point-in-time answer. The framing matches the company’s earlier work [tracking account behavior histories to catch fraud that beats identity checks](https://rews.cc/a/cloudflare-tracks-accounts-over-time-to-catch-fraud-that-bea-3c90f4). ![A robotic hand holds a blank card with a robot icon, while a small wheeled robot approaches a red barrier on a winding cream path.](https://rews.cc/img/50ca13c017c1fbd2de8bf98b478059f96b99d03e.webp) ## What the system measures The initial signals are deliberately mundane. First: does the bot’s observed traffic stay consistent with the site’s robots.txt file over time, per RFC 9309? Isolated violations and one-offs don’t count; what the system looks for are repeated patterns that disregard published preferences. Cloudflare is explicit that robots.txt is a floor, not a ceiling — the file states what bots must not do, but nothing in it implies that everything unstated is allowed, and a site can communicate that continued access is unwelcome through other signals. Second: has the bot changed its behavior in response to direct feedback, specifically HTTP 429 “Too Many Requests” responses? A 429 is a directive to reduce request rate, sometimes with a hint on how long to wait. A bot can stay entirely within allowed paths and still bury an origin in requests — a denial-of-service incident wearing a polite signature. The trust band rises for bots that consistently slow down or pause after a 429, and falls for those that keep sending at the same pace. Cloudflare acknowledges that misbehavior is often misconfiguration or a bug rather than malice, which is why the band grades sustained patterns rather than permanent culpability. ## Trust band, not verdict The company draws a clean line between risk and trust. Risk is the likelihood that a single request or point-in-time action is harmful, and it can change moment to moment. Trust accumulates through repeated, consistent behavior. The output is a range showing how consistently a bot respects preferences, annotated with the observed behaviors that produced it — “rather than a permanent judgment about a bot or its operator,” in the post’s words. The thesis: verify, then continuously re-verify. ![A cream-colored cartoon tree stump with golden growth rings and two dark eyes wears navy headphones against a bright blue background.](https://rews.cc/img/f0844f47bef8e0c84ed87d34438810350df474c0.webp) Cloudflare says the scoring runs across internal evaluation of the Verified bot population now, and this initial phase is for building behavioral history and refining what the observations mean. It plans to publish the scores on Radar, expand the measured signals, and eventually open the system to all customers, who will be able to choose their own trust threshold for handling Verified bot traffic. Together with Bot Preference Sync, which lets sites publish preferences, and Bot Management, which enforces them, the pieces form what the company calls an accountability loop: site declares, bot is observed, customer decides. Two qualifications before granting the model its due. First, no data yet: the post announces a scoring system with no published scores, no population statistics, and no false-positive methodology, so today there is nothing to audit. Second, the one concrete enforcement action on record is instructive. Cloudflare de-listed Perplexity’s bot from the Verified list in 2025 after it attempted to access resources on websites that had opted out of scraping and then tried to circumvent the controls blocking it. That precedent — deliberate evasion, judged at the network layer — is exactly the behavior a trust band is meant to surface earlier. It is also Cloudflare acting as identity issuer, behavior scorer, and traffic gatekeeper simultaneously, a vertical stack worth remembering when the scores go public.