# Domains that disallow Bingbot in robots.txt # agent: bingbot # generated: 2026-08-25T07:08:21.000Z # count: 50 # # What this count is, so it reconciles with the rate published elsewhere: # 50 of 46897 domains with a known policy for this agent disallow it (0.1%). # Of those that also served a page, 44 of 30338 disallow it (0.1%). # https://crawlcensus.com/operator pages quote the second figure, because a site that refuses every # programmatic client is not evidence about AI policy. This file lists the first, # because a deny list must include a site whose robots.txt disallows you even when # its homepage did not answer our probe. Dividing this file's length by the corpus # size gives neither figure. # measured by direct request, not inferred. Method: https://crawlcensus.com/methodology # cursor: 1787641701.0 # resume exactly here, no gap and no replay: # https://crawlcensus.com/agents/bingbot/changes.json?cursor=1787641701.0 # this list is robots.txt only. Some domains additionally refuse you at the edge: # https://crawlcensus.com/api/v1/preflight # licence: CC BY 4.0, attribute as "Source: Crawl Census (https://crawlcensus.com)" 1c.ru aagag.com ad-contents.jp aif.ru amzlink.to ascii.jp audacy.com bttn.io cia.gov cnnic.cn ecode360.com endnote.com enrollbusiness.com europeanfilmgateway.eu farrow-ball.com inveragil.com izap24.ru jwz.org kashalot.com masutabe.info medibang.com merlion.ru muumuu-domain.com netvasco.com.br niji-gazo.com nukistream.com onelaunch.com page-stats.de petsathome.com privatter.net productreview.com.au psytests.org pucsp.br readli.net retail.ru roland.com rossko.ru simplebooking.it tender.pro texasattorneygeneral.gov thegef.org thieme-connect.com thieme-connect.de thisdaylive.com timeedit.net times-info.net tpprf.ru webofcomics.top winespectator.com xxxvideosind.com