# Domains that disallow AI2Bot in robots.txt
# agent: ai2bot
# generated: 2026-08-25T19:32:12.000Z
# count: 718
#
# What this count is, so it reconciles with the rate published elsewhere:
#   718 of 46914 domains with a known policy for this agent disallow it (1.5%).
#   Of those that also served a page, 663 of 30344 disallow it (2.2%).
#   https://crawlcensus.com/operator pages quote the second figure, because a site that refuses every
#   programmatic client is not evidence about AI policy. This file lists the first,
#   because a deny list must include a site whose robots.txt disallows you even when
#   its homepage did not answer our probe. Dividing this file's length by the corpus
#   size gives neither figure.
# measured by direct request, not inferred. Method: https://crawlcensus.com/methodology
# cursor: 1787467015.1465
# resume exactly here, no gap and no replay:
#   https://crawlcensus.com/agents/ai2bot/changes.json?cursor=1787467015.1465
# this list is robots.txt only. Some domains additionally refuse you at the edge:
#   https://crawlcensus.com/api/v1/preflight
# licence: CC BY 4.0, attribute as "Source: Crawl Census (https://crawlcensus.com)"
01net.com
11freunde.de
15min.lt
20minutos.es
4players.de
588ku.com
aamulehti.fi
aan.com
abc.es
abendblatt.de
adorocinema.com
adressa.no
adweek.com
afr.com
aftenbladet.no
aftenposten.no
aftonbladet.se
agrarheute.com
alisweb.org
allgaeuer-zeitung.de
allocine.fr
amazon.ae
amazon.ca
amazon.co.jp
amazon.co.uk
amazon.co.za
amazon.com
amazon.com.au
amazon.com.br
amazon.com.mx
amazon.com.tr
amazon.de
amazon.eg
amazon.es
amazon.fr
amazon.ie
amazon.in
amazon.it
amazon.jp
amazon.nl
amazon.pl
amazon.sa
amazon.se
amazon.sg
amazonvideo.com
amny.com
amzn.com
androidcentral.com
anyflip.com
app.com
archinect.com
argusleader.com
arkansasonline.com
asahi.com
askmen.com
audizine.com
augsburger-allgemeine.de
auto-motor-und-sport.de
autobild.de
autozeitung.de
azcentral.com
babycenter.com
babycentre.co.uk
backpacker.com
bakersfield.com
baltimoresun.com
barcablaugranes.com
barcelona.cat
baynews9.com
bbcgoodfood.com
beaconjournal.com
beatstars.com
beatstars.net
bengo4.com
bibliocommons.com
bild.de
blipfoto.com
bnn.de
bntu.by
bonn.de
bostonherald.com
bournemouthecho.co.uk
braunschweiger-zeitung.de
bravotv.com
breakingnews.ie
brigitte.de
bt.no
burlingtonfreepress.com
businessinsider.de
businessinsider.es
bz-berlin.de
cagesideseats.com
calculator.name
canarias7.es
canva.cn
canva.com
canva.in
carter-cash.com
ceskenoviny.cz
championcross.jp
channel3000.com
chicagotribune.com
chinatimes.com
chroniclejournal.com
chugoku-np.co.jp
chunichi.co.jp
cincinnati.com
cinemablend.com
cio.com
citizen-times.com
cjonline.com
clarionledger.com
cnet.com
cnn.com
commercialappeal.com
computerbild.de
computerhope.com
computerwoche.de
computerworld.com
congress.gov
conjur.com.br
cookpad.com
courier-journal.com
courrierinternational.com
coursehero.com
creativebloq.com
crunchbase.com
csoonline.com
ctee.com.tw
curbed.com
cyclingnews.com
cyclingweekly.com
dailybreeze.com
dailycal.org
dailycamera.com
dailyecho.co.uk
dailygazette.com
dailymail.co.uk
dailymail.com
dailynews.com
dailypress.com
darkhorse.com
daytondailynews.com
dazeddigital.com
dealereprocess.com
decanter.com
defcon.org
delawareonline.com
delfi.ee
delfi.lt
delfi.lv
democratandchronicle.com
denik.cz
denverpost.com
derwesten.de
desertsun.com
desmoinesregister.com
detroitnews.com
dhnet.be
diariodecadiz.es
diariodesevilla.es
diariosur.es
diariovasco.com
digitalcameraworld.com
directoalpaladar.com
diretta.it
dispatch.com
dobrzemieszkaj.pl
drugs.com
duden.de
dvdprime.com
dw.com
dwds.de
e24.no
ea.com
eadt.co.uk
eastbaytimes.com
eater.com
echo-news.co.uk
edmunds.com
edp24.co.uk
elcomercio.es
elcorreo.com
eldiariomontanes.es
elgaronline.com
elnortedecastilla.es
elpasotimes.com
eltern.de
empireonline.com
equidia.fr
eredmenyek.com
espinof.com
essen-und-trinken.de
estantevirtual.com.br
eumetsat.int
eurogamer.de
eurogamer.net
everydayhealth.com
expresso.pt
extremetech.com
f95zone.to
faz.net
fd.nl
filmstarts.de
financemagnates.com
flashscore.co
flashscore.co.id
flashscore.co.uk
flashscore.co.za
flashscore.com
flashscore.com.br
flashscore.com.mx
flashscore.com.ng
flashscore.com.tr
flashscore.com.ua
flashscore.com.ve
flashscore.de
flashscore.es
flashscore.fr
flashscore.gr
flashscore.in
flashscore.mobi
flashscore.pe
flashscore.pl
flashscore.pt
flashscore.ro
flashscore.sk
flashscore.ua
flashscore.vn
flashscorekz.com
flashscoreusa.com
flipboard.com
fliphtml5.com
floridatoday.com
flourish.studio
forth.gr
fourfourtwo.com
france24.com
francebleu.fr
franceculture.fr
franceinter.fr
freep.com
furaffinity.net
futureplc.com
futurezone.de
gainesville.com
gala.de
gamesindustry.biz
gamesradar.com
gaoding.com
gardenersworld.com
gardeningknowhow.com
gds.it
geek.com
gentoo.org
glasgowtimes.co.uk
gofeminin.de
golem.de
golf.com
golfmonthly.com
goop.com
govtrack.us
gp.se
gpb.org
greenme.it
greenvilleonline.com
grindosaur.com
grubstreet.com
gtaforums.com
guitarworld.com
gunbark.dev
hankookilbo.com
healio.com
heraldo.es
heraldscotland.com
heraldtribune.com
hinative.com
historyextra.com
hoerzu.de
homesandgardens.com
hoopshype.com
horseracingnation.com
houstonpublicmedia.org
howlongtobeat.com
hoy.es
hrs.com
hs.fi
huffingtonpost.fr
iacr.org
icecat.biz
icecat.dk
icecat.it
icecat.se
icecat.us
ici.fr
ideal.es
idealhome.co.uk
iex.nl
ign.com
igt.com
ilex.by
iltalehti.fi
imageshack.com
imageshack.us
imore.com
ina.fr
indystar.com
inews.co.uk
inforum.com
infoworld.com
ingramcontent.com
instiz.net
intechopen.com
interestingengineering.com
is.fi
itpro.com
iza.ne.jp
jacksonville.com
jamanetwork.com
jarir.com
jeuxvideo.com
jfklibrary.org
joinhandshake.com
joongang.co.kr
journaldugeek.com
jsonline.com
jtbc.co.kr
jwz.org
kauppalehti.fi
kentonline.co.uk
kiplinger.com
kittl.com
knoxnews.com
kob.com
kotaku.com
kstp.com
kupi.cz
lalibre.be
lancashiretelegraph.co.uk
lancasteronline.com
lansingstatejournal.com
laptopmag.com
larazon.es
larioja.com
lasillavacia.com
lasprovincias.es
launchpad.net
lavenir.net
laverdad.es
law.com
lci.fr
lecker.de
leckerschmecker.me
lemonde.fr
letelegramme.fr
letterboxd.com
letudiant.fr
levels.fyi
lexpress.fr
liberation.fr
lifehacker.com
livescience.com
livescore.cz
livescore.in
livesport.com
livesport.cz
livestrong.com
lohud.com
loudersound.com
lyricstranslate.com
macworld.com
mailonsunday.co.uk
mailplus.co.uk
main-echo.de
mainichi.jp
mainpost.de
manager-magazin.de
mania.kr
mapgenie.io
mapquest.com
marieclaire.co.uk
marieclaire.com
marinij.com
mashable.com
maxim.com
maxroll.gg
mbta.com
mcall.com
mediamarkt.de
mediamarkt.nl
mediamarkt.pl
mediaworld.it
medpagetoday.com
mercurynews.com
metro.co.uk
milanofinanza.it
miricanvas.com
mmafighting.com
moly.hu
moneyweek.com
monoskop.org
montgomeryadvertiser.com
mora.jp
morgenpost.de
motorradonline.de
moviemaker.com
mparticle.com
msn.com
mt.co.kr
mtv.fi
mujerhoy.com
musicradar.com
myanimelist.net
myhora.com
myporntube.com
naplesnews.com
nationalgeographic.de
ncaa.com
nd-aktuell.de
nefisyemektarifleri.com
nettavisen.no
news-journalonline.com
news-leader.com
news-postseven.com
news-press.com
news9.com
newsandstar.co.uk
newscientist.com
newson6.com
newspapers.com
nicematin.com
niche.com
nintendo.co.jp
nintendo.com
nishinippon.co.jp
nn.de
nola.com
nordbayern.de
northjersey.com
nos.nl
noticiasaominuto.com
novinky.cz
noz.de
nrc.nl
nrk.no
nrtool.st
nrz.de
nwaonline.com
ny1.com
nybooks.com
nydailynews.com
nymag.com
nzherald.co.nz
observador.pt
occrp.org
ocregister.com
oefb.at
ohtuleht.ee
oklahoman.com
omim.org
omroepwest.nl
onlamp.com
opinautos.com
opovo.com.br
ora.com
oreilly.com
oreillynet.com
otz.de
outsideonline.com
oxfordmail.co.uk
palmbeachpost.com
pandora.com
pantone.com
parismatch.com
pcgamer.com
pcmag.com
pcworld.com
penguin.de
piliapp.com
pilotonline.com
pimpbunny.com
pjstar.com
pnj.com
politico.eu
popsugar.com
portableapps.com
portalsamorzadowy.pl
posfie.com
postandcourier.com
posteo.de
postimees.ee
postype.com
premiumoutlets.com
pressdemocrat.com
presse-citron.net
presstelegram.com
pricesmart.com
primevideo.com
proofpoint.com
providencejournal.com
prozeny.cz
prx.org
pubhtml5.com
purepeople.com
purepeople.com.br
r777.net
radiofrance.fr
radiotimes.com
ranker.com
realcommercial.com.au
realestate.com.au
refinery29.com
registerguard.com
rezultati.com
rfi.fr
rgj.com
road.cc
roadsideamerica.com
roanoke.com
rockpapershotgun.com
rtbf.be
rtl.be
rtl.nl
rtlnieuws.nl
rtvdrenthe.nl
rtvnoord.nl
rtvutrecht.nl
ruhrnachrichten.de
runescape.wiki
rynekzdrowia.pl
sandiegouniontribune.com
sankei.com
sanspo.com
santacruzsentinel.com
santafenewmexican.com
saturn.de
savannahnow.com
sayweee.com
sbnation.com
schulferien.org
scotusblog.com
sdstate.edu
seacoastonline.com
seekingalpha.com
sensacine.com
seznamzpravy.cz
sfexaminer.com
sfstandard.com
sfweekly.com
shz.de
sicnoticias.pt
simon.com
sj-r.com
slate.com
slideserve.com
smartbrief.com
sme.sk
snapchat.com
soccerway.com
songkick.com
southwalesargus.co.uk
space.com
spectrumlocalnews.com
spectrumnews1.com
spiegel.de
sport.cz
sportler.com
sporza.be
spoton.com
startmycar.com
statesman.com
statesmanjournal.com
stimme.de
storyblocks.com
stream.cz
suedkurier.de
super.cz
superbet.pl
superbet.ro
superbet.rs
svd.se
svt.se
svtplay.se
syfy.com
t3.com
t3n.de
tagesspiegel.de
taipeitimes.com
tallahassee.com
tcpalm.com
techadvisor.com
techbook.de
techradar.com
telegram.com
telegraph.co.uk
tennessean.com
tennis.com
tf1.fr
tf1info.fr
theadvocate.com
theanalyst.com
theargus.co.uk
theblast.com
thecentersquare.com
theconversation.com
thecut.com
thedodo.com
theepochtimes.com
thegazette.com
theglobeandmail.com
thegrio.com
theledger.com
themoviedb.org
thenational.scot
thenorthernecho.co.uk
thespec.com
thestar.com
thetelegraphandargus.co.uk
theverge.com
theweek.co.uk
theweek.com
thisismoney.co.uk
thomascook.com
thrillist.com
thueringer-allgemeine.de
tiktok.com
timesfreepress.com
tmdb.org
togetter.com
tokyo-np.co.jp
tomsguide.com
tomshardware.com
topekalibraries.info
toptenreviews.com
townnews.com
travelbook.de
trendencias.com
trinidadexpress.com
trustedshops.com
trustedshops.de
tu.berlin
tudogostoso.com.br
tv2.dk
tv2.no
tv4.se
tv4play.se
tvmovie.de
tvnet.lv
twincities.com
u-cursos.cl
uesp.net
ultimahora.es
unibz.it
unil.ch
unionleader.com
uol.com
uol.com.br
usatoday.com
vcstar.com
veikkaus.fi
vermont.gov
vg.no
vg247.com
videocardz.com
vivino.com
vndb.org
vox.com
vozpopuli.com
vulture.com
wallpaper.com
waz.de
wbur.org
wccftech.com
wdrb.com
wellsky.com
welt.de
werstreamt.es
westfalen-blatt.de
wfmz.com
whathifi.com
whattoexpect.com
whowhatwear.com
wickedlocal.com
wikihow.com
wikihow.life
windowscentral.com
winknews.com
wn.de
wnp.pl
womanandhome.com
worcesternews.co.uk
worldsurfleague.com
wowhead.com
wp.de
wral.com
wunderweib.de
xataka.com
xatakamovil.com
xe.gr
xink.io
ybw.com
yle.fi
ylilauta.org
yna.co.kr
yogajournal.com
yorkpress.co.uk
youngchampion.jp
ytmnd.com
yugipedia.com
zakzak.co.jp
zawya.com
zdnet.com
zepto.com
zeptonow.com
zetaglobal.net
zillow.com
zin.ru
zoominfo.com