# Domains that disallow DuckAssistBot in robots.txt
# agent: duckassistbot
# generated: 2026-08-24T08:31:57.000Z
# count: 594
#
# What this count is, so it reconciles with the rate published elsewhere:
#   594 of 46128 domains with a known policy for this agent disallow it (1.3%).
#   Of those that also served a page, 511 of 29841 disallow it (1.7%).
#   https://crawlcensus.com/operator pages quote the second figure, because a site that refuses every
#   programmatic client is not evidence about AI policy. This file lists the first,
#   because a deny list must include a site whose robots.txt disallows you even when
#   its homepage did not answer our probe. Dividing this file's length by the corpus
#   size gives neither figure.
# measured by direct request, not inferred. Method: https://crawlcensus.com/methodology
# cursor: 1787467015.1468
# resume exactly here, no gap and no replay:
#   https://crawlcensus.com/agents/duckassistbot/changes.json?cursor=1787467015.1468
# this list is robots.txt only. Some domains additionally refuse you at the edge:
#   https://crawlcensus.com/api/v1/preflight
# licence: CC BY 4.0, attribute as "Source: Crawl Census (https://crawlcensus.com)"
15min.lt
4players.de
588ku.com
aan.com
abc.es
acronymfinder.com
ad-magazin.de
adressa.no
afr.com
agrarheute.com
agriculture.com
aktuality.sk
al.com
alisweb.org
allgaeuer-zeitung.de
allmusic.com
allrecipes.com
allure.com
amazon.ae
amazon.ca
amazon.co.jp
amazon.co.uk
amazon.co.za
amazon.com
amazon.com.au
amazon.com.br
amazon.com.mx
amazon.com.tr
amazon.de
amazon.eg
amazon.es
amazon.fr
amazon.ie
amazon.in
amazon.it
amazon.jp
amazon.nl
amazon.pl
amazon.sa
amazon.se
amazon.sg
amazonvideo.com
amny.com
amzn.com
ansa.it
app.com
archinect.com
architecturaldigest.com
architecturaldigest.in
argusleader.com
arkansasonline.com
arstechnica.com
as.com
askmen.com
augsburger-allgemeine.de
autobild.de
azcentral.com
babycenter.com
babycentre.co.uk
badische-zeitung.de
baltimoresun.com
barcablaugranes.com
barcelona.cat
beaconjournal.com
bhg.com
bild.de
bisnis.com
bizapedia.com
blipfoto.com
bloomberg.com
bnn.de
bntu.by
bolasport.com
bonappetit.com
bonn.de
bostonherald.com
bournemouthecho.co.uk
brides.com
brigitte.de
brisbanetimes.com.au
bsc.es
burlingtonfreepress.com
businessinsider.de
buzzfeed.com
buzzfeednews.com
byrdie.com
bz-berlin.de
cadenaser.com
cagesideseats.com
canarias7.es
championcross.jp
channel4.com
chicagotribune.com
chinatimes.com
chron.com
chugoku-np.co.jp
chunichi.co.jp
cincinnati.com
citizen-times.com
cjonline.com
clarionledger.com
cleveland.com
cnet.com
cnn.com
cntraveler.com
cntraveller.com
cntraveller.in
commercialappeal.com
complex.com
computerbild.de
computerhope.com
congress.gov
conjur.com.br
courier-journal.com
courrierinternational.com
ctee.com.tw
ctinsider.com
ctpost.com
curbed.com
dailybreeze.com
dailycamera.com
dailyecho.co.uk
dailymail.co.uk
dailymail.com
dailynews.com
dailypress.com
dallasnews.com
defcon.org
delawareonline.com
delfi.ee
delfi.lt
delfi.lv
democratandchronicle.com
denik.cz
denverpost.com
derwesten.de
desertsun.com
desmoinesregister.com
detroitnews.com
dhnet.be
di.se
diariosur.es
diariovasco.com
diretta.it
dispatch.com
dn.se
dobrzemieszkaj.pl
donaukurier.de
dr.dk
dwds.de
eadt.co.uk
eastbaytimes.com
eater.com
eatingwell.com
echo-news.co.uk
edp24.co.uk
ekstrabladet.dk
elcomercio.es
elcorreo.com
eldiariomontanes.es
elgaronline.com
elnortedecastilla.es
elpais.com
elpasotimes.com
eltern.de
epicurious.com
eredmenyek.com
essen-und-trinken.de
eumetsat.int
eurogamer.de
eurogamer.net
everydayhealth.com
ew.com
expressen.se
expressnews.com
expresso.pt
extremetech.com
f95zone.to
falter.at
faz.net
fd.nl
flashscore.co
flashscore.co.id
flashscore.co.uk
flashscore.co.za
flashscore.com
flashscore.com.br
flashscore.com.mx
flashscore.com.ng
flashscore.com.tr
flashscore.com.ua
flashscore.com.ve
flashscore.de
flashscore.es
flashscore.fr
flashscore.gr
flashscore.in
flashscore.mobi
flashscore.pe
flashscore.pl
flashscore.pt
flashscore.ro
flashscore.sk
flashscore.ua
flashscore.vn
flashscorekz.com
flashscoreusa.com
flipboard.com
flo.com.tr
floridatoday.com
foodandwine.com
forth.gr
france24.com
francebleu.fr
franceculture.fr
franceinter.fr
freep.com
furaffinity.net
futurezone.de
gainesville.com
gala.de
gamesindustry.biz
gds.it
geek.com
gentoo.org
glamour.com
glamourmagazine.co.uk
glasgowtimes.co.uk
gofeminin.de
govtrack.us
gq-magazin.de
gq-magazine.co.uk
gq.com
gqindia.com
gqitalia.it
gqmagazine.fr
greenme.it
greenvilleonline.com
grid.id
grindosaur.com
grubstreet.com
guardian.co.uk
gunbark.dev
hankookilbo.com
hd.se
health.com
hellomagazine.com
heraldscotland.com
heraldtribune.com
hinative.com
hoerzu.de
hola.com
hoopshype.com
houseandgarden.co.uk
houstonchronicle.com
houstonpublicmedia.org
howlongtobeat.com
hoy.es
huffingtonpost.co.uk
huffingtonpost.com
huffingtonpost.fr
huffingtonpost.it
huffingtonpost.jp
huffpost.com
iacr.org
ibtimes.com
ici.fr
ideal.es
iex.nl
ign.com
igt.com
ilex.by
ilgazzettino.it
ilmattino.it
ilmessaggero.it
ilsole24ore.com
iltalehti.fi
imageshack.com
imageshack.us
ina.fr
indystar.com
inews.co.uk
inforum.com
instyle.com
interestingengineering.com
investopedia.com
iol.co.za
iza.ne.jp
jacksonville.com
jsonline.com
jtbc.co.kr
jwz.org
kauppalehti.fi
kentonline.co.uk
knoxnews.com
kompas.com
kompas.id
kompas.tv
kompasiana.com
kontan.co.id
lacucinaitaliana.it
lalibre.be
lancashiretelegraph.co.uk
lansingstatejournal.com
larioja.com
lasprovincias.es
lastampa.it
launchpad.net
lavenir.net
laverdad.es
law.com
lci.fr
learnreligions.com
leckerschmecker.me
leggo.it
lehighvalleylive.com
lemonde.fr
lesechos.fr
letelegramme.fr
letudiant.fr
lexpress.fr
liberation.fr
lifehacker.com
lifewire.com
liquor.com
liveabout.com
livescore.cz
livescore.in
livesport.com
livesport.cz
livestrong.com
lohud.com
mailonsunday.co.uk
mailplus.co.uk
main-echo.de
mainpost.de
mamamia.com.au
mapgenie.io
marinij.com
marthastewart.com
mashable.com
masslive.com
maville.com
maxroll.gg
mbta.com
mcall.com
medietall.no
medpagetoday.com
mercurynews.com
metro.co.uk
mittelbayerische.de
mlive.com
mmafighting.com
mnn.com
monoskop.org
montgomeryadvertiser.com
msn.com
mujerhoy.com
myrecipes.com
mysanantonio.com
nachrichten.at
naplesnews.com
nationalgeographic.de
nd-aktuell.de
nefisyemektarifleri.com
nesn.com
news-journalonline.com
news-leader.com
news-postseven.com
news-press.com
newsandstar.co.uk
newsbreak.com
newscientist.com
newspapers.com
newyorker.com
nhregister.com
nicematin.com
nine.com.au
nishinippon.co.jp
nj.com
nn.de
nordbayern.de
northjersey.com
nos.nl
noticiasaominuto.com
noz.de
nrc.nl
nrk.no
nydailynews.com
nymag.com
nyt.com
nyt.net
nytimes.com
nzherald.co.nz
observador.pt
ocregister.com
ohtuleht.ee
oklahoman.com
omroepwest.nl
oregonlive.com
oxfordmail.co.uk
palmbeachpost.com
parents.com
parismatch.com
pcmag.com
pennlive.com
people.com
peopleenespanol.com
pilotonline.com
pimpbunny.com
pitchfork.com
pjstar.com
pnj.com
pnp.de
politico.eu
politiken.dk
popsugar.com
portableapps.com
portalsamorzadowy.pl
posfie.com
postimees.ee
postype.com
pressdemocrat.com
presstelegram.com
primevideo.com
providencejournal.com
prx.org
r777.net
radiofrance.fr
realsimple.com
refinery29.com
registerguard.com
repubblica.it
revistagq.com
revistavanityfair.es
rezultati.com
rfi.fr
rgj.com
rnz.co.nz
rockpapershotgun.com
rtbf.be
rtl.be
rtvdrenthe.nl
rtvnoord.nl
rtvutrecht.nl
rynekzdrowia.pl
salzburg24.at
sandiegouniontribune.com
sanspo.com
santacruzsentinel.com
savannahnow.com
sbnation.com
scotusblog.com
seacoastonline.com
seattlepi.com
seekingalpha.com
seia.org
self.com
seriouseats.com
sfchronicle.com
sfgate.com
shape.com
shz.de
sicnoticias.pt
silive.com
simplyrecipes.com
sj-r.com
smh.com.au
sn.at
soccerway.com
southernliving.com
southwalesargus.co.uk
sportingnews.com
sportler.com
sportsbusinessjournal.com
sporza.be
staradvertiser.com
statesman.com
statesmanjournal.com
stimme.de
suedkurier.de
sydsvenskan.se
syracuse.com
tallahassee.com
tasty.co
tatler.com
tcpalm.com
techbook.de
technologyreview.com
teenvogue.com
telegram.com
telegraph.co.uk
tennessean.com
tf1.fr
tf1info.fr
tfd.com
theage.com.au
theargus.co.uk
thebalancemoney.com
theconversation.com
thecut.com
thedodo.com
thefreedictionary.com
thefreelibrary.com
theglobeandmail.com
theguardian.com
theledger.com
themoviedb.org
thenational.scot
thenorthernecho.co.uk
thespec.com
thespruce.com
thesprucecrafts.com
thespruceeats.com
thesprucepets.com
thestar.com
thetelegraphandargus.co.uk
theverge.com
thisismoney.co.uk
thoughtco.com
thrillist.com
tiktok.com
timesfreepress.com
timesunion.com
tmdb.org
today.it
togetter.com
tokyo-np.co.jp
topekalibraries.info
toyota.com
trademe.co.nz
travelandleisure.com
travelbook.de
treehugger.com
tripsavvy.com
tu.berlin
tv2.dk
tv2.no
tvnet.lv
twincities.com
u-cursos.cl
unil.ch
uol.com
uol.com.br
usatoday.com
vanityfair.com
vanityfair.fr
vanityfair.it
vcstar.com
veikkaus.fi
vermont.gov
verywellfit.com
verywellhealth.com
verywellmind.com
vg247.com
vndb.org
vogue.co.jp
vogue.co.uk
vogue.com
vogue.com.tw
vogue.de
vogue.es
vogue.fr
vogue.in
vogue.it
vogue.mx
vox.com
vulture.com
waptrick.com
watoday.com.au
welt.de
werstreamt.es
whattoexpect.com
wickedlocal.com
wikihow.com
wikihow.life
wired.co.uk
wired.com
wired.it
wired.jp
wnp.pl
worcesternews.co.uk
worldsurfleague.com
ylilauta.org
yorkpress.co.uk
youngchampion.jp
ytmnd.com
yugipedia.com
zakzak.co.jp
zdnet.com
zepto.com
zeptonow.com
zin.ru
zoominfo.com