# Domains that disallow MistralAI-User in robots.txt
# agent: mistralai-user
# generated: 2026-08-24T16:46:37.000Z
# count: 509
#
# What this count is, so it reconciles with the rate published elsewhere:
#   509 of 46771 domains with a known policy for this agent disallow it (1.1%).
#   Of those that also served a page, 410 of 30274 disallow it (1.4%).
#   https://crawlcensus.com/operator pages quote the second figure, because a site that refuses every
#   programmatic client is not evidence about AI policy. This file lists the first,
#   because a deny list must include a site whose robots.txt disallows you even when
#   its homepage did not answer our probe. Dividing this file's length by the corpus
#   size gives neither figure.
# measured by direct request, not inferred. Method: https://crawlcensus.com/methodology
# cursor: 1787466175.1402
# resume exactly here, no gap and no replay:
#   https://crawlcensus.com/agents/mistralai-user/changes.json?cursor=1787466175.1402
# this list is robots.txt only. Some domains additionally refuse you at the edge:
#   https://crawlcensus.com/api/v1/preflight
# licence: CC BY 4.0, attribute as "Source: Crawl Census (https://crawlcensus.com)"
11freunde.de
20minutes.fr
4players.de
7sur7.be
aamulehti.fi
aan.com
actu.fr
ad-magazin.de
ad.nl
afr.com
agrarheute.com
agriculture.com
al.com
alisweb.org
allgaeuer-zeitung.de
allrecipes.com
allure.com
amazon.ae
amazon.ca
amazon.co.jp
amazon.co.uk
amazon.co.za
amazon.com
amazon.com.au
amazon.com.br
amazon.com.mx
amazon.com.tr
amazon.de
amazon.eg
amazon.es
amazon.fr
amazon.ie
amazon.in
amazon.it
amazon.jp
amazon.nl
amazon.pl
amazon.sa
amazon.se
amazon.sg
amazonvideo.com
amny.com
amzn.com
androidcentral.com
app.com
architecturaldigest.com
architecturaldigest.in
argusleader.com
arkansasonline.com
arstechnica.com
as.com
askmen.com
augsburger-allgemeine.de
autobild.de
azcentral.com
babycenter.com
babycentre.co.uk
baltimoresun.com
barcablaugranes.com
barcelona.cat
baynews9.com
bd.nl
beaconjournal.com
bfmtv.com
bhg.com
bibliocommons.com
bild.de
blipfoto.com
bluewin.ch
bndestem.nl
bnn.de
bntu.by
bonappetit.com
bostonherald.com
bournemouthecho.co.uk
brides.com
brigitte.de
burlingtonfreepress.com
businessinsider.de
byrdie.com
bz-berlin.de
cadenaser.com
cagesideseats.com
cas.sk
championcross.jp
chicagotribune.com
cincinnati.com
cinemablend.com
citizen-times.com
cjonline.com
clarionledger.com
cleveland.com
cnet.com
cnn.com
cntraveler.com
cntraveller.com
cntraveller.in
commercialappeal.com
computerbild.de
congress.gov
cope.es
courier-journal.com
creativebloq.com
curbed.com
cyclingnews.com
cyclingweekly.com
dailybreeze.com
dailycamera.com
dailyecho.co.uk
dailymail.co.uk
dailymail.com
dailynews.com
dailypress.com
darkhorse.com
decanter.com
delawareonline.com
democratandchronicle.com
demorgen.be
denverpost.com
derwesten.de
desertsun.com
desmoinesregister.com
destentor.nl
detroitnews.com
di.se
digitalcameraworld.com
dispatch.com
dn.se
dr.dk
eadt.co.uk
eastbaytimes.com
eater.com
eatingwell.com
ebsco.com
echo-news.co.uk
ed.nl
edp24.co.uk
elgaronline.com
elpais.com
elpasotimes.com
eltern.de
epicurious.com
essen-und-trinken.de
estantevirtual.com.br
eurogamer.de
eurogamer.net
everydayhealth.com
ew.com
expressen.se
expresso.pt
extremetech.com
faz.net
fd.nl
flipboard.com
flo.com.tr
floridatoday.com
foodandwine.com
fourfourtwo.com
france.tv
france24.com
francebleu.fr
franceculture.fr
franceinfo.fr
franceinter.fr
francetvinfo.fr
freep.com
furaffinity.net
futureplc.com
futurezone.de
gainesville.com
gala.de
gamesindustry.biz
gamesradar.com
gardenersworld.com
gardeningknowhow.com
geek.com
gelderlander.nl
gentoo.org
glamour.com
glamourmagazine.co.uk
glasgowtimes.co.uk
gofeminin.de
golfmonthly.com
gq-magazin.de
gq-magazine.co.uk
gq.com
gq.com.mx
gqindia.com
gqitalia.it
gqmagazine.fr
greenvilleonline.com
grubstreet.com
guitarworld.com
gunbark.dev
gva.be
handelsblatt.com
hbvl.be
hd.se
health.com
heraldscotland.com
heraldtribune.com
hinative.com
hln.be
hoerzu.de
homesandgardens.com
hoopshype.com
houseandgarden.co.uk
houstonpublicmedia.org
howlongtobeat.com
hs.fi
humo.be
ici.fr
idealhome.co.uk
iex.nl
ign.com
igt.com
ilex.by
iltalehti.fi
imageshack.com
imageshack.us
imore.com
ina.fr
indebuurt.nl
independent.ie
indystar.com
inews.co.uk
inforum.com
instyle.com
investopedia.com
is.fi
itpro.com
jacksonville.com
jeuneafrique.com
jsonline.com
jwz.org
kauppalehti.fi
kiplinger.com
knoxnews.com
kompas.com
lacucinaitaliana.it
lancashiretelegraph.co.uk
lansingstatejournal.com
laptopmag.com
law.com
lci.fr
learnreligions.com
leckerschmecker.me
lefigaro.fr
lehighvalleylive.com
lemonde.fr
lesechos.fr
letelegramme.fr
lexpress.fr
libelle.nl
lifehacker.com
lifewire.com
liquor.com
liveabout.com
livescience.com
livestrong.com
lohud.com
loudersound.com
lyricstranslate.com
mailonsunday.co.uk
mailplus.co.uk
main-echo.de
mainpost.de
manager-magazin.de
mapgenie.io
marieclaire.co.uk
marieclaire.com
marinij.com
marthastewart.com
mashable.com
masslive.com
maville.com
maxroll.gg
mcall.com
mediamarkt.de
mediamarkt.pl
mediaworld.it
medpagetoday.com
mercurynews.com
metro.co.uk
mlive.com
mmafighting.com
mnn.com
moneyweek.com
montgomeryadvertiser.com
msn.com
mtv.fi
musicradar.com
myrecipes.com
naplesnews.com
nbcnews.com
nettavisen.no
news-journalonline.com
news-leader.com
news-postseven.com
news-press.com
newsandstar.co.uk
newsbreak.com
newscientist.com
newspapers.com
newyorker.com
nicematin.com
nj.com
noordhollandsdagblad.nl
northjersey.com
noz.de
nrc.nl
nrk.no
nu.nl
nwaonline.com
ny1.com
nydailynews.com
nymag.com
nzherald.co.nz
nzz.ch
ocregister.com
oklahoman.com
omim.org
omroepwest.nl
oregonlive.com
oxfordmail.co.uk
palmbeachpost.com
parents.com
parool.nl
pcgamer.com
pcmag.com
pennlive.com
people.com
peopleenespanol.com
pilotonline.com
pitchfork.com
pjstar.com
pluska.sk
pnj.com
politico.eu
popsugar.com
portableapps.com
posfie.com
postimees.ee
premiumoutlets.com
pressdemocrat.com
presstelegram.com
primevideo.com
providencejournal.com
pzc.nl
radiofrance.fr
realsimple.com
refinery29.com
registerguard.com
revistagq.com
revistavanityfair.es
rfi.fr
rgj.com
rnz.co.nz
road.cc
rockpapershotgun.com
rtvdrenthe.nl
rtvnoord.nl
rtvutrecht.nl
ruhrnachrichten.de
sandiegouniontribune.com
santacruzsentinel.com
saturn.de
savannahnow.com
sayweee.com
sbnation.com
seacoastonline.com
seekingalpha.com
self.com
seriouseats.com
shape.com
shz.de
sicnoticias.pt
silive.com
simon.com
simplyrecipes.com
sj-r.com
smartbrief.com
sme.sk
southernliving.com
southwalesargus.co.uk
space.com
spectrumlocalnews.com
spectrumnews1.com
spiegel.de
sportsbusinessjournal.com
statesmanjournal.com
stimme.de
suedkurier.de
sydsvenskan.se
syracuse.com
t3.com
tallahassee.com
tatler.com
tcpalm.com
techbook.de
techradar.com
teenvogue.com
telegraaf.nl
telegram.com
telegraph.co.uk
tennessean.com
tf1.fr
tf1info.fr
theafricareport.com
theargus.co.uk
theatlantic.com
thebalancemoney.com
theconversation.com
thecut.com
thedodo.com
theglobeandmail.com
theledger.com
themoviedb.org
thenational.scot
thenorthernecho.co.uk
thespec.com
thespruce.com
thesprucecrafts.com
thespruceeats.com
thesprucepets.com
thestar.com
thetelegraphandargus.co.uk
theverge.com
theweek.co.uk
theweek.com
thisismoney.co.uk
thoughtco.com
thrillist.com
tiktok.com
timesfreepress.com
tmdb.org
today.com
togetter.com
tomsguide.com
tomshardware.com
topekalibraries.info
toptenreviews.com
travelandleisure.com
travelbook.de
treehugger.com
tripsavvy.com
trouw.nl
tu.berlin
tubantia.nl
tv2.dk
tv4play.se
tvnet.lv
twincities.com
u-cursos.cl
unil.ch
uol.com
uol.com.br
usatoday.com
usmagazine.com
vanityfair.com
vanityfair.fr
vanityfair.it
vcstar.com
verywellfit.com
verywellhealth.com
verywellmind.com
vg247.com
vndb.org
vogue.co.jp
vogue.co.uk
vogue.com
vogue.com.tw
vogue.de
vogue.es
vogue.fr
vogue.in
vogue.it
vogue.mx
volkskrant.nl
vox.com
vulture.com
wallpaper.com
wbur.org
welt.de
werstreamt.es
whathifi.com
whattoexpect.com
whowhatwear.com
wickedlocal.com
wikihow.com
wikihow.life
windowscentral.com
wired.co.uk
wired.com
wired.it
wired.jp
wiwo.de
womanandhome.com
womansworld.com
worcesternews.co.uk
ybw.com
ylilauta.org
yorkpress.co.uk
youngchampion.jp
ytmnd.com
zdnet.com
zepto.com
zeptonow.com
zin.ru