Blocklist · You.com

Sites that block YouBot

876 of the 29,991 domains we have measured, 3%, disallow youbot at the site root in robots.txt. YouBot builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from You.com's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

876 of 29,991 measured domains (3%) disallow youbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block YouBot", measured 2026-11-22. https://crawlcensus.com/blocked/youbot

The list 876 domains

RankDomainTLDAI access score
26 amazon.com .com 40
58 yahoo.com .com 67
62 msn.com .com 41
121 amazonvideo.com .com 55
160 tumblr.com .com 41
190 cnn.com .com 50
193 theguardian.com .com 60
207 bbc.com .com 76
226 bbc.co.uk .uk 77
260 launchpad.net .net 52
281 nature.com .com 56
282 weather.com .com 86
318 amazon.co.uk .uk 40
377 amazon.de .de 40
494 amazon.co.jp .jp 40
499 aol.com .com 66
533 wired.com .com 81
571 uol.com.br .br 82
609 amazon.fr .fr 40
623 amazon.co.za .za 40
627 dailymail.co.uk .uk 61
635 amazon.ca .ca 40
659 amazon.in .in 40
662 cnet.com .com 79
671 lemonde.fr .fr 81
681 dailymail.com .com 61
682 primevideo.com .com 71
706 amazon.it .it 40
725 amazon.es .es 40
763 bild.de .de 81
812 amazon.com.au .au 40
815 theverge.com .com 77
823 nbcnews.com .com 59
833 amazon.com.br .br 40
834 lefigaro.fr .fr 71
926 genius.com .com 65
942 repubblica.it .it 85
955 corriere.it .it 63
977 onet.pl .pl 76
983 elmundo.es .es 84
1,019 theconversation.com .com 62
1,032 amazon.com.mx .mx 40
1,042 spiegel.de .de 64
1,056 marca.com .com 85
1,073 bfmtv.com .com 81
1,074 flashscore.com .com 73
1,098 franceinfo.fr .fr 60
1,104 idnes.cz .cz 80
1,177 pimpbunny.com .com 56
1,189 aftonbladet.se .se 77
1,196 vnexpress.net .net 88
1,208 vg.no .no 79
1,223 furaffinity.net .net 39
1,258 abc.net.au .au 87
1,262 mundodeportivo.com .com 82
1,272 vietnam.vn .vn 74
1,273 ladepeche.fr .fr 86
1,300 f95zone.to .to 59
1,305 dw.com .com 42
1,376 dagbladet.no .no 74
1,394 usnews.com .com 83
1,409 iltalehti.fi .fi 58
1,446 ekstrabladet.dk .dk 67
1,613 arabx.cam .cam 66
1,624 flashscore.mobi .mobi 69
1,640 flashscore.fr .fr 71
1,676 livescore.cz .cz 69
1,793 newyorker.com .com 73
1,831 zdnet.com .com 76
1,835 wikihow.com .com 76
1,984 themoviedb.org .org 74
1,993 ign.com .com 62
2,006 amazon.eg .eg 40
2,010 mashable.com .com 61
2,115 mirror.co.uk .uk 47
2,118 variety.com .com 80
2,215 techradar.com .com 87
2,230 meb.gov.tr .tr 66
2,299 guardian.co.uk .uk 60
2,346 slate.com .com 80
2,351 diretta.it .it 70
2,423 pcmag.com .com 63
2,430 sfgate.com .com 54
2,433 express.co.uk .uk 48
2,530 smh.com.au .au 82
2,561 technologyreview.com .com 82
2,581 hollywoodreporter.com .com 79
2,585 livescience.com .com 85
2,691 newscientist.com .com 76
2,786 chicagotribune.com .com 79
2,899 vanityfair.it .it 77
2,917 faz.net .net 39
2,969 today.com .com 62
2,974 thehindu.com .com 66
2,975 imageshack.us .us 71
2,989 vogue.com .com 82
3,198 metro.co.uk .uk 59
3,253 billboard.com .com 78
3,286 flipboard.com .com 64
3,331 tomshardware.com .com 87
3,342 lifehacker.com .com 65
3,368 nzherald.co.nz .nz 56
3,371 nymag.com .com 74
3,418 deadline.com .com 81
3,499 amazon.se .se 40
3,520 amazon.ae .ae 40
3,524 chron.com .com 54
3,533 vanityfair.com .com 81
3,542 orf.at .at 69
3,576 abc.es .es 81
3,595 politico.eu .eu 62
3,598 amazon.nl .nl 40
3,642 yahoo.net .net 67
3,676 handelsblatt.com .com 82
3,717 ansa.it .it 85
3,771 yle.fi .fi 85
3,800 ebsco.com .com 65
3,873 space.com .com 85
3,909 aol.de .de 65
3,916 news24.com .com 56
3,963 nos.nl .nl 69
4,061 screenrant.com .com 84
4,108 howtogeek.com .com 84
4,161 kompas.com .com 77
4,190 zoominfo.com .com 90
4,304 tvtropes.org .org 55
4,512 amazon.com.tr .tr 40
4,551 yna.co.kr .kr 87
4,592 lastampa.it .it 81
4,753 xda-developers.com .com 83
4,766 adweek.com .com 69
4,791 collider.com .com 83
4,816 polygon.com .com 83
4,987 dr.dk .dk 81
5,026 pcgamer.com .com 85
5,037 imageshack.com .com 71
5,068 gq.com .com 81
5,096 nine.com.au .au 71
5,146 everydayhealth.com .com 60
5,153 theage.com.au .au 75
5,158 mercurynews.com .com 78
5,205 lapresse.ca .ca 65
5,223 oilprice.com .com 90
5,251 dallasnews.com .com 54
5,297 chinatimes.com .com 64
5,323 cbr.com .com 83
5,363 allocine.fr .fr 81
5,395 makeuseof.com .com 84
5,401 mediamarkt.de .de 90
5,413 expressen.se .se 61
5,455 gamerant.com .com 81
5,457 diigo.com .com 35
5,523 architecturaldigest.com .com 83
5,547 amazon.ie .ie 40
5,678 dn.se .se 60
5,681 nationalpost.com .com 71
5,714 francetvinfo.fr .fr 53
5,726 chefkoch.de .de 81
5,752 sankei.com .com 76
5,788 ibtimes.com .com 84
5,846 runescape.wiki .wiki 81
5,874 hs.fi .fi 68
5,895 movieweb.com .com 84
5,904 windstream.net .net 53
5,948 suntimes.com .com 81
6,089 azcentral.com .com 74
6,122 jeuxvideo.com .com 71
6,125 denverpost.com .com 78
6,159 instiz.net .net 58
6,180 tf1info.fr .fr 84
6,228 sbnation.com .com 69
6,246 eurogamer.net .net 57
6,288 vulture.com .com 72
6,308 wwd.com .com 78
6,352 tv2.dk .dk 74
6,356 freep.com .com 73
6,377 gamesradar.com .com 85
6,429 emol.com .com 70
6,473 sfchronicle.com .com 54
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 78
6,522 eater.com .com 59
6,549 midilibre.fr .fr 86
6,557 sportingnews.com .com 82
6,593 wired.co.uk .uk 64
6,610 is.fi .fi 68
6,737 altavista.com .com 53
6,793 svt.se .se 66
6,808 chunichi.co.jp .jp 71
6,816 maxroll.gg .gg 59
6,817 funnyjunk.com .com 52
6,822 elcorreo.com .com 80
6,910 xataka.com .com 94
6,922 livescore.in .in 71
6,948 nicematin.com .com 77
7,042 cntraveler.com .com 82
7,082 aktuality.sk .sk 85
7,084 spacebattles.com .com 72
7,091 kathimerini.gr .gr 68
7,106 nrc.nl .nl 84
Page 1 of 5Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the YouBot reference, the full census, and the JSON endpoint.