Blocklist · You.com

Sites that block YouBot

876 of the 29,991 domains we have measured, 3%, disallow youbot at the site root in robots.txt. YouBot builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from You.com's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

876 of 29,991 measured domains (3%) disallow youbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block YouBot", measured 2026-11-22. https://crawlcensus.com/blocked/youbot

The list 876 domains

RankDomainTLDAI access score
18,147 volksstimme.de .de 68
18,280 hoerzu.de .de 45
18,309 mans.io .io 60
18,402 xueqiu.com .com 47
18,442 empireonline.com .com 62
18,490 click2houston.com .com 85
18,522 thenews.com.pk .pk 65
18,595 trycloudflare.com .com 52
18,608 sensacine.com .com 81
18,687 romatoday.it .it 67
18,863 dailynews.com .com 76
18,867 adressa.no .no 66
18,870 scotusblog.com .com 83
18,883 courier-journal.com .com 74
18,894 courrierinternational.com .com 83
18,978 dexerto.com .com 78
19,182 clickorlando.com .com 85
19,283 denik.cz .cz 77
19,312 pinayflix.uk .uk 46
19,314 milanotoday.it .it 67
19,342 bold.dk .dk 58
19,696 givemesport.com .com 83
19,860 zawya.com .com 80
19,891 musicradar.com .com 87
19,941 ottawacitizen.com .com 71
20,035 plejada.pl .pl 83
20,080 chroniclelive.co.uk .uk 47
20,081 flashscore.com.tr .tr 70
20,232 uv.mx .mx 75
20,318 supersport.hr .hr 51
20,362 thedodo.com .com 70
20,364 derwesten.de .de 83
20,481 flashscore.pt .pt 72
20,575 artnews.com .com 80
20,718 itpro.com .com 83
20,847 gardeningknowhow.com .com 83
20,858 flashscore.vn .vn 72
20,958 pressdemocrat.com .com 80
21,004 femina.hu .hu 82
21,030 rugbyrama.fr .fr 89
21,038 mainpost.de .de 80
21,139 ledevoir.com .com 70
21,267 seher.no .no 70
21,331 winbet.bg .bg 50
21,431 thelocal.se .se 87
21,481 futureplc.com .com 86
21,566 whowhatwear.com .com 82
21,586 madison.com .com 80
21,685 calgaryherald.com .com 71
21,707 khronos.org .org 68
21,736 slate.fr .fr 68
21,809 cyclingnews.com .com 85
21,823 desy.de .de 62
21,828 wallpaper.com .com 85
21,889 scoop.co.nz .nz 67
21,890 telerama.fr .fr 83
22,007 vg247.com .com 58
22,031 news4jax.com .com 85
22,216 airliners.net .net 44
22,365 petitesannonces.ch .ch 67
22,479 ksat.com .com 85
22,596 flashscore.gr .gr 72
22,623 mcall.com .com 78
22,867 fb77.one .one 50
23,084 app.com .com 69
23,157 mlnba.fun .fun 54
23,875 imore.com .com 81
24,014 askmen.com .com 47
24,022 twinkl.co.uk .uk 37
24,028 canberratimes.com.au .au 79
24,102 knoxnews.com .com 74
24,141 cntraveller.com .com 81
24,157 whathifi.com .com 87
24,190 jacksonville.com .com 74
24,233 glamourmagazine.co.uk .uk 81
24,413 irishmirror.ie .ie 47
24,468 flashscore.de .de 71
24,525 zin.ru .ru 41
24,593 vogue.in .in 64
24,754 aol.co.uk .uk 66
24,802 bristolpost.co.uk .uk 47
24,842 sdstate.edu .edu 74
24,882 uol.com .com 82
24,892 eldiariomontanes.es .es 82
24,934 svtplay.se .se 78
24,946 vogue.fr .fr 81
24,961 miaminewtimes.com .com 73
24,977 wired.it .it 83
24,983 sn.at .at 70
25,007 digitalcameraworld.com .com 86
25,048 tokyo-np.co.jp .jp 81
25,108 wheree.com .com 86
25,115 pilotonline.com .com 79
25,133 myrecipes.com .com 77
25,184 kompasiana.com .com 85
25,237 flashscorekz.com .com 70
25,270 startersites.io .io 46
25,451 omaha.com .com 80
25,564 theadvocate.com .com 71
25,587 sfexaminer.com .com 69
25,653 livesport.com .com 69
25,677 flashscore.in .in 70
25,756 ctinsider.com .com 54
25,800 trendencias.com .com 94
25,839 delawareonline.com .com 74
25,903 heraldtribune.com .com 73
25,911 arkansasonline.com .com 71
26,005 people.inc .inc 64
26,114 democratandchronicle.com .com 73
26,249 guitarworld.com .com 84
26,284 daytondailynews.com .com 67
26,342 straightdope.com .com 76
26,379 brisbanetimes.com.au .au 75
26,474 mbta.com .com 64
26,567 spox.com .com 81
26,637 brigitte.de .de 79
26,806 golf.com .com 88
26,842 unherd.com .com 83
26,853 tallahassee.com .com 74
26,963 premiumoutlets.com .com 31
27,047 thelocal.de .de 87
27,136 purepeople.com.br .br 85
27,243 themirror.com .com 47
27,306 stylecaster.com .com 77
27,315 fourfourtwo.com .com 84
27,361 westfalen-blatt.de .de 81
27,425 carbuzz.com .com 85
27,467 wfmz.com .com 81
27,482 tug.org .org 44
27,696 staradvertiser.com .com 84
27,723 geek.com .com 38
27,833 apotheken-umschau.de .de 45
27,838 mtvuutiset.fi .fi 72
28,038 topspeed.com .com 81
28,170 suedkurier.de .de 81
28,320 laptopmag.com .com 87
28,616 amzn.com .com 40
28,642 nachrichten.at .at 82
28,678 thenational.scot .scot 81
28,757 federalnewsnetwork.com .com 86
28,775 vogue.de .de 65
28,792 uesp.net .net 55
28,794 eastbaytimes.com .com 79
28,930 championcross.jp .jp 62
29,001 completesports.com .com 88
29,011 fraserinstitute.org .org 51
29,048 gq-magazin.de .de 66
29,115 gp.se .se 78
29,139 u-cursos.cl .cl 51
29,273 expressnews.com .com 54
29,293 ceu.edu .edu 58
29,390 kompas.id .id 64
29,402 mmafighting.com .com 68
29,499 conjur.com.br .br 73
29,553 hankookilbo.com .com 84
29,567 ctpost.com .com 54
29,572 flashscore.com.mx .mx 70
29,634 westword.com .com 77
29,639 edmontonjournal.com .com 71
29,821 providencejournal.com .com 73
29,980 lidovky.cz .cz 67
30,004 sfstandard.com .com 72
30,052 lasprovincias.es .es 82
30,074 yugipedia.com .com 58
30,196 youngchampion.jp .jp 62
30,237 clarionledger.com .com 74
30,356 edaily.co.kr .kr 74
30,387 bsc.es .es 37
30,443 dobrzemieszkaj.pl .pl 61
30,703 wickedlocal.com .com 75
30,712 artforum.com .com 74
30,737 diariosur.es .es 82
30,810 kentonline.co.uk .uk 76
30,828 bowdoin.edu .edu 71
30,986 commercialappeal.com .com 74
30,992 huffingtonpost.it .it 83
31,023 phoenixnewtimes.com .com 71
31,032 howlongtobeat.com .com 49
31,138 ideal.es .es 82
31,149 idealhome.co.uk .uk 82
31,176 edp24.co.uk .uk 79
31,282 vndb.org .org 60
31,320 lumendatabase.org .org 51
31,352 sportsbusinessjournal.com .com 53
31,381 coventrytelegraph.net .net 47
31,499 bolasport.com .com 76
31,549 lohud.com .com 74
31,554 currencyconverter.io .io 46
31,638 marieclaire.co.uk .uk 84
31,786 thespec.com .com 64
31,850 avsforum.com .com 43
31,871 diariodesevilla.es .es 84
32,032 prx.org .org 74
32,125 elmercurio.com .com 43
32,151 devonlive.com .com 47
32,263 shacknews.com .com 73
32,267 theargus.co.uk .uk 79
32,388 rgj.com .com 73
32,437 hoopshype.com .com 70
32,460 saturn.de .de 91
PreviousPage 3 of 5Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the YouBot reference, the full census, and the JSON endpoint.