Blocklist · You.com

Sites that block YouBot

876 of the 29,991 domains we have measured, 3%, disallow youbot at the site root in robots.txt. YouBot builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from You.com's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

876 of 29,991 measured domains (3%) disallow youbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block YouBot", measured 2026-11-22. https://crawlcensus.com/blocked/youbot

The list 876 domains

RankDomainTLDAI access score
32,498 toptenreviews.com .com 87
32,579 dallasobserver.com .com 77
32,617 flo.com.tr .tr 52
32,713 opovo.com.br .br 85
32,718 athletic.net .net 68
32,790 tatler.com .com 82
32,803 oxfordmail.co.uk .uk 79
32,806 tvnet.lv .lv 79
32,918 laverdad.es .es 82
33,054 biologicaldiversity.org .org 66
33,063 dailypost.co.uk .uk 47
33,254 aan.com .com 74
33,349 qq8893.com .com 68
33,562 myporntube.com .com 86
33,569 nottinghampost.com .com 47
33,654 liegagego33.com .com 49
33,738 ruv.is .is 59
33,931 lancasteronline.com .com 79
34,012 womanandhome.com .com 85
34,097 vogue.es .es 65
34,113 kentek.ru .ru 83
34,309 unionleader.com .com 63
34,350 juju.com .com 72
34,445 monoskop.org .org 43
34,449 mylondon.news .news 47
34,491 sfweekly.com .com 74
34,718 sportsyou.com .com 86
34,737 vibe.com .com 80
34,750 mailplus.co.uk .uk 44
34,822 milanofinanza.it .it 82
34,847 sydsvenskan.se .se 52
34,870 mel.org .org 55
34,973 naplesnews.com .com 73
34,983 wbez.org .org 78
34,986 ruhrnachrichten.de .de 75
35,058 gaoding.com .com 62
35,062 dailypress.com .com 79
35,322 tk999vip.com .com 51
35,324 northdata.de .de 81
35,404 mapadeapellidos.com .com 75
35,458 beritasatu.com .com 63
35,543 news-journalonline.com .com 73
35,684 americanrhetoric.com .com 45
35,759 houseandgarden.co.uk .uk 81
35,760 montgomerycountymd.gov .gov 65
35,763 penguin.de .de 88
35,800 wn.de .de 80
35,828 dotdashmeredith.com .com 64
35,933 hrs.com .com 82
35,953 redflagdeals.com .com 44
36,120 elcomercio.es .es 82
36,237 saltwire.com .com 70
36,334 ytmnd.com .com 41
36,356 elgaronline.com .com 45
36,434 businessday.co.za .za 82
36,436 liquor.com .com 77
36,539 nd-aktuell.de .de 78
36,575 dailyecho.co.uk .uk 81
36,646 cyclingweekly.com .com 85
36,702 299betvip.com .com 51
36,718 arynews.tv .tv 91
36,778 cwmars.org .org 43
37,154 r777.net .net 51
37,305 hotcars.com .com 82
37,367 mittelbayerische.de .de 77
37,425 jamestown.org .org 82
37,522 twinkl.com .com 37
37,547 bt.no .no 77
37,613 unescap.org .org 72
37,643 bk333.app .app 51
37,674 leicestermercury.co.uk .uk 47
37,700 rutracker.net .net 43
37,738 infoclimat.fr .fr 49
37,821 golfmonthly.com .com 83
38,007 wired.jp .jp 78
38,015 thetravel.com .com 81
38,114 sportico.com .com 64
38,129 11freunde.de .de 58
38,148 news-press.com .com 73
38,327 inforum.com .com 70
38,388 goldderby.com .com 74
38,409 igt.com .com 48
38,507 bournemouthecho.co.uk .uk 79
38,519 kochbar.de .de 62
38,526 haystack.tv .tv 88
38,668 stimme.de .de 72
38,739 thueringer-allgemeine.de .de 82
38,814 mas.to .to 31
38,876 elnortedecastilla.es .es 82
38,945 postype.com .com 81
39,206 futurezone.de .de 86
39,210 thecentersquare.com .com 78
39,303 watoday.com.au .au 75
39,349 hoy.es .es 82
39,350 thehighline.org .org 77
39,369 santafenewmexican.com .com 76
39,393 zakzak.co.jp .jp 72
39,471 vwvortex.com .com 43
39,587 hulldailymail.co.uk .uk 47
39,706 indiehackers.com .com 54
39,824 werstreamt.es .es 59
40,013 belfastlive.co.uk .uk 47
40,317 iza.ne.jp .jp 79
40,328 bntu.by .by 80
40,339 kompas.tv .tv 76
40,403 thelocal.fr .fr 87
40,413 dualshockers.com .com 83
40,495 yahoo.com.tw .tw 59
40,512 thetelegraphandargus.co.uk .uk 78
40,638 main-echo.de .de 64
40,713 dwds.de .de 64
40,726 flashscoreusa.com .com 71
40,728 mediamarkt.pl .pl 89
40,796 elpasotimes.com .com 73
40,830 xink.io .io 78
40,874 grid.id .id 64
40,876 cntraveller.in .in 81
40,904 nesn.com .com 73
40,968 kontan.co.id .id 77
41,088 ispe.org .org 67
41,098 gqmagazine.fr .fr 82
41,104 stokesentinel.co.uk .uk 47
41,121 lancashiretelegraph.co.uk .uk 78
41,140 rixos.com .com 69
41,204 elpuntavui.cat .cat 36
41,290 theweek.co.uk .uk 84
41,295 newson6.com .com 82
41,336 archinect.com .com 68
41,519 greenme.it .it 82
41,542 oefb.at .at 50
41,603 lci.fr .fr 84
41,647 cjonline.com .com 73
41,653 donaukurier.de .de 78
41,713 roanoke.com .com 80
41,721 canarias7.es .es 83
41,736 eadt.co.uk .uk 79
41,760 dailybreeze.com .com 76
41,826 defcon.org .org 66
41,872 vanityfair.fr .fr 66
41,934 crikey.com.au .au 88
41,947 vogue.co.jp .jp 65
41,958 theledger.com .com 74
42,076 iex.nl .nl 48
42,105 odi.org .org 54
42,112 news9.com .com 85
42,113 glasgowtimes.co.uk .uk 79
42,160 tcpalm.com .com 73
42,260 rcgroups.com .com 41
42,262 advrider.com .com 41
42,582 decanter.com .com 85
42,614 citizen-times.com .com 74
42,666 nhregister.com .com 54
42,698 sol.no .no 74
42,728 wishtv.com .com 80
42,835 iodonna.it .it 86
42,862 horseracingnation.com .com 72
42,919 tcrf.net .net 29
42,981 viewbug.com .com 83
43,089 port.hu .hu 68
43,132 jtbc.co.kr .kr 70
43,207 tv4play.se .se 77
43,251 sufficientvelocity.com .com 73
43,268 candlepowerforums.com .com 61
43,302 seacoastonline.com .com 69
43,310 3djuegos.com .com 93
43,315 stateline.org .org 69
43,437 derbytelegraph.co.uk .uk 47
43,587 admagazine.fr .fr 81
43,593 mediamarkt.at .at 89
43,616 appian.com .com 77
43,659 medonet.pl .pl 84
43,890 21jingji.com .com 65
43,974 ybw.com .com 87
44,026 gainesville.com .com 66
44,041 cambridge-news.co.uk .uk 47
44,183 barcablaugranes.com .com 68
44,252 braunschweiger-zeitung.de .de 65
44,255 cagesideseats.com .com 72
44,299 theprovince.com .com 71
44,547 vavel.com .com 89
44,723 dailygazette.com .com 80
44,807 gofeminin.de .de 74
44,874 news-leader.com .com 62
44,928 wgt.com .com 80
44,930 jogatina.com .com 80
44,946 cornwalllive.com .com 47
44,968 savannahnow.com .com 64
45,002 babycentre.co.uk .uk 62
45,133 northdata.com .com 81
45,158 autoguide.com .com 87
45,159 lansingstatejournal.com .com 62
45,180 pivotalweather.com .com 25
45,240 vogue.mx .mx 65
45,298 osapublishing.org .org 65
45,316 eumetsat.int .int 80
45,349 wp.de .de 65
45,390 tupalo.com .com 57
45,440 nrz.de .de 80
45,578 chugoku-np.co.jp .jp 80
45,604 beaconjournal.com .com 66
PreviousPage 4 of 5Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the YouBot reference, the full census, and the JSON endpoint.