Blocklist · Google

Sites that block Google-Extended

2,269 of the 29,991 domains we have measured, 8%, disallow google-extended at the site root in robots.txt. Google-Extended collects content that may be used to train a model.

Every domain here has withheld its content from Google's training crawl. That is a separate decision from appearing in the operator's answers, which is governed by a different token.

The measurement

2,269 of 29,991 measured domains (8%) disallow google-extended at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Google-Extended", measured 2026-11-22. https://crawlcensus.com/blocked/google-extended

The list 2,269 domains

RankDomainTLDAI access score
4,084 uiowa.edu .edu 70
4,108 howtogeek.com .com 84
4,181 sbs.com.au .au 83
4,326 wiki.gg .gg 67
4,333 derstandard.at .at 56
4,334 blender.org .org 63
4,354 computerworld.com .com 83
4,431 swr.de .de 76
4,484 mdr.de .de 76
4,493 wdr.de .de 86
4,512 amazon.com.tr .tr 40
4,551 yna.co.kr .kr 87
4,553 20minutes.fr .fr 71
4,609 123rf.com .com 43
4,684 msnbc.com .com 75
4,753 xda-developers.com .com 83
4,791 collider.com .com 83
4,816 polygon.com .com 83
4,866 channel4.com .com 56
4,965 vinted.co.uk .uk 91
4,987 dr.dk .dk 81
5,037 imageshack.com .com 71
5,057 deccanherald.com .com 64
5,068 gq.com .com 81
5,096 nine.com.au .au 71
5,153 theage.com.au .au 75
5,158 mercurynews.com .com 78
5,205 lapresse.ca .ca 65
5,213 game8.co .co 76
5,251 dallasnews.com .com 54
5,272 depositphotos.com .com 63
5,274 csmonitor.com .com 76
5,277 bookshop.org .org 74
5,293 turbify.com .com 85
5,323 cbr.com .com 83
5,395 makeuseof.com .com 84
5,413 expressen.se .se 61
5,455 gamerant.com .com 81
5,457 diigo.com .com 35
5,463 rainews.it .it 53
5,522 abc.com .com 82
5,523 architecturaldigest.com .com 83
5,547 amazon.ie .ie 40
5,609 rtl.fr .fr 87
5,678 dn.se .se 60
5,684 theepochtimes.com .com 80
5,714 francetvinfo.fr .fr 53
5,749 autoscout24.de .de 86
5,752 sankei.com .com 76
5,769 degruyter.com .com 61
5,770 boingboing.net .net 79
5,786 cloudpath.live .live 44
5,846 runescape.wiki .wiki 81
5,874 hs.fi .fi 68
5,876 dmitriy-mevedev-tor.top .top 53
5,879 cio.com .com 86
5,895 movieweb.com .com 84
5,904 windstream.net .net 53
5,912 pagesix.com .com 80
5,936 qobuz.com .com 79
5,948 suntimes.com .com 81
5,993 eonline.com .com 81
6,019 vinted.de .de 67
6,020 giga.de .de 81
6,032 rtl.de .de 85
6,035 opaque.net .net 39
6,062 nhk.jp .jp 64
6,089 azcentral.com .com 74
6,111 ledauphine.com .com 82
6,125 denverpost.com .com 78
6,159 instiz.net .net 58
6,180 tf1info.fr .fr 84
6,228 sbnation.com .com 69
6,245 raiplay.it .it 55
6,275 leprogres.fr .fr 82
6,288 vulture.com .com 72
6,301 iata.org .org 68
6,308 wwd.com .com 78
6,335 royalroad.com .com 48
6,340 vinted.it .it 68
6,342 vinted.pl .pl 86
6,352 tv2.dk .dk 74
6,356 freep.com .com 73
6,361 ici.fr .fr 87
6,391 sex-studentki.live .live 56
6,421 chavesnamao.com.br .br 80
6,429 emol.com .com 70
6,473 sfchronicle.com .com 54
6,512 telegram.com .com 73
6,513 sandiegouniontribune.com .com 78
6,522 eater.com .com 59
6,530 theintercept.com .com 86
6,535 literotica.com .com 85
6,549 midilibre.fr .fr 86
6,557 sportingnews.com .com 82
6,593 wired.co.uk .uk 64
6,609 procyclingstats.com .com 56
6,610 is.fi .fi 68
6,702 infoworld.com .com 83
6,726 rtbf.be .be 67
6,737 altavista.com .com 53
6,759 degruyterbrill.com .com 61
6,793 svt.se .se 66
6,803 schwaebische.de .de 70
6,804 sportschau.de .de 85
6,808 chunichi.co.jp .jp 71
6,817 funnyjunk.com .com 52
6,916 raptive.com .com 90
6,926 bankier.pl .pl 72
6,945 countryliving.com .com 74
6,948 nicematin.com .com 77
6,988 buzzfeednews.com .com 79
7,042 cntraveler.com .com 82
7,058 playmogo.com .com 44
7,084 spacebattles.com .com 72
7,091 kathimerini.gr .gr 68
7,106 nrc.nl .nl 84
7,109 military.com .com 84
7,119 rp-online.de .de 79
7,178 sanspo.com .com 77
7,190 espn.in .in 48
7,206 hentaiera.com .com 57
7,225 ygosu.com .com 64
7,239 gala.de .de 76
7,247 nishinippon.co.jp .jp 69
7,268 nzz.ch .ch 76
7,275 baltimoresun.com .com 76
7,324 nordkurier.de .de 69
7,345 rpp.pe .pe 90
7,349 rae.es .es 61
7,353 hentaifox.com .com 59
7,355 bandlab.com .com 59
7,363 bhaskar.com .com 75
7,392 delish.com .com 76
7,402 bitchute.com .com 42
7,423 jiji.com .com 81
7,427 nami.org .org 84
7,432 nouvelobs.com .com 87
7,437 espn.com.br .br 48
7,455 amazon.sg .sg 40
7,465 nationalgeographic.org .org 82
7,468 giallozafferano.it .it 87
7,510 indiewire.com .com 75
7,514 biography.com .com 76
7,527 statnews.com .com 89
7,531 theonion.com .com 84
7,553 ridewithvia.com .com 86
7,556 lindependant.fr .fr 86
7,562 nettavisen.no .no 63
7,582 americanas.com.br .br 81
7,589 brasil247.com .com 80
7,614 cmjornal.pt .pt 86
7,647 sporza.be .be 87
7,680 wordplays.com .com 56
7,692 jetpunk.com .com 46
7,738 pitchfork.com .com 81
7,774 autoscout24.it .it 86
7,821 gala.fr .fr 90
7,834 record.pt .pt 69
7,856 popsugar.com .com 75
7,862 forocoches.com .com 59
7,863 refinery29.com .com 78
7,981 afr.com .com 72
8,015 allporncomic.com .com 65
8,030 oceanofpdf.com .com 59
8,055 gendai.media .media 83
8,094 nefisyemektarifleri.com .com 76
8,100 loudecho.ai .ai 56
8,104 x-x-x.tube .tube 84
8,127 my-personaltrainer.it .it 79
8,173 leggo.it .it 62
8,193 khaleejtimes.com .com 82
8,221 tubidy.llc .llc 75
8,227 sex-studentki.sale .sale 57
8,263 hochi.news .news 78
8,268 litmarket.ru .ru 87
8,270 bestjavporn.com .com 66
8,378 revistaoeste.com .com 69
8,380 veikkaus.fi .fi 67
8,385 emojicombos.com .com 47
8,412 anikototv.to .to 62
8,435 thecut.com .com 84
8,436 beatport.com .com 54
8,439 espn.com.ar .ar 48
8,449 sextb.net .net 62
8,450 complex.com .com 91
8,484 sciencenews.org .org 86
8,511 demonicscans.org .org 57
8,528 diariodocomercio.com.br .br 89
8,541 91porna.com .com 88
8,560 presseportal.de .de 87
8,568 kriesi.at .at 77
8,606 javdb.com .com 52
8,621 momvids.com .com 50
8,692 olympusxyz.com .com 56
8,773 epey.com .com 52
8,789 supremevalues.com .com 70
8,793 vinted.es .es 86
8,803 libramemoria.com .com 91
8,815 olimpbet.kz .kz 44
PreviousPage 2 of 12Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Google-Extended reference, the full census, and the JSON endpoint.