Blocklist · Google

Sites that block Google-CloudVertexBot

1,361 of the 29,991 domains we have measured, 5%, disallow google-cloudvertexbot at the site root in robots.txt. Google-CloudVertexBot fetches a page live when a user asks the assistant for it.

Every domain here has removed itself from Google's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

1,361 of 29,991 measured domains (5%) disallow google-cloudvertexbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-21 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block Google-CloudVertexBot", measured 2026-11-21. https://crawlcensus.com/blocked/google-cloudvertexbot

The list 1,361 domains

RankDomainTLDAI access score
15,868 holoo12.com .com 84
15,873 mawartotogoku.com .com 74
15,879 mkvmoviespoint.cab .cab 72
15,882 moviesda.talk .talk 56
15,885 olxcun.com .com 75
15,888 pinayflix.app .app 51
15,890 prmovies.farm .farm 70
15,898 skymovieshd.spot .spot 53
15,900 tamilprint60.art .art 55
15,909 toto12ultra.com .com 75
15,910 toto12wind.com .com 77
15,918 yallashots.top .top 72
16,013 bunte.de .de 87
16,015 blogfreely.net .net 34
16,047 longisland.com .com 51
16,103 dweb.link .link 65
16,182 sermonaudio.com .com 71
16,209 ipfs.io .io 65
16,254 unil.ch .ch 83
16,256 runnerspace.com .com 47
16,365 xozilla.com .com 58
16,423 iwm.org.uk .uk 77
16,607 mediapost.com .com 45
16,664 lightningmaps.org .org 51
16,672 pressherald.com .com 76
16,765 scimagojr.com .com 57
16,991 u.gg .gg 42
17,042 northjersey.com .com 74
17,055 official.link .link 72
17,183 semafor.com .com 72
17,189 heavyfetish.com .com 62
17,224 harpers.org .org 79
17,285 nextech.com .com 78
17,303 businessoffashion.com .com 68
17,420 jsbin.com .com 53
17,469 palmbeachpost.com .com 73
17,492 ina.fr .fr 76
17,554 abc13.com .com 79
17,596 iisd.org .org 59
17,607 philpapers.org .org 65
17,634 dice.fm .fm 65
17,671 iafd.com .com 45
17,673 demilked.com .com 64
17,750 6abc.com .com 79
17,785 trakt.tv .tv 58
17,796 list.ly .ly 52
17,833 novelphoenix.com .com 61
17,877 browser-update.org .org 57
17,886 agrarheute.com .com 75
17,993 cawsg.com .com 55
18,218 globed.dev .dev 51
18,228 metafilter.com .com 51
18,266 thedirecthor.com .com 57
18,280 hoerzu.de .de 45
18,309 mans.io .io 60
18,373 instyle.de .de 89
18,444 ext.to .to 57
18,514 radaris.com .com 58
18,526 researchsquare.com .com 50
18,595 trycloudflare.com .com 52
18,687 romatoday.it .it 67
18,710 cftc.gov .gov 54
18,859 huntshowdown.com .com 60
18,883 courier-journal.com .com 74
18,964 theagenticx.ai .ai 71
18,973 fictionzone.net .net 69
19,014 lovart.ai .ai 51
19,060 internetsociety.org .org 75
19,063 opensecrets.org .org 46
19,206 pseb.ac.in .in 54
19,250 beatstars.com .com 51
19,278 laowang.vip .vip 53
19,291 gifyu.com .com 54
19,312 pinayflix.uk .uk 46
19,314 milanotoday.it .it 67
19,439 econsultancy.com .com 88
19,461 shamela.ws .ws 55
19,478 french-stream.one .one 70
19,538 xn--l3ca4b6dsc1b.com .com 77
19,564 playbill.com .com 68
19,576 moviesmod.at .at 77
19,614 daaz.com .com 43
19,662 aagmaal.kiwi .kiwi 69
19,672 wallhaven.cc .cc 45
19,678 skymovieshd.ceo .ceo 53
19,719 moviesda.blue .blue 56
19,755 100crorescompany.com .com 40
19,762 desixxx.blog .blog 65
19,810 boutique1st.com .com 56
19,860 zawya.com .com 80
19,861 meteo.gr .gr 76
19,902 kuttymovies.red .red 52
19,906 artificialintelligenceact.eu .eu 62
19,918 mkvmoviespoint.uno .uno 72
19,957 9xflix.gdn .gdn 80
19,979 maxwinexch99.com .com 27
19,989 uyeshare.cc .cc 64
20,037 flixtor.to .to 45
20,152 au.int .int 63
20,166 skift.com .com 82
20,225 niemanlab.org .org 51
20,281 3dporndude.com .com 60
20,318 supersport.hr .hr 51
20,356 fmoviess.org .org 67
20,362 thedodo.com .com 70
20,364 derwesten.de .de 83
20,676 haufe.de .de 76
20,825 newspulsenow.net .net 49
20,888 alibris.com .com 72
20,964 betsfan.com .com 59
21,038 mainpost.de .de 80
21,044 bps.org.uk .uk 66
21,179 odibets.com .com 66
21,182 desiporn.one .one 52
21,231 epochtimes.com .com 85
21,248 happystove.com .com 58
21,263 pornobengala.com .com 72
21,267 seher.no .no 70
21,388 indiaxnxxx.com .com 67
21,428 plovdiv24.bg .bg 80
21,449 thisamericanlife.org .org 58
21,504 designspiration.com .com 48
21,602 becekku.com .com 81
21,678 hdmaal.la .la 55
21,704 codeigniter.com .com 43
21,706 1x-chickenroad.com .com 44
21,786 tamilprint62.art .art 55
21,792 mysiswaplace.my .my 52
21,816 footba11.co .co 61
21,823 desy.de .de 62
21,874 jadlog.com.br .br 62
21,875 dlpsgame.com .com 53
21,923 romspure.cc .cc 66
21,929 automobile.tn .tn 64
21,934 paradoxscans.com .com 76
21,961 animesalt.link .link 70
21,976 deltaexploits.gg .gg 63
21,999 dailyhive.com .com 74
22,002 uzmovi.net .net 78
22,006 bfd269.com .com 56
22,114 onlybokep.com .com 84
22,128 niadd.com .com 55
22,201 mmsgram.com .com 74
22,220 servg1.net .net 35
22,230 durationmedia.net .net 61
22,256 wsrv.nl .nl 55
22,277 tiamanhwa.com .com 69
22,279 ldtax.gov.bd .bd 52
22,294 thepornbang.com .com 60
22,428 gametracker.com .com 52
22,478 freakyza.co.za .za 76
22,503 49s.co.uk .uk 39
22,547 vipyonoslots.com .com 58
22,548 mydesihd.com .com 72
22,555 burgerpixel.net .net 56
22,557 foxaholic.com .com 58
22,569 happymod.tube .tube 76
22,583 videystream.store .store 53
22,626 medicaldaily.com .com 72
22,633 mynimeku.com .com 73
22,657 believesport.com .com 56
22,666 trevts.com .com 60
22,671 slotor777.ua .ua 76
22,740 zbzoub.com .com 58
22,771 movies4u.clinic .clinic 82
22,804 udea.edu.co .co 57
22,810 gperya.com .com 41
22,820 voir-anime.to .to 73
22,827 bestguiltypleasure.com .com 57
22,838 desimms.com.co .co 77
22,854 alexisbarcelona.com .com 66
22,869 videq-l.ink .ink 68
22,891 lolpol.com .com 60
22,923 2trk.info .info 55
22,970 meujogodobicho.com.br .br 69
22,971 turskeserije.tv .tv 55
22,972 multimovies.study .study 64
23,007 ogomovies1.com.pk .pk 82
23,048 alexismiami.com .com 77
23,055 tiktokio.net .net 76
23,071 yomovies.business .business 70
23,073 ttks.tw .tw 49
23,081 satsitset.living .living 39
23,084 app.com .com 69
23,097 cinefreak.net .net 72
23,112 toto12express.org .org 69
23,123 1668manga.com .com 62
23,128 eauctionsindia.com .com 77
23,130 financemasterpro.com .com 79
23,133 javmyanmar.com .com 53
23,137 loteriasdehonduras.com .com 59
23,141 poizdato.net .net 60
23,143 samusar.com .com 46
23,157 mlnba.fun .fun 54
23,167 dubmv.xyz .xyz 35
23,177 pyppo.com .com 55
23,185 hoathinh3d.st .st 78
23,244 pinayot.com .com 73
23,246 mobilejsr.com .com 54
23,264 olxsob.com .com 83
PreviousPage 3 of 7Next

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the Google-CloudVertexBot reference, the full census, and the JSON endpoint.