Blocklist · Microsoft

Sites that block msnbot

18 of the 29,991 domains we have measured, 0.1%, disallow msnbot at the site root in robots.txt. msnbot builds the index that decides whether the operator's assistant cites a page.

Every domain here has removed itself from Microsoft's answers. Blocking a retrieval agent withholds nothing from model training, because training uses a different token.

The measurement

18 of 29,991 measured domains (0.1%) disallow msnbot at the site root in robots.txt. Every domain named on this page was fetched directly and its robots.txt parsed against RFC 9309; none of it is inferred from a third-party index.

Measured 2026-11-22 by direct request to every domain in the corpus. Method: how this is measured. Reuse under CC BY 4.0 with attribution.

Cite as: Crawl Census, "Sites that block msnbot", measured 2026-11-22. https://crawlcensus.com/blocked/msnbot

The list 18 domains

RankDomainTLDAI access score
3,872 cia.gov .gov 49
5,840 uzh.ch .ch 79
14,805 redalyc.org .org 52
16,122 page-stats.de .de 49
16,556 tourvisor.ru .ru 91
21,162 psytests.org .org 69
22,908 redvelvet.co.za .za 73
24,525 zin.ru .ru 41
25,638 truenetwork.ru .ru 70
25,826 cnnic.cn .cn 58
25,857 ly.com .com 69
28,163 segmento.ru .ru 70
28,335 giant-bicycles.com .com 89
29,121 pucsp.br .br 55
30,393 ecode360.com .com 44
34,818 securitm.ru .ru 92
37,425 jamestown.org .org 82
43,837 primicias.ec .ec 93

Rank is the domain's position in the popularity list used to seed the corpus; a dash means it was scanned on request rather than seeded. Score is the site's overall AI access score, which blocking depresses but does not by itself determine.

Check a specific domain

See also the msnbot reference, the full census, and the JSON endpoint.