# fairly.day — crawler policy (AUDIT-2026-09-22.md §6)
# Search-and-cite AI crawlers are allowed on purpose so AI answers can cite Fairly.
# Content-Signal (https://contentsignals.org) states how the content may be used once fetched:
# search indexing and AI answers grounded in it (ai-input) yes, model training (ai-train) no.
# A crawler obeys only the most specific group that names it, so the AI crawlers named here share
# the wildcard group (its signal and its Disallow lines) instead of standing in groups of their own.
# CCBot (Common Crawl, a training corpus) is blocked.

User-agent: *
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: ClaudeBot
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: Google-Extended
User-agent: Bingbot
Content-Signal: search=yes, ai-input=yes, ai-train=no
Allow: /
Disallow: /get
Disallow: /es/get
Disallow: /aso/
Disallow: /es/aso/
Disallow: /demo/
Disallow: /es/demo/

User-agent: CCBot
Disallow: /

Sitemap: https://fairly.day/sitemap-index.xml