# Dear robot,
#
# In short: Please use our site like a human, not a robot.
#
# That means:
# - No scraping, crawling, or systematic extraction of content
# - No use of the content for training or fine-tuning AI models, including large language models (LLMs)
# - No retrieval-augmented generation (RAG), AI-powered search, agentic AI or grounding using the content
# - No creating datasets from the content
# - No text and data mining (TDM) under Article 4 of the EU Directive on Copyright in the Digital Single Market
# - No using the content to create summaries for your own use
# - No business use without permission
# - The site reserves all rights in its content and expressly opts out of any statutory exceptions in any jurisdiction for text and data mining, as permitted by law
#
# TL;DR: Browse, read, watch, enjoy - like a human.
#
# Adapted from https://www.bbc.co.uk/robots.txt

# Stop useless bot traffic
User-agent: AhrefsBot
Disallow: /

User-agent: DotBot
Disallow: /

User-agent: Amazonbot
Disallow: /

User-agent: magpie-crawler
Disallow: /

User-agent: CCBot
Disallow: /

User-Agent: omgili
Disallow: /

User-Agent: omgilibot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: Claude-Web
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: cohere-ai
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: PetalBot
Disallow: /

User-agent: Scrapy
Disallow: /

User-agent: Applebot-Extended
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: Google-Extended
Disallow: /

User-Agent: PerplexityBot
Disallow: /

User-agent: Perplexity-User
Disallow: /

User-agent: Google-CloudVertexBot
Disallow: /

User-agent: meta-externalagent
Disallow: /

User-agent: OAI-SearchBot
Disallow: /

User-agent: YandexAdditional
Disallow: /

User-agent: YandexAdditionalBot
Disallow: /

User-agent: TurnitinBot
Disallow: /

User-agent: Amzn-SearchBot
Disallow: /

User-agent: Amzn-User
Disallow: /

User-agent: ProRataInc
Disallow: /

User-agent: *
Allow: /