EUAIScannerBot

EUAIScannerBot/1.0 is the crawler behind EU AI Scanner. It fetches publicly accessible pages of a site when a user requests an EU AI Act transparency scan of that site.

Behaviour

• Honors robots.txt (disallow rules and crawl-delay, capped at 2s)
• Fetches at most 250 pages per scan (usually 10–50), 4 requests in parallel
• Fetches a capped sample of images (≤20) and media files (≤5) to read provenance metadata, plus /llms.txt and /ai.txt if present
• Identifies itself with a stable User-Agent linking to this page
• Only crawls same-site links from the submitted URL
• Never submits forms, never attempts authentication

Blocking the bot

User-agent: EUAIScannerBot
Disallow: /

Questions or abuse reports: bot@euaiscanner.com