# These rules apply to all crawlers, including search and AI crawlers. # Public guide downloads remain crawlable even though they use a storage API. User-agent: * Allow: /api/storage/objects/ Allow: /api/storage/public-objects/ Disallow: /admin Disallow: /cms Disallow: /api/ Allow: / Sitemap: https://castlestonebridge.co.uk/sitemap.xml # Human-readable page directory: https://castlestonebridge.co.uk/site-index # AI-readable website summary: https://castlestonebridge.co.uk/llms.txt # Crawl directives are not authentication; private routes also require sign-in.