Applebot-Extended
Applebot-Extended is a robots.txt control token — not a crawler — that lets publishers opt their content out of training Apple's generative-AI foundation models (Apple Intelligence), without affecting Siri or Spotlight search.
Why it matters: it separates 'appear in Apple's search products' from 'be used to train Apple Intelligence' — blocking it keeps you out of AI training while your pages still show up for Apple users.
Introduced in 2024 for Apple Intelligence, Applebot-Extended has no separate user-agent: the actual crawling is done by Applebot. We leave it allowed here, as we do every -Extended token — the open-robots baseline under which Google indexed 10 of our 16 glossary test pages in a median of 3 days while Bing crawled 0 of 16 (our index-lag data). Adding User-agent: Applebot-Extended / Disallow: / to robots.txt tells Apple that content it already indexed is not eligible for training its general-purpose generative-AI models. Apple confirms pages that disallow Applebot-Extended “can still be included in search results,” so it’s the training opt-out that costs you nothing in Apple’s search surfaces. It works exactly like Google-Extended — a control token, not a bot. See the full AI crawlers reference. (Source: Apple — about Applebot.)