How it discovers and cites pages
Claude's web search retrieves candidate pages via the Brave Search index, then fetches selected URLs to read their content before answering, attributing claims to the sources it used. Anthropic operates distinct agents for different purposes — ClaudeBot for training-data crawling, Claude-SearchBot for building search results, and Claude-User for live user-initiated fetches — each of which you can allow or disallow individually in robots.txt.
What this means for getting cited
To be cited by Claude, your page first has to be reachable through the index it draws on (Brave Search API (plus Anthropic's own crawlers)) and crawlable by the relevant bots, then it has to answer the user's question clearly enough to be quoted. I test exactly these variables at HZ and publish what moves the needle.