How Does Claude Find and Cite Sources? What Anthropic Documents

Two Anthropic documents cover the mechanics: the web search tool reference for developers building on the Claude API, and the crawler article for site owners. Neither is written for marketers, which is why they are more useful than most GEO advice: they describe what the system does rather than what someone hopes it rewards.

When Claude decides to search

The web search reference states the control plainly: “Claude determines when to search based on the prompt.” It then lists what pushes a request toward a live search: “Recent events, news, or announcements,” “Current prices, rates, scores, or statistics,” and “Information about specific organizations, people, or products that might have changed.” Established facts, creative writing and analysis of material already in the conversation are answered directly.

For a business, the third item is the one to underline. A question about your services, your prices, your locations or your reputation is, by Anthropic’s own description, the kind of question that gets a search rather than a recollection. What Claude reads when it searches is your live web presence as indexed and fetched today, which is the whole reason entity clarity on your current pages matters more than what the model may have seen in training.

How the search loop runs

The reference describes three steps:

1. “Claude determines when to search based on the prompt.”
2. “The API runs the searches and provides Claude with the results. This process can repeat multiple times throughout a single request.”
3. “At the end of its turn, Claude provides a final response with cited sources.”

On volume, Anthropic says “simple factual queries typically use 1–3 searches; comparative or multientity research can use 10 or more.” That is the same shape as the fan-out behavior we describe for Google AI Mode: a comparison or a “best option for my situation” question becomes a set of narrower retrievals, and a page earns a citation by being the right answer to one of them.

What a citation actually contains

This is the most useful paragraph in the document for anyone who writes web pages. Citations are “always enabled for web search,” and each one includes the source url, its title, and cited_text, defined as “up to 150 characters of the cited content.” Search results also carry page_age, documented as “when the site was last updated.”

A 150-character window is roughly one plain sentence. If the sentence that answers the question exists on your page, it can be quoted. If the answer is spread across a paragraph, hedged into a clause, or only implied by a table the model has to interpret, the quote is weaker or goes to a competitor whose sentence fits. This is the concrete, documented reason behind the advice to write self-contained, quotable answers, and behind our glossary entry on what an AI citation is.

Documented behavior, and what it implies for your page

The left column is Anthropic’s documentation, quoted or closely paraphrased. The right column is our reading of what it means for a business site. Keep the two separate in your own thinking; the left column is fact and the right column is inference.

The three agents, and which ones decide visibility

Anthropic’s crawler article names three user agents and gives each a job:

• Claude-SearchBot indexes pages to “improve the relevance and accuracy of search results.” This is the agent that determines whether your page is in the pool a search can return.
• Claude-User “supports Claude AI users” by fetching a page when a person asks about it. Anthropic says disabling it “may reduce visibility in user-directed web search.”
• ClaudeBot collects content for training. That is a separate decision about whether your words shape future models, and Anthropic does not document it as affecting search visibility.

All three “respect ‘do not crawl’ signals by honoring industry standard directives in robots.txt,” and Anthropic publishes its IP ranges at claude.com/crawling/bots.json. The failure we see most often is a 2023-era robots rule or firewall list that blocked “anthropic” wholesale, which removed the site from Claude answers while the owner believed they had only opted out of training. The full agent-by-agent breakdown across vendors is in which AI crawlers visit your site.

Developer controls you cannot influence, and one you can

Two parameters in the reference are set by whoever builds a Claude-powered product, not by publishers. allowed_domains and blocked_domains restrict which sites a given application searches (“provide allowed_domains or blocked_domains, not both”), so a vertical assistant built on Claude may search only an allow-list you are not on. There is no documented way to buy onto that list.

The third parameter, user_location, localizes results to a city, region, country or timezone. You cannot set it, but you can make your pages match it: named service areas, a real address, and location facts in the HTML rather than in a map embed. For a local business this is the one documented lever that is entirely in your hands.

What to do with this

1. Confirm access. Claude-SearchBot and Claude-User must get a 200 from your server and CDN, not a challenge page. Check robots.txt and firewall rules, then verify against the published IP list.
2. Write the sentence. For every question a page is meant to answer, make sure a plain sentence of 150 characters or fewer answers it, visible in the HTML.
3. Keep modified dates honest and recent. page_age is in the data the model sees; update content and the date together.
4. Say who you are, consistently. Questions about organizations trigger live search, so the organization facts on your pages are what gets read.
5. Measure it. Record what Claude answers for your customer questions each month and watch the trend, as described in how to measure AI search visibility.

This post is one engine in the “Engine by engine” section of our AI Search Playbook, which sequences the full program from access to measurement.

Frequently asked questions

About the work

Webvello is an AI search optimization and index engineering company founded in 2024, based in Roseville, California. We’ve delivered 50+ projects across 37+ cities, with average client ROI in the 3–5x range on properties where the technical foundation was the binding constraint. Founder George Shvaya came to search from cybersecurity and systems engineering, which is why this article reads the developer reference instead of repeating claims about what Claude “likes.” Our generative engine optimization work is built on the same documents.

Sources: Anthropic’s web search tool reference and crawler article, linked above and re-read on October 16, 2026. The implications column is Webvello’s inference from that documentation, not an Anthropic statement. Vendors change behavior; check the linked pages before relying on a detail.