Crawl access, extractable passages, schema
Audit your website for AI search.
You get one score out of 100 for your site, with AI readiness as one of eight pillars. You see whether your robots.txt lets crawlers in, whether your pages hold quotable passages, and what schema they carry. Analytics, logins and citation tracking aren't part of it, by design (the assessment reads only your public pages, nothing behind a door).
On this page 8 sections
- 01What is an AI SEO audit?
- 02Which AI crawlers should your robots.txt allow?
- 03Do AI assistants actually read llms.txt?
- 04How do AI answers choose which passages to quote?
- 05Does structured data help your AI visibility?
- 06What does Google say its AI search features need from you?
- 07Questions, answered
- 08Your next read
What is an AI SEO audit?
An AI SEO audit checks whether AI search systems can reach, read and quote your site. Some of it you can measure on your own pages: crawler access, passage clarity and schema. Open your robots.txt and check which bots you allow. Then open your key page. Does a direct answer follow each heading, or does the page make the reader hunt for one?
Much of what gets sold under this label is guesswork. Google says you need no specific optimization for AI Overviews and AI Mode, and that your existing SEO fundamentals remain worthwhile. So you don't need a separate checklist. What you need is your normal fundamentals, checked with AI systems in mind. And before you pay for anything, check your pages against them.
The free assessment reads the signals on your site as its AI readiness pillar, worth 10% of your score. It samples your homepage and up to eight other pages. But because it reads only your public pages, it cannot tell you whether any assistant cites you today.
Which AI crawlers should your robots.txt allow?
Allow the crawlers that surface your pages in AI search, then decide whether the one that gathers training content may read your site. Search your robots.txt for PerplexityBot, OAI-SearchBot and GPTBot. Perplexity says PerplexityBot is designed to surface and link sites like yours, and is not used to crawl for foundation models. It recommends you allow it. And for most sites, that is a fair default for your file.
OpenAI treats its settings as independent. You can allow OAI-SearchBot to show your pages in search results and, if you want, disallow GPTBot, which signals your content should not be used for training. Your site, your call. Changes can take about 24 hours to reach its search systems (so don't panic if nothing moves right away), so wait before you retest your file. Perplexity-User generally ignores your robots.txt, because a person requested one of your pages.
Your firewall can block a bot your robots.txt allows. Perplexity advises that if your site sits behind a WAF, you may need a rule that combines the user agent (PerplexityBot or Perplexity-User) with its published IP ranges. The assessment reads your robots.txt and your response headers, so you see what it can see of your site. But it cannot test your firewall rules from outside. Those checks are yours to run.
Do AI assistants actually read llms.txt?
Mostly not, so treat llms.txt as optional. Check your server logs for requests to /llms.txt and compare them with total AI bot hits. OtterlyAI's 90-day experiment recorded 84 visits to the file out of 62.1K AI bot hits. Across over 500M AI bot visits in 90 days, Limy counted just 408 targeting it. That is a rounding error, and nobody should build a strategy on it.
SE Ranking studied nearly 300,000 domains. Only 10.13% had the file, and it had no effect on how often LLMs cite a domain. Google says you don't need llms.txt for its generative AI features, since Google Search doesn't use it. And as of Q1 2026, one review found that no major AI company had committed to reading it.
One real use exists. Limy reports that coding agents such as Cursor and Claude Code routinely fetch /llms.txt when pointed at a documentation site. If you run docs, the file costs little to add. On a marketing site, spend that hour elsewhere: crawler access and clear passages come first (they're where the payoff is).
How do AI answers choose which passages to quote?
Your pages get quoted when an AI system retrieves candidate passages, ranks them, then attaches citations to the ones that support its claims. You want your passage among them. Read the sentence under each heading on its own. If you need the paragraphs around it to make sense, that chunk is weak. Passionfruit states that the model sees passages, not whole pages, so your page is only as quotable as its chunks. And you can rewrite them.
One practitioner explainer lists the factors that rank your passages: relevance, clarity, freshness, authority, corroboration and source diversity. You can check your page against all six. Passionfruit adds that your pages must match the sub-queries a model generates, called query fan-out, and not only the user's exact prompt. Cover one topic under specific headings and you give the system more to match. Blur several together, and you hand it less.
Ziptie.ai reports that only 12% of AI-cited URLs appear in Google's top 10 organic results for the same query. Treat that as one vendor's figure, not a law. Test it against your own queries. The assessment cannot see how a model retrieves your pages. Your headings and page structure, though, are visible, and you can change both.
Does structured data help your AI visibility?
Not by itself, and you don't need special markup. Google says you don't need structured data for generative AI search. There is no special schema.org markup to add. View source on your key page and find your structured data, then check that it matches what you and your visitors read on the page. And a mismatch is worse than no markup at all (a wrong claim in your markup is still a claim, and someone will read it). Your markup and your page should say the same thing, and you can verify that in your own browser.
Google still recommends structured data as part of your overall SEO, because it helps your pages qualify for rich results. That is the honest case. You can measure a benefit in classic search, but nothing is promised for AI answers. If a vendor tells you a schema type will get you cited, ask for evidence you can check on your own site. Do that first, before you touch your markup.
The assessment scores your schema as its own pillar, worth 10% of your score, separate from your AI readiness at another 10%. It reads the structured data on your sampled pages and shows you what it finds, so you can compare it with your source. Did any assistant use your markup to build an answer? The assessment can't tell you. And no honest tool reading your public pages could.
What does Google say its AI search features need from you?
Google says your page needs to be indexed and eligible to show in Search with a snippet to be a supporting link. There are no additional technical requirements, so check your key pages. You don't need new machine-readable files, AI text files, or Markdown on your site. And you can skip llms.txt too, because Google Search doesn't use them.
Google says you don't need specific optimization, and your SEO fundamentals stay worthwhile. Its advice: focus on your visitors and give them unique, satisfying content. Chase inauthentic mentions of your site across the web, and Google says you gain less than you think. Its generative AI features depend on core ranking systems and spam blocking, so there's no side door.
Measure your outcomes beyond clicks. Google says clicks from search results pages with AI Overviews are higher quality, so you can watch your sales, signups, engaged audience or information lookups. Google is rolling out Search Console reports for AI Overviews and AI Mode impressions to a subset of sites. So check whether yours is in that subset.
Questions, answered.
01Is an AI SEO audit different from a regular SEO audit?
Mostly no. Google says specific optimization isn't required for AI Overviews and AI Mode, and that SEO fundamentals continue to be worthwhile. Your page still needs to be indexed and eligible for a snippet. The extra checks are crawler access for other AI bots and clear, self-contained passages.
02Should I block GPTBot?
That is your decision, and the two settings are separate. OpenAI says GPTBot crawls content that may be used for training, and disallowing it signals you don't want that. Its OAI-SearchBot setting is independent. You can allow search appearance while opting out of training, and neither choice forces the other. Check your robots.txt for both names.
03Will the assessment tell me whether ChatGPT or Perplexity cites my site?
No. The assessment reads your public pages: headers, markup, structured data, headings, and robots and canonical signals. It does not log in, read analytics or interview anyone. It shows whether your site is reachable and quotable, which you can measure. Has any assistant chosen to cite you? That is outside its view, and it says so.
04What does the free assessment ask for, and how often can I run it?
It takes about two minutes and asks for a work email, where it sends your report link once. You can run three checks per hour per connection. And you get one score out of 100, with the evidence behind each of the eight pillars, including AI readiness.
Sources
Facts on this page come from the documents below, checked on 2026-10-24. Figures about the assessment itself describe how it works.
- Perplexity Crawlers - Perplexity
- Overview of OpenAI Crawlers
- llms.txt and AI Visibility: Results from OtterlyAI's GEO Study
- LLMs.txt in 2026: The Full Guide
- LLMs.txt Guide: What It Does and Doesn't Do (2026)
- LLMs.txt: Why Brands Rely On It and Why It Doesn’t Work
- AI Features and Your Website | Google Search Central | Documentation | Google for Developers
- Introducing Search Generative AI performance reports in Search Console | Google Search Central Blog | Google for Developers
Free automated assessment