Audit a website for AI search readiness for free
Awesome GEO is a free, open-source catalog for Generative Engine Optimization with 8 free browser tools. This page uses them to audit one website for AI search readiness in five steps, each ending with what to write down.
Before you start
This page is written by the maker of Awesome GEO, Dmitry (Dee) Kargaev. The tool pages behind these steps were read on 11 November 2026. You need the full address of your home page and of one key page, plus a notes file. Step 5 also uses a server access log and your analytics, if you have them.
Step 1: Run Site check on two pages
Site check shows whether AI crawlers can reach and read a page. Choose the “URL” button, type the full address into “Page URL” and press “Check”. The results open with “Problems”, a count and a list, or “No problems found”. “Crawler access” gives one chip per AI crawler: allowed, partial, blocked or unknown. “Live fetch as each bot” shows the HTTP status code each crawler gets. A footnote says it tests the user agent only, not verified IP ranges.
Further down sit “Indexing”, “Page”, “Structured data” and “Files”. The “Page” section includes “Words without JavaScript”. Repeat the run for your key page: change the address and press the button, which now reads “Check again”.
Write down, for both pages, the “Problems” list, every crawler chip that is not allowed, the status codes, the value of “Words without JavaScript”, the “JSON-LD types” and any “Parse errors”.
Step 2: Compare your robots.txt with a preset
The robots.txt tool generates and checks rules for AI crawlers. Type your domain into “Site” and press “Load”. A note says the file was loaded, or that the site has no robots.txt, so every bot is allowed. Then pick a preset: “Let AI search read me, keep training bots out” (selected by default), “Allow all” or “Block all AI”. Under “Bots”, crawlers are grouped as Search, Training, User-triggered and Other. Click a chip to switch one crawler; a crawler set differently from its group is tagged “override”.
Compare the file in the “robots.txt” section with your live file. Record the preset you want and each crawler where the two differ.
Step 3: Check entity markup and the six profile platforms
Person or Organization markup with a “sameAs” list is what this step looks for. The “JSON-LD types” row from step 1 shows whether the type is present; open that markup and look for a list of profile URLs. The Schema tool generates “Person” and “Organization” JSON-LD, each with a “Profiles (sameAs)” field.
The Presence planner picks where to set up profiles. Choose “Person” or “Organization” under “Type” and, if you like, add a few words to “Focus”. “Start here” lists six platforms from the Awesome GEO catalog, each with its rank, name, description and score. The “sameAs” section shows a JSON-LD block whose profile URLs end in placeholders for your handle.
Record whether the markup exists and has a sameAs list, then the six platforms in order, marking where you already have a profile.
Step 4: Fill in the Answer panel tally
The Answer panel tracks whether ChatGPT, Claude, Gemini and Perplexity name your product, but it never queries them. You ask the assistants yourself, and the tallies stay in your browser. Fill in “Product” and “Category” (both required) and up to three “Competitor” fields, then press “Write questions”. You get five: comparison, alternatives, best-of, how-to and pricing. “Copy all” copies them together.
Ask each question in each of the four assistants. In the “Tally” table, click a cell to switch it between absent, named and cited only. The “Named” row then shows counts such as “2 of 5”.
Write down the “Named” count per assistant and the questions where your product is absent.
Step 5: Read who crawls and who sends visits
The Crawler log reader finds AI crawler requests in a server access log. Paste log lines, drop a file or press “Choose file”; “Try the sample log” loads sample data. It reads combined-format lines from nginx and Apache and JSON lines from Caddy and Cloudflare, and nothing is uploaded. “AI crawlers” gives a request total and, per crawler, its operator, its three most requested paths, “Requests” and “Last seen”. If none appear, you see “No AI crawler requests in this log.” The “Files” section shows what /robots.txt, /sitemap.xml and /llms.txt received and the latest status code.
The AI traffic filter builds a filter for visits referred by AI assistants. Ten assistants are checked under “Assistants”, and a “More” toggle shows 15 further ones. Under “Filter”, “GA4 regex” gives one expression labelled “Session source matches regex”, “Hostnames” lists referrer hostnames, and “grep” gives a grep -E command for a combined access log. Its “What this misses” section says Google AI Overviews and AI Mode count as normal Google search clicks, Bing’s AI Performance report shows citations and no clicks, and assistant apps mostly send no referrer, so those visits land in Direct. Treat any number as a floor.
Write down the crawlers, their operators and last-seen dates, the status codes for your three files, and the AI-referred visits your analytics reports with the filter applied.
If a tool does not work
Site check refuses local and private addresses and responses over 1,000,000 bytes. If its fetch fails, it offers “Paste HTML instead” and “Try again”. In Paste HTML mode you paste the page source into “Page HTML”, and the tool skips “Crawler access”, “Live fetch as each bot”, “X-Robots-Tag”, “In sitemap.xml” and “Files”. If the robots.txt tool finds no file, its note says every bot is allowed.
Other free checkers and what each adds
The four checkers below are described on their own pages, as read in November 2026.
Digital Visibility needs no sign-up or card. It crawls up to 25 pages, runs Google’s speed test on a phone and checks which search engines, AI answer tools and AI training bots the site lets in. One free audit is allowed every 30 days.
Albright Labs asks Perplexity, Claude and OpenAI “what does your website do?” and shows each answer side by side. It asks for a website URL, a business name and an email, and sends the report link by email.
index365 scores schema, llms.txt, robots and AI-crawler policy from 0 to 100. Its free plan costs $0 and includes 2 homepage scans a month; paid plans add any single page on your domain.
queryclear checks up to five public pages and writes up to five modeled buyer-answer scenarios. It does not query ChatGPT, Claude, Google AI Overviews or another named answer engine, and it calls the scenarios model-generated hypotheses, not live engine measurements.
A checklist beyond the five steps
SEEN, the four-layer framework of Structure, Evidence, Entity and Notability, includes a 41-item practitioner checklist in its SSRN paper.