Website Crawler
Use this tool to investigate the signal category highlighted in the report.
This report is based on a stored AlphaCrawler crawl snapshot and summary metrics.
Stored crawl snapshot available.
3 Jul 2026 12:27:25
Crawling time:00:01:28
Pages/Seconds2.3
AI search crawlers are allowed and the homepage content is readable without JavaScript. Review the breakdown below to push the score higher.
No robots.txt found — by default every crawler below is allowed.
| Googlebot Google · Google Search & AI Overviews | Allowed |
| Google-Extended Google · Gemini grounding & training | Allowed |
| GPTBot OpenAI · ChatGPT training & retrieval | Allowed |
| OAI-SearchBot OpenAI · ChatGPT Search index | Allowed |
| ChatGPT-User OpenAI · ChatGPT live browsing | Allowed |
| ClaudeBot Anthropic · Claude crawler | Allowed |
| Claude-User Anthropic · Claude live browsing | Allowed |
| PerplexityBot Perplexity · Perplexity index | Allowed |
| Perplexity-User Perplexity · Perplexity live fetch | Allowed |
| Bingbot Microsoft · Bing & Copilot | Allowed |
| Applebot-Extended Apple · Apple Intelligence training | Allowed |
| CCBot Common Crawl · Open dataset many LLMs use | Allowed |
| Meta-ExternalAgent Meta · Meta AI training | Allowed |
| Bytespider ByteDance · ByteDance / TikTok AI | Allowed |
| Amazonbot Amazon · Alexa & Amazon AI | Allowed |
| Weight | Status Code | Url | Issues |
|---|---|---|---|
| 0 | 200 |
https://www.theaa.com/used-cars/used-car-market-trends
Title: Used Car Market Trends | Pricing & Deman ... Description: [empty] H1: Used car market trends |
Description is empty
|
| 0 | 301 |
https://www.theaa.com/cars/advice/best-used-cars/best-used-second-hand ...
|
301 Redirects
HTTPS to HTTP redirect
HTTPS/HTTP mixed internal links
Redirect
|
| 0 | 200 |
https://www.theaa.com/leasing/vans/volkswagen
Title: Volkswagen Van Lease Deals | AA Lease Description: [empty] H1: Volkswagen van lease deals |
Description is empty
|
| 0 | 200 |
https://www.theaa.com/used-vans/local/greater-manchester
Title: Used Vans for Sale in Greater Manchester ... Description: High quality used and second hand vans f ... H1: Used vans for sale in Greater Manchester ... |
Title too long
|
| 0 | 200 |
https://www.theaa.com/cars/finance/eligibility-check
Title: Car Finance Eligibility Checker, Car Loa ... Description: Free finance eligibility check from AA C ... H1: Car Finance Eligibility Checker |
Title too long
|
| 0 | 200 |
https://www.theaa.com/cars/advice/general-advice
Title: General Advice | AA Cars Description: [empty] H1: General Advice |
Title too short
Description is empty
|
| 0 | 200 |
https://www.theaa.com/cars/advice/best-cars
Title: Best Cars | AA Cars Description: [empty] H1: Best Cars |
Title too short
Description is empty
|
| 0 | 200 |
https://www.theaa.com/leasing/vans/by-type/high-roof
Title: High-Roof Van Leasing | AA Lease Description: [empty] H1: [empty] |
Description is empty
H1 is missing
|
| 0 | 200 |
https://www.theaa.com/cars/account/login
Title: Sign in to your account | AA Cars Description: [empty] H1: Sign in |
Meta Nofollow Pages
Meta Noindex Pages
Description is empty
Canonical is missing
|
| 0 | 200 |
https://www.theaa.com/leasing/vans/nissan
Title: Nissan Van Lease Deals | AA Lease Description: [empty] H1: Nissan van lease deals |
Description is empty
|