Website Crawler
Use this tool to investigate the signal category highlighted in the report.
This report is based on a stored AlphaCrawler crawl snapshot and summary metrics.
Stored crawl snapshot available.
17 Jul 2026 20:50:58
Crawling time:00:01:21
Pages/Seconds2.5
AI search crawlers are allowed and the homepage content is readable without JavaScript. Review the breakdown below to push the score higher.
No robots.txt found — by default every crawler below is allowed.
| Googlebot Google · Google Search & AI Overviews | Allowed |
| Google-Extended Google · Gemini grounding & training | Allowed |
| GPTBot OpenAI · ChatGPT training & retrieval | Allowed |
| OAI-SearchBot OpenAI · ChatGPT Search index | Allowed |
| ChatGPT-User OpenAI · ChatGPT live browsing | Allowed |
| ClaudeBot Anthropic · Claude crawler | Allowed |
| Claude-User Anthropic · Claude live browsing | Allowed |
| PerplexityBot Perplexity · Perplexity index | Allowed |
| Perplexity-User Perplexity · Perplexity live fetch | Allowed |
| Bingbot Microsoft · Bing & Copilot | Allowed |
| Applebot-Extended Apple · Apple Intelligence training | Allowed |
| CCBot Common Crawl · Open dataset many LLMs use | Allowed |
| Meta-ExternalAgent Meta · Meta AI training | Allowed |
| Bytespider ByteDance · ByteDance / TikTok AI | Allowed |
| Amazonbot Amazon · Alexa & Amazon AI | Allowed |
| Weight | Status Code | Url | Issues |
|---|---|---|---|
| 0 | 200 |
https://robinsonarmament.com/product-category/inventory-blowout/?add-t ...
Title: Inventory Blowout Archives - Robinson Ar ... Description: heavily discounted xcr rifles and pistol ... H1: Inventory Blowout |
Canonical ≠ URL
More than one Title tag on page
|
| 0 | 200 |
https://robinsonarmament.com/configure/configure/14829/
Title: Configure - Robinson Armament Co. Description: [empty] H1: Configure |
Canonical ≠ URL
More than one Title tag on page
Description is missing
|
| 0 | 200 |
https://robinsonarmament.com/product-category/inventory-blowout/?add-t ...
Title: Inventory Blowout Archives - Robinson Ar ... Description: heavily discounted xcr rifles and pistol ... H1: Inventory Blowout |
Canonical ≠ URL
More than one Title tag on page
|
| 0 | 200 |
https://robinsonarmament.com/300-blackout/
Title: 300 Blackout - Robinson Armament Co. Description: 300 blackout is perfect for the XCR-L wh ... H1: 300 Blackout |
More than one Title tag on page
|
| 0 | 404 |
https://robinsonarmament.com/300blackout/
|
4xx Client errors
404 Page
Broken page
|
| 0 | 301 |
https://robinsonarmament.com/xcr-l-light-caliber
|
301 Redirects
Redirect
|
| 0 | 200 |
https://robinsonarmament.com/category/featured/
Title: Featured Archives - Robinson Armament Co ... Description: [empty] H1: Featured Archives - Robinson Armament Co ... |
More than one Title tag on page
Description is missing
|
| 0 | 301 |
https://robinsonarmament.com/xcr-rifle-sbr-or-pistol
|
301 Redirects
Redirect
|
| 0 | 200 |
https://robinsonarmament.com/wp-login.php?redirect_to=https://robinson ...
Title: Log In ‹ Robinson Armament Co. — WordPre ... Description: [empty] H1: Log In |
Meta Noindex Pages
More than one H1 on page
Canonical is missing
Description is missing
|
| 0 | 200 |
https://robinsonarmament.com/author/ajr/page/2/
Title: Alex Robinson, Author at Robinson Armame ... Description: [empty] H1: Alex Robinson, Author at Robinson Armame ... |
More than one Title tag on page
Description is missing
|