Crawl and indexation checks sit at the centre of technical SEO.
Quick answer
To check whether Google can crawl and index your site, review Google Search Console, test important URLs with URL Inspection, crawl the site, check robots.txt, review meta robots tags, inspect canonicals and make sure priority pages are included in clean internal links and XML sitemaps.
Crawled does not always mean indexed.
Google can discover a URL without indexing it. It can also index a page that is not the version you expected. That is why crawl and indexation checks need to look at several signals together.
Start with important pages.
Do not begin by checking every URL. Start with the pages that matter most: service pages, category pages, high-value articles, product listing pages and pages that support enquiries or revenue.
For Peacock Search, this same logic supports technical SEO work and audit reviews.
What to check.
- URL Inspection in Google Search Console
- Indexing reports for excluded or discovered URLs
- robots.txt rules
- Meta robots tags
- Canonical tags
- HTTP status codes
- Internal links to priority pages
- XML sitemap inclusion
Look for conflicting signals.
Indexation problems often come from mixed messages. A page might be in the sitemap but canonicalised elsewhere. It might be internally linked but blocked by robots.txt. It might be indexable but buried too deep in the site.
Practical checklist
- Pick priority URLs
- Inspect each URL in GSC
- Check crawl status and index status
- Review canonicals and directives
- Confirm internal links exist
- Request indexing only after fixing blockers
Common mistakes
- Assuming sitemap inclusion means indexation
- Only checking the homepage
- Missing accidental noindex tags
- Ignoring canonical conflicts
- Not comparing crawl data with GSC
When to get support
If this sounds familiar, Technical SEO gives you practical SEO recommendations, clear priorities and next steps that are easier to implement. This note also supports The Visibility Review.
FAQ
Can a page be crawled but not indexed?
Yes. Google can crawl a page and decide not to index it, especially if the page is duplicate, weak, blocked or not clearly valuable.
Should every page be indexed?
No. The aim is to index the pages that deserve to appear in search and keep low-value URL patterns under control.