On a large site, a technical fault is almost never one page. It is a template, a rule or a setting that repeats across thousands of URLs.
Quick answer
Segment the crawl by template and section, compare it with Search Console, check how indexation is controlled, deal with duplicate URL patterns such as facets and parameters, keep sitemaps clean, and monitor at template level. Treat crawl budget as a real concern only when the site is very large or has many pages that Google has found but not indexed.
1. Segment by template
Start by listing the page types: categories, products, articles, tags, authors, locations, search pages and so on. Review each separately, because each has its own risks and its own owner. See how to scope an SEO audit for a large website.
The same applies to reporting. A finding such as "eight H1 tags on the homepage" or "no canonicals on filtered URLs" is a template-level finding. On a retail audit with 288 indexed URLs, a single template fault, faceted navigation with no canonical tags, caused about 130 near-duplicate pages. Fixing the template removed the problem.
2. Crawl and indexation checks
- Which sections get crawled and indexed, and which should
- Indexable versus non-indexable URL patterns
- Canonicals on every template
- robots.txt and meta robots directives
- XML sitemap coverage by section
- Crawled URLs against Search Console indexing data, and the reasons for "Discovered" and "Crawled, currently not indexed"
3. Crawl budget
Crawl budget is often overstated. Google's guidance is that it mainly matters for very large sites, or for sites where content changes very quickly, or where a large number of pages show as discovered but not indexed. For most sites, the problem is the quality and structure of the pages, not how many times Googlebot visits.
If it does apply:
- Stop generating crawlable duplicates, such as parameter and filter combinations
- Keep internal search and basket pages out of the crawl
- Remove redirect chains and soft 404s
- Keep sitemaps to canonical, indexable URLs
- Use server logs to see what Googlebot actually requests
4. Facets, parameters and pagination
- Decide which filtered pages deserve to rank, and canonicalise or block the rest
- Make sure paginated pages are crawlable and self-canonical
- Control tracking and session parameters
- Check internal search results are not indexed
5. Internal linking and architecture
Check how deep important templates sit, whether priority pages have strong links, and whether any templates are orphaned. See internal linking for SEO.
6. Status codes and redirects
- Redirect chains and loops, by pattern
- Soft 404s and thin error pages
- Server errors on key templates
- Consistent handling of www, HTTPS and trailing slashes
7. International setups
If you use multiple languages or regions, check hreflang at template level. On a software agency audit, hreflang was broken on 422 pages because every tag pointed at the homepage. It was a template-level fault, and a conflicting hreflang signal across a whole template is easy to miss in a page-by-page review.
8. Structured data and performance by template
Check that structured data is present and valid on every relevant template, and measure Core Web Vitals by template, not only for the homepage. On the same agency audit, the homepage had a Largest Contentful Paint of about 4.7 seconds, while the average across the sample was about 7 seconds. Key service pages were worse than the homepage.
9. Monitoring at scale
- Regular crawls compared with the previous one
- Alerts for template-level changes in status, canonicals and noindex
- Search Console indexing and crawl reports by section
- QA before releases that touch templates, and after them
Common mistakes
- Reviewing individual URLs instead of patterns
- Ignoring faceted navigation
- Missing canonicals at template level
- Letting weak URLs into sitemaps
- Not checking how a fix scales across the template
When to get support
If repeated technical issues keep appearing, Technical SEO covers template-level crawl, index and performance reviews, and the Visibility Review puts the findings in priority order. For checking the basics first, see how to check if Google can crawl and index your site.
FAQ
What makes technical SEO different for large websites?
Scale. A single template issue can affect thousands of URLs, so the review focuses on patterns and systems.
Does crawl budget matter for my site?
Usually not, unless the site is very large, changes very quickly, or has many pages that are discovered but not indexed.
How often should a large site be crawled?
Regularly, and always after releases that change templates, so new issues are caught early.