The Complete Technical SEO Handbook: August 2025 Edition

Picture this: a client comes to you convinced their content is the problem. They've rewritten every page twice. They hired a copywriter. They're now debating a full rebrand. But when you pull a crawl report, you find 40% of their pages aren't even being indexed. The content was never the issue. The foundation was broken. That's what a solid technical SEO guide saves you from — wasted months chasing the wrong fix.

This handbook covers everything that lives under the hood of a well-optimized site. Crawlability. Indexation. Site speed. Core Web Vitals. Mobile. Structured data. Canonicals. Redirects. Log file analysis. JavaScript SEO. Each section is built to give you both the why and the what-to-do-about-it. This is the August 2025 edition — updated to reflect where Google's infrastructure and AI-driven search are pushing the discipline right now.

Whether you're an agency owner running audits for clients or an in-house marketer trying to explain crawl budget to your dev team, this is the reference you keep open in a tab.

Table of Contents

1. Crawlability: Making Sure Google Can Find Your Pages
2. Indexation: From Crawled to Ranking
3. Site Speed and Core Web Vitals
4. Mobile SEO and Responsive Architecture
5. Structured Data and Schema Markup
6. Canonicals: Controlling Duplicate Content
7. Redirects Done Right
8. Log File Analysis: What Googlebot Is Actually Doing
9. JavaScript SEO
10. Quick Reference Cheat Sheet
11. Where to Start

Crawlability: Making Sure Google Can Find Your Pages

Crawlability is the most foundational layer of technical SEO and the most commonly overlooked. If Googlebot can't reach a page, nothing else you do matters. No amount of great content or backlinks will move a page that's blocked, orphaned, or buried six clicks deep in your site architecture.

Start with your robots.txt file. It's a simple text file, but I've seen enterprise sites accidentally disallowing their entire /products/ directory after a platform migration. Happens more than agencies want to admit. Use Google Search Console's robots.txt tester to validate it before you assume it's clean.

Crawl budget matters most for large sites — typically anything above 10,000 pages. Google allocates a crawl rate based on your server's response time and the perceived value of your pages. Slow servers burn crawl budget on latency. Thin or duplicate pages waste it on content that shouldn't be crawled in the first place. The fix is usually twofold: improve server response time and reduce indexable junk.

For internal linking, depth is everything. Pages more than three clicks from the homepage tend to get crawled less frequently. Map your most important pages and make sure they're reachable within two to three hops from high-authority pages. A flat architecture almost always outperforms a deep one for crawl efficiency.

Your XML sitemap should be a clean list of canonical, indexable URLs — nothing more. Don't include noindex pages, redirect chains, or broken URLs. Submit it in Search Console and check the coverage report regularly. Discrepancies between submitted URLs and indexed URLs are a useful diagnostic signal.

Common Crawlability Mistakes

• Blocking CSS and JavaScript in robots.txt, which prevents Google from rendering pages accurately
• Leaving staging or parameter-based URLs crawlable and indexable
• Using noindex directives in HTTP headers inconsistently with on-page meta robots tags
• Forgetting to update the sitemap after a large-scale content migration
• Setting crawl delay in robots.txt for Googlebot — Google ignores it, and it can confuse other bots

Indexation: From Crawled to Ranking

Crawling and indexing are not the same thing. Google can crawl a page and still decide not to index it. This distinction trips up a lot of marketers, and honestly it's one of the more frustrating parts of the job — you can do everything right on the crawl side and still find pages stuck in "Crawled, currently not indexed" limbo in Search Console.

The Coverage report in Google Search Console is your first diagnostic tool. Pay close attention to these statuses: "Discovered, currently not indexed" means Google found the URL but hasn't prioritized crawling it yet — often a crawl budget issue. "Crawled, currently not indexed" means Google crawled it and chose not to index it — usually a quality signal.

Quality is the single biggest lever on indexation decisions since the Helpful Content system became part of Google's core ranking infrastructure. Thin pages, near-duplicate pages, auto-generated pages with no unique value — these don't just fail to rank, they actively drag down how Google perceives the rest of your site. Consolidating weak content is often the fastest path to recovering indexation rates.

Noindex tags should be used deliberately and audited regularly. Common legitimate uses: thank-you pages, internal search results, filtered product listing pages with no unique content, admin pages. Common mistake: a developer adds noindex during staging and it never gets removed post-launch. Always verify with a post-launch crawl.

The URL Inspection tool in Search Console is underused. For any page you can't find in the index, inspect it, run a live test, and look at the rendered HTML. That rendered output is what Google actually sees — and sometimes it's very different from what your browser shows you.

Site Speed and Core Web Vitals

Core Web Vitals became a ranking signal in 2021 and have only grown in importance since. As of August 2025, the three metrics Google uses are LCP (Largest Contentful Paint), INP (Interaction to Next Paint, which replaced FID in 2024), and CLS (Cumulative Layout Shift). Each measures a different dimension of perceived page experience.

LCP measures how fast your main content loads. The target is under 2.5 seconds. The biggest culprits are unoptimized hero images, render-blocking resources, and slow server response times. Preloading your LCP image element and serving images in next-gen formats like WebP or AVIF are reliable wins. Using a CDN to reduce time-to-first-byte is often the highest-leverage single fix.

INP replaced FID because FID only measured the delay before the first interaction — not the full response time. INP captures every interaction on the page and reports the worst one. Heavy JavaScript execution on the main thread is almost always the root cause. Long tasks over 50ms are the thing to hunt down using Chrome DevTools' Performance panel.

CLS is about visual stability — elements jumping around as the page loads. The most common cause is images and embeds without declared dimensions, or fonts loading and causing text reflow. Set explicit width and height attributes on all images. Use font-display: swap carefully — it can trade invisible text for layout shift if not implemented well.

Use Google's PageSpeed Insights for field data (real user measurements from the Chrome User Experience Report) and lab data in one place. Field data is what actually affects rankings. Lab data tells you what to fix. You need both. Cross-reference with the Core Web Vitals report in Search Console to understand how your entire site is performing, not just individual pages.

Don't forget Time to First Byte (TTFB). It's not a Core Web Vital but it's upstream of all of them. If your server is slow, everything else suffers. Target under 800ms. Shared hosting, underpowered databases, and no server-side caching are the usual suspects for sites that consistently miss this.

Mobile SEO and Responsive Architecture

Google has been mobile-first since 2019. That means the mobile version of your site is what gets crawled, indexed, and ranked — not the desktop version. If your mobile experience is weaker, your rankings reflect that. This is not a future concern. It's the current reality.

Responsive design is the recommended implementation. A single URL that serves the same HTML to all devices and adapts layout via CSS is cleaner to crawl, easier to maintain, and avoids the duplicate content headaches that came with separate m-dot sites. If you're still running a separate mobile site, migration to responsive should be a priority.

Common mobile-specific issues to audit: tap targets too small (buttons or links under 48x48px), viewport not configured, content wider than the screen, text too small to read without zooming, and interstitials that cover the main content on mobile entry. Google's Mobile Usability report in Search Console flags most of these directly.

If you're using a dynamic serving setup — same URL, different HTML based on user agent — make sure you're using the Vary: User-Agent HTTP header. This signals to Google that content differs by device. Inconsistent implementations here cause indexation headaches that can take months to untangle.

Test real device performance, not just emulation. Chrome DevTools' device emulation is useful but imperfect. Google's Search Console mobile usability data comes from real users. For deeper insight, tools like WebPageTest let you test on actual mobile hardware and networks, which reveals real-world performance gaps that emulation misses.

Structured Data and Schema Markup

Structured data is one of those areas where the gap between sites doing it well and sites ignoring it has widened significantly. In August 2025, it's not just about rich results in traditional search. Structured data is increasingly how AI search systems — including Google's AI Overviews, Perplexity, and ChatGPT's browsing mode — understand and extract facts from your pages.

The most impactful schema types depend on your site type. For e-commerce: Product, Offer, AggregateRating, and BreadcrumbList. For publishers: Article, NewsArticle, FAQPage, HowTo. For local businesses: LocalBusiness with full NAP markup, OpeningHoursSpecification, and Review. For SaaS: SoftwareApplication. Start with the types most relevant to your content, implement them correctly, and expand from there.

JSON-LD is Google's preferred format. It lives in a script tag in the head of your page, which makes it easy to implement and maintain without touching your visible HTML. Microdata and RDFa are still supported but significantly harder to manage at scale. If you're choosing an implementation method today, JSON-LD wins.

Validate everything with Google's Rich Results Test and Schema.org's official documentation. Common errors: missing required properties, incorrect property nesting, using schema types that don't match your actual content. The last one is worth emphasizing — don't mark up a FAQ section that's just marketing copy as FAQPage schema. Google's quality reviewers catch this, and it can result in schema being ignored or worse.

Beyond rich results, structured data improves how clearly your content communicates its meaning. For AI visibility specifically, marking up entities — people, organizations, products, locations — helps AI systems attribute content to you accurately when generating answers. This is a growing competitive advantage that most sites aren't thinking about yet.

Canonicals: Controlling Duplicate Content

Canonical tags are one of the most powerful and most misused tools in technical SEO. The rel=canonical tag tells Google which version of a page is the preferred one for indexing. Used correctly, it consolidates link equity and prevents duplicate content from splitting your ranking signal. Used incorrectly, it can quietly suppress pages you actually want to rank.

Every page should have a canonical tag — even pages with no duplicates. Self-referencing canonicals are best practice. They explicitly tell Google which URL is canonical without relying on Google to infer it, which it sometimes gets wrong. Generate them automatically through your CMS or framework.

Watch out for these specific failure modes. Parameterized URLs: when URL parameters for tracking, filtering, or sorting generate indexable pages with near-identical content, canonical tags (or parameter handling in Search Console, or both) are the fix. HTTP vs HTTPS: if your site canonicalizes to HTTP URLs, Google may index those instead of your HTTPS versions. Trailing slash vs no trailing slash: treat these as two different URLs in your canonical implementation and be consistent.

Canonical chains are a real problem. If Page A canonicalizes to Page B, and Page B canonicalizes to Page C, Google may honor the chain — or it may not. Flatten all canonicals to point directly to the final preferred URL. This is especially common after migrations when old canonical tags weren't updated.

Canonicals and hreflang interact. For international sites, make sure your canonical tags point to the correct language/region version of a page, not to a default URL across all regions. Getting this wrong is one of the most common causes of international SEO failures I've seen in cross-border site audits.

Redirects Done Right

Redirects seem simple until you're dealing with a site migration, a URL restructure, or a CMS switch — and suddenly you have redirect chains four hops long, mixed HTTP/HTTPS loops, and canonical tags pointing to pages that redirect somewhere else. Getting redirects right from the start saves weeks of untangling later.

301 redirects are permanent. Use them for any URL that's changed location permanently. They pass the vast majority of link equity to the destination URL — Google has confirmed that the small amount historically lost is negligible in most cases. 302 redirects are temporary. Use them when the original URL will come back, like during A/B tests or temporary campaign pages. Using a 302 where a 301 is appropriate means the original URL stays in the index when you don't want it to.

Redirect chains slow down crawling and can dilute link equity with each additional hop. The rule is simple: every redirect should point directly to the final destination URL. When you add a new redirect, check whether the target URL already redirects somewhere. Audit your full redirect map at least once a quarter if your site changes frequently.

Redirect loops — where URL A redirects to URL B which redirects back to URL A — will make a page completely inaccessible to users and crawlers. They surface quickly in browser testing but can hide in server configurations. Screaming Frog's redirect chain crawler catches these reliably.

After any migration, keep a redirect log of all old-to-new URL mappings. You'll need it when old links surface, when you audit your backlink profile, and when clients ask why a specific page disappeared from rankings. The sites that recover fastest from migrations are always the ones with clean, documented redirect maps.

Log File Analysis: What Googlebot Is Actually Doing

Log file analysis is the most underused diagnostic tool in the average SEO's workflow. Your server logs record every request — including every request from Googlebot. That means you can see exactly which pages are being crawled, how often, and what response codes Googlebot is getting. No crawl simulation tool gives you this. This is ground truth.

What you're looking for in a log analysis: pages crawled frequently that shouldn't be (thin content, parameter URLs, admin pages), important pages crawled rarely or never, pages returning 404s or 500s to Googlebot, and crawl frequency trends over time. A spike in 500 errors is often how you discover a server-side issue before it shows up in rankings data.

For small sites, you can parse logs manually with a spreadsheet. For anything bigger, use a purpose-built tool. Screaming Frog Log File Analyser is the go-to for most agencies. Botify and Lumar (formerly DeepCrawl) handle enterprise-scale log analysis where you're dealing with millions of rows. The tool matters less than building log analysis into your audit workflow at all.

Cross-reference your log data with your crawl data and your Search Console index coverage report. Pages that Googlebot is crawling frequently but not indexing are your clearest signal of a quality or duplicate content problem. Pages that Googlebot isn't crawling at all but that you want indexed are your clearest signal of an architecture or crawl budget problem.

One pattern I've seen consistently on sites recovering from core updates: Googlebot's crawl frequency increases in the weeks after significant content quality improvements are deployed. That correlation doesn't prove causation, but it's a useful signal that changes are being noticed. Log files are how you watch that happen in real time.

JavaScript SEO

JavaScript SEO is its own discipline at this point, and it's only grown more important as frameworks like React, Vue, Next.js, and Nuxt have become standard. The core challenge is this: Googlebot fetches HTML first, then queues JavaScript rendering as a second step. That second step can be delayed by seconds, minutes, or in edge cases, much longer. If your critical content only exists after JavaScript executes, Google may not see it reliably.

The three main rendering approaches and how they affect SEO. Client-Side Rendering (CSR): the browser builds the page from JavaScript after the initial HTML loads. Worst for SEO — content is invisible in the initial HTML response. Server-Side Rendering (SSR): the server generates full HTML before sending it. Best for SEO — content is immediately available. Static Site Generation (SSG): pages are pre-rendered at build time. Excellent for SEO when used for the right content types.

Hydration is an important middle concept. Frameworks like Next.js can SSR the initial page load (for SEO and perceived performance) and then hand off to client-side JavaScript for interactivity. This is generally the right architecture for SEO-critical sites. The critical question to ask your dev team: is the content a user sees on first load also present in the raw HTML response from the server?

Use the URL Inspection tool in Search Console's live test to see Google's rendered view of any page. Compare that to your raw HTML source (Ctrl+U or right-click > View Page Source). If your page source shows a sparse skeleton and the rendered view shows full content, you're dependent on JavaScript rendering — which is a risk you need to manage. If both show full content, you're in good shape.

Key things to verify in a JavaScript SEO audit: internal links in JavaScript-injected navigation are being followed, metadata (title, description, canonical, hreflang) is present in SSR output not just after client-side render, lazy-loaded content that matters for rankings is getting indexed, and infinite scroll or load-more patterns have a crawlable pagination fallback.

The single most common JavaScript SEO failure I've audited: a navigation menu built entirely in JavaScript with no server-rendered HTML fallback. It looks fine in a browser. But Googlebot following links from that page effectively hits a dead end. Pages linked only through JavaScript navigation often end up with dramatically fewer internal links in Google's index than they should have.

Quick Reference Cheat Sheet

Crawlability Checklist

• robots.txt validated and not blocking key resources
• XML sitemap submitted in Search Console, contains only canonical, indexable URLs
• No important pages more than 3 clicks from the homepage
• Crawl budget not wasted on thin, parameterized, or duplicate pages
• CSS and JS files accessible to Googlebot

Indexation Checklist

• Coverage report reviewed monthly for status changes
• Noindex tags used deliberately and audited post-launch
• Thin and duplicate content consolidated or removed
• URL Inspection tool used to diagnose specific indexation failures
• Quality bar maintained across all indexable pages

Core Web Vitals Targets

• LCP: under 2.5 seconds
• INP: under 200 milliseconds
• CLS: under 0.1
• TTFB: under 800 milliseconds
• Field data (CrUX) used for ranking, lab data used for diagnosis

Redirect Rules

• 301 for permanent moves, 302 for temporary
• No redirect chains — all redirects point to final destination
• No redirect loops
• Redirect map documented and maintained across migrations
• Canonical tags updated to match redirect destinations

JavaScript SEO Quick Checks

• Compare raw HTML source to Google's rendered view in URL Inspection
• Internal navigation links present in server-rendered HTML
• Metadata (title, canonical, hreflang) in SSR output
• Lazy-loaded ranking content gets crawled
• Pagination is crawlable without JavaScript execution

Where to Start

Technical SEO audits feel overwhelming because they cover so much ground. Here's how to cut through it. Run a full crawl with Screaming Frog or Sitebulb. Pull your Coverage and Core Web Vitals reports from Search Console. Request your server logs from your hosting provider or DevOps team. Those three data sources, reviewed together, will surface 80% of the issues worth fixing.

Prioritize by impact. An unintentional noindex on a key category page beats any amount of schema optimization. A critical page stuck in a redirect chain beats fixing image dimensions. Fix the things that are actively preventing indexation and crawling first. Speed and enhancement work comes after.

Use site audits to schedule recurring crawls so issues surface before they become ranking problems. Technical SEO isn't a one-time fix. Sites change, code gets deployed, platforms get updated. The difference between sites that stay healthy and sites that slide is whether someone is watching the foundation.

If you want to track how your technical fixes translate to ranking movement over time, rank tracking that segments by page type or site section makes it much easier to connect infrastructure work to actual outcomes. That story matters both for your own analysis and for keeping stakeholders bought in.

Technical SEO is not glamorous work. It doesn't get the conference keynotes that AI content and link strategy get. But I've watched sites that were stuck for months unlock meaningful ranking growth simply by cleaning up their crawl configuration, fixing their canonical structure, and getting their Core Web Vitals into the green. The fundamentals still win. Build them right, monitor them consistently, and everything else you do in SEO gets more effective.

If you want a quick read on how your site's technical health is trending alongside your AI search visibility, Aergos surfaces both in one place — worth a look if you're managing multiple sites or reporting to clients regularly.