The most important thing: Verify crawl access, indexability, rendering and the complete user journey on the URLs that matter.
Related play: Publish a business context page — a factual reference with service fit, comparisons and a practical 2027 v1 implementation.
1Audit the revenue-critical URLs
Check HTTP status, redirects, canonical destination, robots directives and page-level noindex. Confirm each important page appears in the sitemap and is linked from another useful page. Diagnose indexing in Search Console; a sitemap submission is not proof of indexing.
A URL-by-URL record shows expected status, canonical and indexing state.
2Check the real content response
Make sure the offer, price context and evidence are available in page text, with sensible headings and links. Compare raw response and rendered page for JavaScript-driven sites. Fix broken resources, login-only content and accidental staging restrictions.
Core information is accessible without an account or a broken rendering dependency.
3Separate discovery from training controls
Review the actual robots.txt plus CDN, firewall and bot rules. Allow the desired search bots and verify genuine requests against vendor guidance. OpenAI and Anthropic expose separate search and training controls. Google Search and Gemini-related controls also differ; inspect the Engine Map.
The owner records a deliberate policy and the developer confirms it works at the network edge.
4Test usability and recovery
Use mobile devices and keyboard navigation. Check layout shifts, loading delays, readable text, forms and booking. Resolve blocking errors before polishing minor scores. Keep monitoring for new 404s, expired certificates, broken forms and changes introduced by releases.
A real customer can read, navigate and complete the intended action.
A concrete example
Avoid this mistake: A successful request using a spoofed bot name does not prove a real crawler visited. Crawler visits do not prove indexing, citation or recommendation.