Make the right pages reachable
Separate crawling, indexing and access security.
Robots.txt asks cooperative crawlers which URLs they may fetch. A noindex directive controls indexing only if the crawler can read it. Neither protects private data; authentication does that. A 200 response can still be an empty JavaScript shell.
Your implementation route
Open SEO & GEO → Tools and settings → Robots.txt Editor. Save a copy of the current file before editing. Use page SEO settings for indexing choices. Publish and read the public robots.txt; do not replace Wix's generated rules blindly.
Platform boundaries & account requirements
Use Wix's native integrations and SEO & GEO tools. Backend examples belong in a supported Wix backend module or an external service, never in a page embed.
Open Wix ↗Five public target URLs, current robots.txt, page SEO settings and CDN/firewall access if present.
Follow the steps
Check only after completingInspect the actual responses
Open each target URL in a private browser. Check HTTPS, HTTP status, canonical hostname and redirects. Use the header recipe below. Then inspect rendered text and the original HTML; an important service description should be available as text.
Check all restriction layers
Read /robots.txt, meta robots, X-Robots-Tag and firewall rules. Look for an accidental Disallow: /, noindex on production, CAPTCHA or login walls. Check relevant user-agent groups rather than only the wildcard group.
Set a deliberate index policy
Allow search access to public service/product pages. Keep account and private material authenticated. Use noindex for accessible pages you intentionally exclude, rather than blocking the crawler from seeing the directive.
Test an important URL in Search Console
Use URL Inspection and the live test. Compare fetched content with what a customer sees. A successful live test is an eligibility check, not confirmation of indexing or a ranking promise. Record the result and request indexing when appropriate.
Implementation recipes
Expand · choose · copy</>A minimal public-site robots.txtOpen recipe
Reference for a new custom site only. Merge with existing rules; do not overwrite a managed host's generated file. This grants crawl access, not authentication.
# Example: a fully public website.
User-agent: *
Allow: /
Sitemap: https://example.com/sitemap.xml
# Keep private information behind authentication.
# Add your existing path exclusions to every applicable specific group.
# Do not block a URL solely to prevent its indexing.</>Page identity in the document headOpen recipe
HTML template / page SEO settings. This example is for an indexable public page. Replace every example value.
<title>Heat-pump repair in Austin | Example Business</title>
<meta name="description" content="Heat-pump diagnostics and repair for Austin homeowners. See service coverage, the assessment process and how to request an appointment.">
<link rel="canonical" href="https://example.com/services/heat-pump-repair">
<meta name="robots" content="index,follow">
<!-- Canonical is a hint; keep redirects, links and sitemap consistent. -->The target returns the intended status, usable content, canonical URL and index directives. Search Console can fetch it. No public caching exposes personalized data.
Restore the previous rules or page setting from the backup. Re-test the public URL after clearing only the relevant cache.
Read the primary instructions
Guidance checked October 7, 2026. “2027” describes this prototype’s visual concept, not a prediction of platform rules.