Audit sitemap scanning helps you discover and include pages from sitemaps and robots.txt files, providing greater page coverage and ensuring you don't miss important issues on your website. Sitemaps can expose pages not easily discoverable through link crawling, expanding your audit reach and ensuring ObservePoint finds the most important pages on your site.
How to Use Audit Sitemap Scanning
Enable Sitemap Discovery
Go to your audit settings.
Turn on Find Pages in Sitemaps and ObservePoint will automatically look for your robots.txt file and any sitemap files.
Or, you can manually add your robots.txt and sitemap URLs to your audit starting URLs:
How Does ObservePoint Sitemap Scanning Work?
Audits can be configured to automatically look for sitemaps listed in robots.txt files.
ObservePoint will then scan sitemaps for page URLs.
If multiple sitemaps are found each will be scanned.
Page URLs discovered from these sources will be added to the audit and treated like any other page, while obeying any inclusion and exclusion filtering you've configured.
URLs found in sitemaps will also be added to the Pages report for your account.
Where Do I See Results?
Sitemap-sourced pages appear alongside other pages in the audit results.
These pages are included in all standard reporting and validation processes.
Frequently Asked Questions
Does this feature validate sitemaps or robots.txt files?
No, this feature processes URLs listed in these files. Validation must be performed using third-party tools.Can it handle large sitemaps?
Yes, it supports sitemaps up to protocol-defined limits (e.g., 50MB or 50,000 URLs per sitemap).Do I need to update existing audits?
Yes, you must enable the setting to include sitemaps in your audit configurations.Is there any cost associated with scanning sitemaps?
No additional cost to scan sitemaps. As sitemaps and robots.txt files are scanned these will count against existing audit page volume.