Free Webinar: International SEO & Localization with Gianluca Fiorelli, Raquel Gonzalez & Zeph Snapp. Sign up now!

Using custom headers

Some websites can only be crawled if custom headers are used, as the server rejects Sitebulb's HTTP request without them. If this is the case, you would need to add the custom headers via the Crawler Settings.

Access the Custom Headers section via the 'Crawler Settings' options on the left-hand menu in the audit setup, then scroll down to find the custom headers setup area.

Add Custom Headers to Sitebulb