Crawl Diagnosis

Occasionally, you will crawl a website and obtain audit results which you do not understand. These diagnosis options are designed to help shed some light on the situation. By switching these on, you will be able to see the exact same data Sitebulb is seeing when it performs a crawl.

You can turn on these diagnosis options via the Advanced Settings.

To get to Advanced Settings, you scroll to the bottom of the main Audit setup page and hit the grey Advanced Settings button.

The Diagnosis section is under Crawler -> Diagnosis

Crawler Diagnosis

In the screenshot above there are 2 options, however the 'Save Screenshots' option is only available if you are using the Chrome Crawler. If you are using the HTML Crawler, this option will not be visible on screen.

  • Save HTML - Sitebulb will literally save the HTML it encounters for each page, which is used to parse and collect data. With the saved HTML, you can look through this to see how it may differ from 'live' HTML you can see in the browser. 
  • Save Screenshots - Sitebulb will render each page, generate a screen capture and save the screenshot for later analysis. With the saved screenshots, you can look through these to see how they may differ from 'live' rendered pages you can see in the browser.