Doing SEO for ecommerce? Our upcoming webinar is just for you! Register now.

Setting scheduled or recurring website audits

Sitebulb Pro and Cloud users are able to set up Projects to run scheduled or recurring audits, which can be done at the initial audit setup or retrospectively added at a later date.

This article will walk you through how scheduling works and how to do it. Don't miss the last section, which describes how you can pair this with automatic exporting to set up a fully automated workflow.

You can jump to a specific area of the guide using the jumplinks below:

Video Demonstration

For those that prefer video guides, we have a complete demonstration of the scheduling feature on the video below:

Scheduled vs Recurring

Before we start, a note on the lexicon. A scheduled audit simply relates to a single audit that is scheduled to run at a particular date, whereas a recurring audit is one that automatically runs at the same time each month/week/day.

Scheduled audits are particularly useful if you need to audit websites during specific time periods. For example, consider a busy ecommerce store, the development team may insist that the site is only ever crawled in the middle of the night when traffic is at its lowest. Scheduling allows you to avoid that 2am alarm clock!

Recurring audits are particularly useful if you need to run regular audits on the same websites. For example, if part of your client contract is to do a technical health check each month on the website. Setting a recurring monthly audit ensures that you have all the data you need, when you need it.

How to set up a scheduled website audit

When setting up a new audit, once you have finished customising the other elements, you should notice an option at the bottom entitled Scheduling.

Scheduling

If you wish to add a schedule, enable the Schedule Audit box;

Enable scheduling

This will open up the Scheduling panel and options. By default, the 'Once' radio button is selected, which essentially means you are setting up a scheduled audit to run only once, but not regularly recur.

All that is left to do is select a date and time, which can be specified to a 5 minute interval:

Setting a schedule

You may notice that the green button at the bottom right, which normally says Start Now, has switched out to Save Audit, since the audit is scheduled for the future and will not actually run immediately.

Save Scheduled Audit

How to set up a recurring website audit

A recurring audit is effectively a special case of a scheduled audit, so the initial steps as above are all the same.

To set up a recurring audit, this time you need to select the Recurring radio button, which will also open up a set of new options underneath.

Select Recurring Audit

Select how often you would like this audit to run

The next set of options are self-explanatory, but I will explain them nonetheless:

  • Monthly - the audit will run every month
  • Weekly - the audit will run every week
  • Daily - the audit will run every day

Select when to start the audit

This section is a little less intuitive. The default option is that the audit will start now and will run in its entirely, and then Sitebulb will schedule the audit to recur every month/week/day (based on the option you selected above).

Using the time selector, you can pick a time in the day for the recurring audits to run.

Since the selected option is to start the audit now, the green button bottom right stays as Start Now.

Start Scheduled Audit now

The alternative option here is to select a specific time to start the first audit, and at the same time set up a recurring schedule. This is effectively a combination of the two concepts - we set up a single scheduled audit (without running one right away) and then set this to be recurring with a specific periodicity.

Here is an example setup:

Select a specific date to start audit

So, in this example we switched to a monthly recurring audit, then set the first audit to run on the 1st of the month, so every following audit will also run on the 1st.

Now, if you check the screenshot below, you can see that the green button has changed to Save Audit.

Save scheduled audit

How to view scheduled audits

Once you have some scheduled audits set up, you can quickly and easily view them by navigating to the Scheduled tab from the main Projects screen. You will see your scheduled audits listed below, along with the 'Next Audit' run date, and the recurring interval, if the audit is set to recur.

View Scheduled Audits

Then, to navigate to a specific project, click the View Project button for the corresponding audit:

Navigate to a Scheduled Project

As an aside, you can also differentiate any Projects that have scheduling set up from the main Projects list, delineated by a little calendar symbol:

Scheduling Icon

Editing Project settings

Let's say you have a recurring audit set up to run every week, and your client calls up to let you know that they have uncerimoniously implemented AMP (without telling you first!!!). You may now wish to include AMP validation as part of your weekly audit in Sitebulb. By editing the project settings, you can go in and add AMP to the crawl settings, so that Sitebulb will also check this every week.

In order to do this, navigate to the project in question (see instructions above), and hit the blue button Edit Settings and Schedule:

Edit settings

This will open up the familiar 'Audit setup' page, which you can then adjust as required. In this case, all we wanted to do was add Structured Data validation, so it is just a case of checking the 'Structured Data' tickbox and then hitting the Save Changes button. However, you can go in and change any element of the audit setup, including Advanced Settings on the left. Just make sure to hit save!

Structured Data

Editing Project schedule

An extension of the above, you can choose to edit the scheduling of an audit as well. Simply navigate to the project, and open up the audit setup page using the blue button Edit Settings and Schedule.

Then, scroll down and press the Schedule Audit thumbnail button, to open up the scheduling panel. You can change any element of the schedule, including the recurrence interval and when the next audit should run. Make sure to hit the Save Changes button!

Starting an ad hoc audit

Let's say you have a recurring audit set up to run every month, but your client calls and tells you they have just made multiple changes to the robots.txt file on a whim (b@5$ards!), so you figure 'I'd better audit the site now!'.

Well, you can do this too, and setup Sitebulb to run an ad hoc audit, without interfering with your pre-scheduled recurring monthly audit.

As before, navigate to the project page, and hit the green button Start and Configure a New Audit, which will allow you to start a new audit right away, and will not change the recurring or scheduled date you have already put in place.

Starting an ad-hoc audit in a scheduled project

How to cancel scheduled audits

If you realise that you no longer want a specific scheduled or recurring audit to run, you can simply cancel the schedule off the project. All this will do is remove the scheduled element - all scheduled or recurring audits will be removed, and it will revert back to a normal project. Note that this will not delete the project, just the schedule.

In order to do this, navigate to the project in question and hit the Cancel Schedule text link.

Cancel schedule

How to access and adjust scheduling settings

The default behaviour of scheduled audits works on the basis of two core assumptions:

  1. That your computer is actually on, when scheduled audits need to run
  2. That Sitebulb is actually running on your computer, when scheduled audits need to run

If both of these assumptions are met, Sitebulb Pro will handle scheduled audits just fine. If either of them is not, then it is not possible for Sitebulb Pro to carry out the scheduled audit because it is desktop software reliant on your machine. (This is one of the major benefits of Sitebulb Cloud; as a cloud crawler, it can run crawls whenever, regardless of whether your machine is on or not.) 

Sitebulb Pro does however have a number of in-built settings that help to mitigate these issues, which are accessible via this link on the Scheduled Audits page:

Accessing Scheduling Settings

This will bring you to a settings page with specific scheduling settings:

Scheduling settings

These tickboxes offer the following options:

  • Disable scheduling - stop Sitebulb running any scheduled audits at any time, while this box is ticked.
  • Load on Startup - enable Sitebulb to load up when you start up your machine, and run silently in the background.
  • Run missed audits - if Sitebulb has not been running for a period of time, and has missed some scheduled audits, force Sitebulb to queue up and run these audits the next time it is running.

These options come with a degree of nuance, so you may wish to read below for a more complete explanation of why they exist and how you might wish to use them.

Disable scheduling

If you tick this option, it will over-ride all scheduled Audits so that none of them run until this box is un-ticked. Let's say you are going on holiday for 2 weeks and know you won't be around to look at any scheduled audits that come in during the time you are away, it might be suitable to stop scheduled audits from running until you are back.

Once you un-tick this box, any recurring Audits will be rescheduled for the next available date.

Load on Startup

If you tick this option, Sitebulb will load up whenever you switch on your machine, and run silently in the background, without a visible user interface. It will sit in your system tray, waiting for when it is needed.

Sitebulb running in background

Then, when scheduled Audits are due to start, Sitebulb will run these in the background. You'll get a notification message when it does:

Sitebulb has started notification

Note: On Mac, Sitebulb will need to be given permission to do this, for which you will need to grant by clicking OK;

Mac Permission

But once you do, you'll also see the notification in the top right of your screen, whenever Sitebulb starts running a scheduled audit in the background:

Audit started on Mac message

Run missed audits

If you tick this option, Sitebulb will queue and run any scheduled Audits that have been missed because Sitebulb was not running.

By default, if you missed an Audit for today’s date, Sitebulb will queue and run the Audit the next time you open the software. However, if you miss an Audit for any other day, Sitebulb will not run them by default. If you tick the box, it will run them regardless of the date.

As an example, let's say you had some audits set up to run like this:

  • Daily audits for client A
  • Weekly audits for client B
  • Monthly audits for client C

If you went on holiday for a month and your machine was off the entire time, when you came back into the office and opened up Sitebulb it would do the following:

  • Schedule and run 1 audit for client A, then set the schedule back to tomorrow
  • Schedule and run 1 audit for client B, then set the schedule back to a week from today
  • Schedule and run 1 audit for client C, then set the schedule back to a month from today

Automatic notification when audit is complete

Since scheduled audits are often run in the background or on a remote machine, it is easy to forget that you sent Sitebulb off to do this task, and so adding an email alert when the crawl is finished can be a helpful reminder.

You can set this up in Sitebulb via the 'Crawler Settings' options on the left-hand menu in the audit setup:

Send email when finished

To set this up, toggle the checkbox on, enter your email address in this box (or multiple addresses, comma separated) and Sitebulb will send you a friendly email whenever your audit is finished:

Send Email When Finished

Automating exports to Google Sheets

This is the cherry on the cake of the scheduling feature. It allows you to build a fully automated workflow where Sitebulb can run audits in the background and automatically upload the export files to Google Sheets once and audit has finished running. This saves the manual legwork of doing it once an audit is complete, and enables you to build out some nifty automated systems - for example, pushing data to Google's Looker Studio (check out our Looker Studio guide here).

To automate uploading to Sheets, you must set this up at the project level (i.e. each project must be handled separately), which you can do when starting a new project or editing an existing project.

Once you are in the audit settings screen, click on the Sheets Exports option from the left hand menu.

Sheets exports

Once you have enabled the option, you will then need to add a Google Sheets account to upload to.

Sheet Exports - Add Google Account

If you already have Google Sheets accounts authenticated through Sitebulb, you can select one of these, otherwise you'll need to authorise permissions for a new account.

User Google Sheets account

Once this is done, all you need to do is select from the checkbox list which options you wish to be automatically uploaded.

Different export options

In a lot of cases, the first 4 options are most useful;

  • Historical Audit Data - A single worksheet containing top-level metrics, one row for each audit within the project. This is designed for building out time-series data or charts, and can power Looker Studio or your own automated reports (check out our Looker Studio guide here).
  • Historical Hint Data - A single worksheet containing aggregated hint metrics across the audit, one column for every hint, and one row for each audit within the project. This is designed to be ingested by your own data warehousing solution to build out highly customized reports.
  • Audit Summary - A tabbed sheet of all Hints and descriptions, showing their status, the number of URLs affected, and links to the corresponding 'Learn More' page on the website.
  • All Hints - A linked worksheet that contains all the triggered hints across the audit - along with the importance level and the number of URLs affected - hyperlinked across to the list of URLs for each hint in question.

Whichever options you choose, Sitebulb will automatically build the exports then upload them to your Google Sheets account, as soon as the audit has completed.

Once the audit is complete, you can navigate there directly from your Sitebulb audit by using the Google Drive dropdown in the top navigation.

Link to Historical Audit Data

Fully automated audits

While the 'auto-upload to sheets' function works for any audit, it is most powerful when you use it alongside recurring audits.

The features combine to allow you to set up fully automated workflows, such as:

  1. Set up a project to run weekly recurring audits on a website
  2. Set up export files to push to Looker Studio
  3. Build Looker Studio reports/dashboards that will automatically update when Sitebulb audits complete
  4. Receive automatic email notifications when audits are completed

Every week, new audit data will be pushed up to Sheets, which in turn will be pulled in by Looker Studio, so your entire team (and clients!) can benefit from the fresh crawl data.

0