Sitebulb Release Notes

Version 6.4

Released on 16th March 2023 

Note: for anyone using a DIY server license, the server version has a different installer, which you need to get from the Server Release Notes page here.

Updates

• We have changed the way that the duplicate content report works, so that it now will not trigger for paginated URLs. To be honest this change is probably overdue, since Google are now able to easily recognise paginated pages and understands how to treat them. Thanks to Berian Reed for refusing to accept 'we'll look at it later' for an answer - good man yourself!

Fixes

• Due to embarrassingly poor testing from our beta testers, we hadn't noticed a bug in our GA4 implementation that meant Sitebulb would only ever load in 5 accounts! On an entirely unrelated note, if anyone wants to be a Sitebulb beta tester, please email us and ask.
• Fixed an issue where Sitebulb was not always correctly reporting external redirects that lead to broken URLs.
• When looking at imported audits, Sitebulb would hang on the spinny spinner of death whenever you tried to view URL Details. This is due to the way we obfuscate data on exported audits so that Google Account data is not carried over. To be completely honest with you, we actually just remove the account data, rather than obfuscate it, but it's not often you get to use a word as fun as 'obfuscate,' and I didn't want to miss the opportunity.
• Obfuscate.

Version 6.3.2

Released on 8th March 2023 (hotfix)

Fixed a small bug in the Windows versions which meant that certain websites would not render properly with the Chrome Crawler.

Version 6.3.1

Released on 1st March 2023 (hotfix)

Fixed a small bug in the new version that meant a handful of users could not open the application, and received a 'Sitebulb failed to load' message. If this happened to you after you installed v6.3, please update to v6.3.1 and accept our apologies!

Version 6.3

Released on 1st March 2023

Note: for anyone using a DIY server license, the server version has a different installer, which you need to get from the Server Release Notes page here.

In a slight change to proceedings, my normal rambling release notes rambled their way into a short story (~1200 words) about Google's approach to GA4 migration. If you aren't interested in reading and just want to skip to the boring stuff, here's a handy jumplink.

The Office on the 30th Floor

As the elevator pinged its arrival on the 30th floor, the junior executive shuffled through the crowd, almost getting trapped by the door as he exited. He straightened his tie, took a deep breath, then walked the short corridor to the executive office, and knocked quietly.

"Enter", boomed the voice from inside. As he poked his head round the door, he was struck by the combination of fear and awe that was so familiar from his previous trips this high up the building. Mr Jefferson, who could politely be described as an egg-shaped man, sat with a pinched look on his face, behind the sort of rich, deep mahogany desk that hums with entitlement.

To any normal observer, this was an office that stank of pretension. The walls were adorned with certificates no one cares about, and photos of its incumbent shaking hands with a variety of ostentatiously dressed middle-aged white men. There was a drinks cabinet in the corner that housed several bottles of single malt whisky. To David, however, it smelled like aspiration, and he soaked it all in as he walked timidly towards the desk.

"Johnson, you took your time," said Mr Jefferson with a dismissive glare. His name was David, or Dave to his friends, but only ever 'Johnson' to the senior executives. Although he dearly wished that he would one day be 'Dave' to Mr Jefferson, he preferred this situation to his previous years of anonymity. It was, at least, a sign of progress.

The senior executive leaned back in his chair so he could size up his quarry, tutting in annoyance as the leather let out an impertinent squeak underneath his considerable mass. His mouth curled up into a sneer as he appraised the pathetic excuse for a human being he saw in front of him, bathing in the effect of his powerful presence.

"I need good news on the GA4 adoption," said Mr Jefferson. It was a command, rather than a request, punching out across the room before David had even had a chance to sit down.

"Oh it's going really well sir,” David replied. “It was such a good idea of yours to set a firm twilight date on Universal." He smiled, gazing up adoringly, then added quietly, “genius, really.” An outside observer may have considered David's behaviour sycophantic, but in this case, his deference was genuine.

At this compliment, Mr Jefferson's chest puffed out even further than its normal hubris-inflated form. "Well, something needed to be done. A firm hand, as they say."

"No one is firmer than you, sir." His mind so addled with admiration, David was oblivious to the undertones of his remark.

Mr Jefferson almost allowed himself a self-satisfied smile, but checked himself, before fixing the underling with a belligerent stare, as if daring him to deliver bad news. "And what of our problem with the... SEOs?" The word came out strangled, his lips tightening around it as if it tasted of gone-off milk.

A bead of sweat formed on David's brow, as he looked up pensively at his superior. He felt the executive's stare bore down into his soul, and his chest felt like someone was sitting on it. As he spoke the pre-prepared response he'd practiced all morning, he felt detached from his body, as if someone else were saying the words. "I'm afraid they’re still resistant, sir."

His eyes dropped, expecting a volley of outraged abuse. When that didn't immediately come, he risked a glance at the older man, whose face was quickly reddening, his jaw clenched tight as if he teeth were trying to escape his mouth. Desperately, David tried to think of how to deliver the repulsive feedback he’d collected, blurting uncontrollably, "they hate it, they say it's... awful". Despite his blind panic, his well-drilled subordinate nature still ensured he substituted 'dogshit' at the last second. Now sweating profusely, his mind bounced around, trying to find anything of value for him to say, hopelessly settling on an incoherent string of verbal diarrhea.
"I can't!"
"They're just-"
"I don't even-"
"It's not!"
"Why do they?"
"I don't... I don't understand them!" He finally blurted out an intelligible sentence. "SEOs are all so weird."

"They're scum, is what they are!" Mr Jefferson roared back, spittle flicking out of his mouth, interrupting the clean lines of his exorbitantly over-priced suit. "I will not be dictated to by a bunch of cowboys. Not on my watch." His eyes bulged out of their sockets, like a constipated frog.

Unconsciously, David had shrunk back, low down into his seat, cowering at the onslaught. Mr Jefferson's colouring had advanced now to a deep burgundy, and his large, round face resembled a ripe plum.

Panicked at how quickly the meeting had escalated, and desperate to avoid having to make a decision, David saw his opportunity and dived for it. "What do you intend to do sir? I can't think of anything, but you must have loads of-"

"We'll just dump everyone in it," Mr Jefferson interrupted, "whether they like it or not. The normals won't even notice, as long as there's still pretty graphs. Let the SEOs complain. Fuck their historical data.”

“Oh wow, sir. That's a truly excellent idea. Brilliant. Erm... how?”

“Do I have to do all the work around here? Figure it out Johnson, it's your job. Some sort of automated migration. I don't give a shit if they lose data, they should have just got on with it in the first place. Give them a month, bunch of useless pricks. Let them burn." He smiled wickedly, his jawline jutting forward in self-satisfaction.

"Yes, yes of course. I will see to it immediately," David replied. His brain whirred into overdrive as he realised he had no idea how he would possibly achieve this, then this solitary thought was immediately tamped down by the immense sense of pride he felt at being asked. This emotional ambivalence almost proved too much, as his brain struggled to keep up, leaving his eyes dazed and his mouth fixed in the type of confused grin normally reserved for a bewildered puppy that doesn’t know why it’s been told off for shitting on the carpet.

David rolled his chair back, desperate to escape the room. But since he had inadvertently slid so far down it, as he pushed back, the plush leather slid out from underneath him, landing him flat on his back and sending the chair careening across the room.

Mr Jefferson was oblivious to the plight of his young chattel, his gaze fixed distantly as he ranted at the room. "And all those bloody SEO tools", he spat, "oh they'll follow suit soon enough, you mark my words. No matter how much they whine and whinge, they always follow along like the obedient little sheep that they are.” His face had returned to a colour more commonly associated with human beings. In fact, his face was beaming now, flushed with a heady mix of vitriol and profound arrogance.

“Dismissed, Johnson. On your way out go and send for that halfwit Smithies, I need to know what we're doing with this blasted AI nonsense.”

David rubbed the back of his head as he sheepishly sat up, trying to figure out how he could sneak out of the room without Mr Jefferson noticing.

“Johnson! Don’t ignore me, boy! What the bloody hell are you doing on the floor?”

Updates

Sitebulb now supports Google Analytics 4

If you hadn't already guessed what the nonsensical short story business was about, it is simply due fanfare for our long-avoided GA4 integration. When setting up GA, you can now select between Universal Analytics and Analytics 4:

As before, Sitebulb will try to match up the website and auto-select the correct property, but you can adjust from the dropdown either way.

The new API allows you to select some filters such as medium and device (thank you to my glamorous assistant Simon Cox for the site demonstration).

We also now present some sample data, so you can make sure you have got the right account BEFORE you run the audit.

We definitely added this feature in order to be helpful to users, and not just because we were so fed up getting support tickets with people moaning about not getting GA data back in their audit - that would always, literally ALWAYS, be because the user selected the wrong fucking account.

So that's it really - everything else is basically the same in terms of the GA integration. Sitebulb will continue to support Universal Analytics for as along as Google keep the data available to access via API, which apparently will be at least 6 months after the sunset event on July 1st 2023. We shall see.

Fixes

• Fixed an issue where data would overlap when horizontally scrolling a URL List - if you'd frozen a column and selected a row.
• Exporting sitemap data from URL Lists was not showing XML sitemap URL data, it was pulling through a random number into the export instead.
• Sitebulb was claiming that srcset images have short cache headers when actually they do not.
• The CTR data pulled through from Google Search Console was absolutely whack (apparently Sitebulb was including 0 click searches in its calculation, whereas GSC does not...)
• Sitebulb will now save/persist settings when generating XML Sitemaps.
• Performance hints were not correctly generating the 'change' values between one crawl and another.

Version 6.2

Released on 25th January 2023

Note: for anyone using a DIY server license, the server version has a different installer, which you need to get from the Server Release Notes page here.

Fixes

The Protocol Error

A cold, bleak January morning found Scott working through his regular deluge of support tickets and phishing emails, only to find an unusual pattern - a number of support tickets that had come in from users unable to crawl certain websites. Four, to be precise.

Just so as you understand it, it is actually not uncommon to come across sites that won't crawl. After all, the internet is awash with websites sitting behind over-aggressive CDNs, websites with peculiarly configured servers, and websites that appear to have been built with bits of old hairnet and Mars bar wrappers.

But four all at once, that IS uncommon. His curiosity piqued, Scott dug deeper, and found that the sites would work with our desktop user-agent, but not our mobile user-agent.

This was not normal.

The problem, apparently, was Chrome rejecting the requests on the basis of an 'HTTP2 protocol error.'

We needed a hero.

We needed a Gareth.

Wordlessly, he zipped up his ill-fitting parka, adjusted his glasses, and sunk his mind and body down deep into the debug zone. Tirelessly he worked, deep into the night, with only a flickering candle for company and a face kept warm by the unnatural hum of the computer screen but 2 inches from his face.

Finally he emerged, his bloodshot eyes vacant, and cried 'it is fixed!' before collapsing in a heap of sweat and prescription glasses.

Coaxed back to consciousness and somewhat restored, thanks to a heady blend of chicken soup and chamomile tea, Gareth revealed what he'd found in the shadows. "It was the version number. The Chrome version number for our mobile user-agent. It was out of date! I updated it, and then all four sites worked, just like that."

Despite his clear relief, he still chattered nervously, his eyes roaming wildly, refusing to rest. "Those four sites were all on nginx servers, but what if it's happening elsewhere?" All of a sudden he sat bolt upright, grabbing my arm fiercely. His eyes now had settled, his voice clear and calm. "We have to tell the world. There could be others. We've fixed our problem, but what about those idiotic users who refuse to update their browsers? What about other crawlers or bots...?"

His voice trailed off as the gravity of his words hit him. He slid back into his seat and closed his eyes, letting sleep finally take him.

Version 6.1

Released on 18th January 2023

Today marks the official release of Sitebulb Server - please check out the launch page for details!

Note: for anyone using a DIY server license, the server version has a different installer, which you need to get from the Server Release Notes page here.

Updates

• 'Generic performance improvements' (wait... what?!) 

Fixes

• Fixed an issue with sites that use a meta refresh, where Sitebulb was trying to jump the gun and process the refresh early.

Version 6.0

Released on 15th December 2022

BTW the server version has a different installer, which you need to get from the Server Release Notes page here.

Updates

#1 Automatic Google Sheets uploads

We experienced a number of complaints from users who were getting repetitive strain injury from clicking the mouse too much when manually exporting to Google Sheets at the end of the audit (at least, that's what they said was the cause...).

Because we like to delight our customers - and in order to shut up these whinging bastards - you can now optionally set Sitebulb to automatically export data into Google Sheets once an audit has finished running.

To automate uploading to Sheets, you must set this up at the project level (i.e. each project must be handled separately), which you can do when starting a new project or editing an existing project.

Once you are in the audit settings screen, click on the Sheets Exports option from the left hand menu.

Once you have enabled the option, you will then need to add a Google Sheets account to upload to. Once this is done, all you need to do is select from the checkbox list which options you wish to be automatically uploaded.

In a lot of cases, the first 4 options are most useful;

• Historical Audit Data - A single worksheet containing top-level metrics, one row for each audit within the project. This is designed for building out time-series data or charts, and can power Looker Studio or your own automated reports (check out our Looker Studio guide here).
• Historical Hint Data - A single worksheet containing aggregated hint metrics across the audit, one column for every hint, and one row for each audit within the project. This is designed to be ingested by your own data warehousing solution to build out highly customized reports.
• Audit Summary - A tabbed sheet of all Hints and descriptions, showing their status, the number of URLs affected, and links to the corresponding 'Learn More' page on the website.
• All Hints - A linked worksheet that contains all the triggered hints across the audit - along with the importance level and the number of URLs affected - hyperlinked across to the list of URLs for each hint in question.

Whichever options you choose, Sitebulb will automatically build the exports then upload them to your Google Sheets account, as soon as the audit has completed.

#2 Looker Studio support!

If you bothered reading the bit above, you'll have realised that I totally buried the lede, and you can now build Looker Studio reports from Sitebulb data!

I did it this way round because it is actually the 'Historical Audit Data' sheet that does all the heavy lifting here, so it makes sense if you actually know what that is first.

HANG ON A SECOND.

I have just realised you probably don't know WTF Looker Studio actually is. This is Google's new name for Google Data Studio. It's up for debate as to why they went ahead and changed the name. Maybe they wanted to move it away from the 'Google' brand because it has become so closely associated with data privacy violations. 

Looker. Huh.

Anyway, you can use this new sheet as a data source in Looker Studio, to build out some sexy reports like this:

Hubba hubba.

The whole thing is designed to work with scheduled audits (i.e. daily/weekly/monthly) as Looker Studio is particularly good at showing 'change over time.'

I know what you're thinking: 'this is all well and good but I will have to wait for 6 months to make use any use of it!'

Let me stop you there - we wouldn't do that to you! ​If you already have existing audit data that you wish to use - for example a monthly audit that you've been running for the last year - you can make Sitebulb generate the spreadsheet without waiting for the next audit to run.

To do this, navigate to your latest audit, then choose the Bulk Exports option from the top navigation, and export the Historical Audit Data directly from there.

Whether you use the auto-export to Sheets option, or the manual 'bulk export' method, the result is the same - you will end up with a new Google Sheet that contains all the historical audit data.

You can navigate there directly from your Sitebulb audit by using the Google Drive dropdown in the top navigation.

When you open the sheet up you should see all your audits in chronological order, with the most recent at the bottom. There are 271 columns of data, each relating to a different metric from the audit. This is every metric that Sitebulb can (currently) provide - you don't need to select the data to include, it will all just be there by default, with 0s in columns where no data is present.

Note: This sheet inherits the global styles you set for Google Sheets formatting - which is why mine looks so funky.

As long as you have the 'Sheets Exports' option set up in the audit (see above), whenever future audits run within this project, Sitebulb will append a new row to the bottom of the Google Sheet, with the latest audit data.

I'm going to say this next bit in caps so that you pay attention:

DON'T EDIT THE FUCKING SHEET!

• Don't delete columns you don't think you need
• Don't rename columns you think have shit names
• Don't add in your own data

Instead, in Looker Studio, just select the bits you want to keep.

The spreadsheet must maintain the same starting set of columns, otherwise appending columns will not work properly.

I have written a whole massive guide on how to create your Looker Studio reports with Sitebulb data, so I shan't repeat it again here.

And since I know most of you won't bother reading it anyway, I might as well just link to our Looker Studio Audit Template as well.

I will at least dump a few images from it here, to get you all juiced up:

Ooohs and ahhhs.

Very data. Much graphs.

Ok, that's enough of that. Go and read the Looker Studio guide.

#3 Copy Project settings

We also had a bunch of folks tired of all the clicking, this time when setting up projects. Add this, don't add that, click there, remember that...

We don't want y'all to do yourselves a misdeed, so we've added in some automation to this process too. Now, if you have a 'standard' or favourite project that you wish to copy settings from, you can do this on the New Project screen:

It will copy all the settings precisely - even if you're auditing a different website - so it is worth double checking the setup is correct before actually starting the audit, as some settings don't carry across well (e.g. sitemaps).

Archives

Access the archives of Sitebulb's Release Notes, to explore the development of this precocious young upstart:

• The Beta Notes (January-September 2017)
• Version 1 (September 2017 - March 2018)
• Version 2 (April 2018 - May 2019)
• Version 3 (October 2019 - July 2020)
• Version 4 (August 2020 - June 2021)
• Version 5 (July 2021 - December 2022)