The Beta Notes

The Beta Notes

This collection details all beta updates during 2017, as Sitebulb went on a journey of self-discovery. The notes themselves are stark and imageless, a reflection of Sitebulb's early struggles with identity.

The beta years were immediately proceeded by Version 1, then in 2018, Version 2, and in 2019, Version 3.

Version 1.0.19 (beta)

Released on 4th September 2017.

Updates

• Added a new global settings area (Dashboard -> Settings), which currently looks a little sparse, having only one thing in it, but in the future we will use this new real estate to house some important global settings some of you have been asking for (e.g. a global proxy setting). Not to get ahead of ourselves, the current global setting we have just added is 'Default User Agent'. This allows you to over-ride the default User Agent that Sitebulb uses (handily named 'sitebulb') and instead choose whichever one you want! This will then become the User Agent used for the little pre-audit check at the start, and the default for all audits, unless you change it in Advanced Settings. Muy emocionante!

Fixes

• Very occasionally Audits would get interrupted, because your computer would restart, or the internet would drop out, like when it's 1999 and you're playing jittery-as-fuck Grand Theft Auto online against your mate on your 56k dial-up modem, and the line would drop because your mum picked up the phone so she could ring her sister, who she just spoke to yesterday anyway. These sort of Interrupted Audits should always allow you to resume, but sometimes they wouldn't, as Sitebulb would mis-classify them as Errored Audits, and the 'Resume' button would be missing. This shouldn't happen anymore. GOURANGA!
• Fixed a very annoying issue where pause/resume would cause Sitebulb to skip the rest of the crawl and move onto building reports once you pressed resume. I say annoying because this would only happen if the number of queued URLs happened to be larger than the crawl limit, and I'd tried many many many MANY MANY MANY many tests before I realised it was this specific scenario that was causing it. In fact it wasn't even me who realised it, it was one of our magnificent beta testers. I might as well give up and go home now.
• Related to the above, we also fixed an issue where the crawl limit would get accidentally triggered early and stop the crawl (it wasn't strictly pause/resume related, yet did result in a similarly embarrassing premature climax).
• In an Audit, if you were on the Overview and you searched for a specific URL and then clicked through to view the URL Details page, you could then not navigate properly within the modal window. Specifically, if you went to look at Incoming Links, for example, you then couldn't go back to the 'Overview' of the URL Details. According to Gareth this is because the URL Details view opens in a modal, and so Sitebulb would have two references to 'Overview', incorrectly believing it was already on the modal Overview. I suggested it would be straightforward to just rename one of the Overviews, to 'Jimmy' perhaps, or 'Roger Federer'. But he decided to fix the underlying problem instead. Spoilsport.

Version 1.0.18 (beta)

Released on 30th August 2017.

Updates

• To make crawl limits more transparent/understandable, and to prevent unnecessarily saving tons of unwanted data on bigger sites, Sitebulb will no longer save disallowed URLs by default. If you want this data, you can switch on a new setting in the Crawl & Audit Setup called 'Save Disallowed URLs.' Good name huh? I came up with that. Ok I didn't really, but I could have if I wanted to.
• Improved the speed that Link Equity Score is calculated. Yes, you could argue that it was simply TOO SLOW before and we have simply fixed the thing that was making it slow, but I'm a glass half full kinda guy.

Fixes

• When aforementioned Link Equity Score was taking fucking ages on larger sites, users were trying to go back to the Dashboard and look at other Audits, as is their wont. But they were getting the Spinny Spinner of Death, followed by the Grey Screen of Death (just an empty Dashboard). When this happened to me, I asked Gareth WTF was going on here, and he said it was because locks. So now you know.
• Follow-up: when I asked him to explain it in non-geek, he said that basically, when Sitebulb was calculating the link graph (for the Link Equity Score), it was reading and writing to the database, and locking the database from being accessed by other areas of the tool, such as the bit that houses the reports. He re-jigged the locks so that it only locks a certain bit of the database, and doesn't lock all of it. Because locks, QED.

Version 1.0.17 (beta)

Released silently (like a ninja) on 29th August 2017.

Updates

• Totally groovy new Hint for the Links section: 'Only has links from URLs that do not pass Link Equity'. This is for obscure situations where you have URLs which ONLY have links from canonicalized URLs, for example. I know it doesn't sound like much now, but it'll be there for you when you need it most. Like that old bottle of Baileys in the back of the cupboard, leftover from a work Christmas party several years ago, which has almost definitely gone off and you can barely open because the lid is crusted over.
• On website, updated 'Changelog' to 'Release Notes'. Because meta.

Fixes

• There was an issue where Sitebulb didn't want to crawl sites protected by Incapsula. It wouldn't even perform a HTTP header check on external links to sites protected by Incapsula, FFS. This was completely unacceptable, so we didn't accept it.
• The HTML parser for the JavaScript crawler was occasionally shitting itself, and so was unable to find simple elements like the meta description, which was pretty lame.
• In the Links report, the anchor text would crash into the side of the page if it was particularly long (e.g. if you had a link with anchor text: "I'm here with my dad. And we never met. And he wants me to sing him a song. And, um, I was adopted. But you didn't know I was born. So I'm here now. I found you, Daddy. And guess what? I love you. I love you. I love you." - that would cause the anchor text to crash). Now we just use ellipsis...
• Changed the way included/excluded URLs are handled on the crawl progress screen. Depending on the website, and the inclusion/exclusion criteria, the numbers recorded on the crawl progress would jump around like a mother fucker. This was to do with the way they were being counted and recorded by the software. To make life easier for everyone, we no longer count them at all.
• If you had a double slash in the URL (e.g. movies.com/is-die-hard-really-a-christmas-movie//discuss) this would trigger BOTH the Hint 'URL contains a double slash' AND 'URL contains repetitive elements', since the double slash was being classed as a repetitive element. Whilst not technically wrong, this wasn't exactly useful, so we've changed them now to be properly unique. The answer, by the way, is clearly YES. I don't even know why there's a discussion about it.
• I know most of you haven't noticed, but there is a 'Feedback' tab in the left hand navigation. On there is a feedback box (I'm explaining because you probably haven't seen it) which you're supposed to write feedback into. If, on the Mac, you tried it drag and drop an image into this box (the one that is meant for feedback), the image would take over the screen and lock Sitebulb completely. It no longer does this, it just rejects the image outright. Not that you'll notice anything different anyways.

Version 1.0.16 (beta)

Released on 25th August 2017.

Updates

• You know how, once or twice in your lifetime, you have to admit that you were wrong about something. Well, this first update is one of those moments for me. We've introduced a new metric, 'Link Equity Score', which is a score from 0 to 10 for every internal URL, and is based on an internal linking algorithm. Gareth put this in the tool about a year ago, and then at some point I decided it needed to go (I can't even remember why). But it's back, by overwhelming public demand, and I was forced to utter the unholy words 'I was wrong'. I hope you appreciate my sacrifice.
• If you know anything about SEO, you'll know that Alt text on images is Kind of a Big Deal. Sitebulb has always had the Hint, 'Images missing Alt text', but up until now, has refused to tell you what these images actually are (you tease, Sitebulb). Now, however, we've moved the Hint to the 'Page Resources' section, and given it the respect it deserves. From the URL List associated with the Hint, you can click through to see the images associated with each page (check it out). Even better, in the Page Resources export you can see the same data, but this time grouped by image, with counts and example images (check it), so you can easily pick out template images that are on lots of pages.
• On the Overview, Sitebulb will sometimes display messages like 'Audit stopped early because crawl limit was hit.' When you only get one of these warnings it looks ok, but sometimes multiple warnings will trigger, so you'll get 2 or 3 of these boxes, and it looks jangly as fuck ('af', if you're down with the kids). So we've added a nifty X button in the top right hand corner of each box, so you can dismiss any that displease your eyes.
• Moved the crawl setting 'Maximum URLs to Audit' onto the first screen of the settings, underneath the Crawler Type selection. This is to give you, dear user, more transparency and control when it comes to bigger Audits. We have also fixed an issue where the maximum URL limit was incorrectly including Disallowed URLs in its calculation (this is really a Fix and should by rights go in the list below, but I do what I fucking well like and don't let anyone tell you otherwise).
• Sitebulb will now parse XML sitemaps that are compressed, so feel free to compress sitemaps to your heart's content. Middle-out for the win, Bro.
• Sitebulb will now pass HTTP headers when content fails to load, so it can at least tell you something about the URLs.
• Added fancy.com to the list of global excluded domains, because if you Audit a site that links to a unique fancy.com share page on every single URL, it's fucking annoying, trust me.

Fixes

• In the previous update we added a clever little change for the mobile rendering checks, which essentially used just a sample of pages rather than the whole lot. In doing this, we managed to fuck everything up. The data was being collected but never saved, and instead duplicate URLs were just inserted into the database! Cue many reports of odd looking issues, such as duplicate content Hints, missing GA codes and the like. If you've seen anything that sounds similar... then this is probably why.
• On re-audits, Sitebulb was going rogue, resetting the settings for crawl speed and maximum URLs, even if you'd changed them previously. We reined Sitebulb in, so it's no longer doing this. Valar Dohaeris.
• A few users reported an issue where the tool was occasionally crashing when switching between the Crawl Map and the Overview. We believe we have fixed this now, but we are not 100% sure. If it happens to you, it would be awfully good of you to let us know about it.
• AMP URLs were being counted as duplicates, because canonicals on AMP pages were not being classified correctly. I'm so sorry.
• Kinda specific, but we fixed an issue where Sitebulb would refuse to crawl sites where the server returned the Status Code as '200' AND the Status Description as '200' - Sitebulb was looking to find 'OK', and would throw a shit-fit if it didn't.

Version 1.0.15 (beta)

Released on 7th August 2017.

Updates

• Quite a few people asked us to make the Crawl Maps more integrated with the rest of the product. So now you'll find the Crawl Maps don't open in a new window, but within the main Sitebulb window. Not only that, but you can now click any node and be taken directly to the URL Details page for that URL. Wait, there's more... you can also now scroll out so far that your Crawl Map becomes a dot (I don't know why people want to do this, but it was a feature request). AND we've also moved the 'Save Crawl Map' button so that it doesn't print onto the captured image, which I think we can all agree looked pretty shit. The eagle eyed among you will also notice that we've tweaked the spacing a bit for how Crawl Maps display, which seems to make most Crawl Maps a bit more legible, but if you think we've made them worse please tell us!
• We've added the option to update crawl and audit settings while an Audit is paused. When you click 'Pause' you should see an 'Update Settings' button alongside 'Resume' on the top right. Bear in mind that whatever settings you change will only come into affect for new URLs. So if you click 'Site Speed' halfway through, then the speed analysis will only come into play for half the URLs. Similarly, if you untick 'Crawl Parameters', then no new parameterized URLs will be added, but there will still be some in the queue that will get crawled. Please make sure you read and remember this forever so I don't have to keep answering the same fucking questions about why it's still crawling parameterized URLs.
• If you've tried crawling a Shopify site with over 500 internal URLs, you may have noticed that those rat bastards eventually stonewall you with 430 HTTP errors (TOO MANY REQUESTS). It only seems to kick in around the 500 URL mark, so if you have a 1000 page site, just delete half your content and you're sorted... Alternatively, you can crawl at 1 URL/second, which is slow enough to stop Shopify blocking you. In the pre-audit, Sitebulb will detect if it's a Shopify site, and give you a cheeky little message to tell you to slow down, lest you blow your load early.
• Ok so here's the thing. JavaScript crawling is necessarily slower than non-JavaScript crawling. In general, users weren't deliberately choosing the JavaScript crawler, but they were selecting 'Mobile Friendly', which automatically and silently turned on the JavaScript crawler! Then we'd get complaints like 'you guys are legends and the product is awesome, but it's a bit slow.' We also regularly had: 'the crawler seems a little slow, but you are the best' (or words to that effect). Anyway we had a change of plan - now, unless you explicitly turn on the JavaScript crawler, Sitebulb won't use it. If you choose Mobile Friendly, it will crawl the site normally using the non-JS crawler, then select a sample (up to 100 URLs) of those crawled and perform the mobile rendering tests on them. Let us know what you think.
• The big one - searching for Projects/Audits in the search bar top right will now also search imported Audits. Whoa.
• While looking at reports in the Audit, the report you are 'in' will now be a darker shade of grey, to help you understand 'where you are.'
• On the Crawl setup screen, Sitebulb will now tell you which Google Analytics code was found in the HTML during the pre-audit, to help you pick the correct View. Note that this will not show if you have Google Tag Manager employed on the website, as Sitebulb does not fire the JavaScript to get the code (otherwise your Analytics data would be fucked).
• Sitebulb also now matches on the URL (www and non-www version) when trying to match the Google Analytics code to pre-select the right View, although the UA- code will take precedence, if found (see above).
• Sitebulb is now checking a few more places to find sitemaps to include in the pre-audit. I could tell you where we check, but I'd have to kill you.

Fixes

• If you delete a Project and then want to start a new Project with the same name, it wouldn't let you do it ('you already have a Project with that name', which WASN'T TRUE!). We coerced Sitebulb to relent in this tyrannical nomenclature. 
• The Hint 'URL received search traffic but 0 goal conversions' was showing a negative number, instead of 0. We conducted some research on this topic, and actually found that it is impossible to have negative URLs in real life, so we've changed this to show 0 instead.
• Some URLs were being incorrectly flagged for the Hint 'Set mobile viewport.' This was a coding error by Gareth, as it happens. Rest assured that he has been soundly thrashed.
• There was a tiny little typo on one of the compare audits fields - an extra, unwelcome lower case 'k' - which has now been forcibly rejected.
• The exclude URLs function, and the robots.txt parser, were not working correctly when handling wildcards (*). They are now.
• Astonishingly, no one noticed this bug, despite it being there from day dot: when doing pause and resume, you'd end up with some duplicate URLs. Whilst we've fixed the duplication issue, we can only conclude that no one's even looking at the fucking data, you're all just distracted by the pretty graphs. FML.
• For Site Speed Hints, if the affected URL figure was more than 1000, it would change to 1k, which then did not look like a number and so you wouldn't get the buttons on the right to click through and see the URLs. Now we do the lookup differently to avoid this issue.
• Not all images are created equal. Some images are embedded page resources, while some are anchored images that have their own image URL. Even if you don't select 'Page Resources', Sitebulb will still crawl anchored images and report on them. However, the reports weren't all hooked up right, so you'd click to view resources, and it would tell you that you hadn't turned them on! Touche, Sitebulb, but not very helpful.
• These self same anchored images, if hosted on a subdomain, would still get crawled even if you had unchecked 'Check subdomains', which was bafflingly annoying. This don't happen no more.
• Hint filters were wrong for the Hints 'Canonicalized URL received organic search traffic' and 'Noindex URL received organic search traffic', so the data looked like bollocks. Fixed the filters and now the data looks sweet as a nut (not that kind of nut!).
• Subdomains URLs weren't being identified when there was a protocol mismatch, which caused significant consternation for almost no one. Rest easy people, we've fixed it.
• If the start URL redirects, it is now updated to the final destination URL found by the pre-audit. I know what you're thinking, 'didn't it do that before?'. Apparently not.
• Fixed an issue where audits that got Interrupted (e.g. computer says no) were not always starting up again - they would say they were running, when they actually weren't (which was frightfully annoying).
• Sitebulb would forget what render timeout you set when re-auditing a site with new settings. It would remember everything else, but just forget this. We added 'Arya Stark mode', and now, The North Remembers.
• Google Analytics code was getting reported as not there, when it was there really.
• We experimented a bit with the selection options for 'results to display' on URL Lists. We tried 1000 and 500, but both of these would make Sitebulb hang occasionally on bigger audits. So we've abandoned this and gone back to the stable max option of 250.
• Fixed and issue where Audits were skipping the current process when resuming. We actually already fixed this bastard problem once before, but then something else started causing the same issue.
• If the tool crashes, as is its wont occasionally, the scheduled URLs (max 300) no longer get dumped/lost, so Sitebulb can pick up and recover more smoothly.
• Fixed issue where some exports (e.g. URL Details) were not generating properly. They were getting to 2% and then stopping indefinitely, which is no use to man nor beast.
• Fixed issue where some exports (e.g. URL Details for external URLs) were not saving when you chose 'Save and Open'. They'd just sit there, like a fucking lemon.
• Can now crawl Medium sites without adding the twatting 'GI' query parameter to every URL.
• Fixed a crazy issue where some URLs were being referred to like this //example.com/resource.js. In JavaScript crawling mode, if Sitebulb found these URLs it would panic and refuse to count any links.
• When filtering URL Lists for 'Content Type', it would only show a few options. Now we show all the options. There's too many, if you ask me.
• When crawling in JavaScript mode, the crawler was not always taking account of the base URL when crawling relative links. Another great reason why you should never use relative links for anything ever. Although we fixed it, so it's actually quite a shit reason.
• When crawling in JavaScript mode, if for some reason the page would throw a JavaScript error, Sitebulb would just fall over and die. Now it doesn't fall over and at least tries to return the HTML - if there is any.

Version 1.0.14 (beta)

Released on 21st July 2017.

Solitary Fix

• When using the software for the first time, users were not prompted to create an account. This meant they could use the software up until the point of creating a Project, at which point a license check would fire, and the software would enter a never ending state of blind panic. It was the shittest back door ever invented, and now it's been removed.

Version 1.0.13 (beta)

Released on 20th July 2017.

Updates

• The one you've all been waiting for... Added the 'Content Type' column to a bunch of Hints: URL contains whitespace, URL contains upper case characters, URL contains repetitive elements, URL contains non-ASCII characters and URL contains a double slash. I know right? Holy fuckballs.
• Sitebulb no longer checks for iframes when doing mobile rendering tests. While it is still true that iframes often don't offer a great mobile experience, in reality we were finding that most of what was reported were scripts like Intercom and Hotjar, which were inserting iframes when Sitebulb fired the JavaScript, so the hint was as much use as a blind guide dog. It now will only fire if frames are found.

Fixes

• Fixed the timeout issues which was causing Sitebulb to shit itself. If the tool has hung or crashed for you since you installed 1.0.12, this is probably why.

Version 1.0.12 (beta)

Released on 19th July 2017.

Updates

• As promised, Mac users can now happily dock the app again, without fear of Armageddon.
• Train mode - you can now view completed Audits while on a train (note, you can only start new Audits if you have access to the internet on said train).
• Plane mode - you can now view completed Audits while on a plane (note, you can only start new Audits if you have access to the internet on said plane).
• Offline mode - you can now view completed Audits while not connected to the internet (i.e. you no longer need to be on a train or plane to take advantage of this feature).
• Sped up the pre-audit process for some websites. Websites, in particular, with the defining characteristic of being utterly shite (e.g. bloated or broken HTML).
• Optimized HTML parsing. Translating from nerd language, this just means that when Sitebulb comes across crappy HTML, it can pick out the bits it needs and ignore all the junk. Sitebulb was previously handling this stuff badly, so CPU would sometimes spike when it came across a few of these shitty pages.
• Added a few default reply parameters to the 'Excluded URLs' tab in Advanced Settings. This stops the tool crawling 'replytocom' URLs, or other such worthless chuff.

Fixes

• We dropped a bollock on this one - a bug that meant if you paused the Audit while it was crawling XML Sitemap URLs, and then hit 'Resume', it would skip the crawling and move on to building reports. It would also do the same for GA and GSC... which might explain some issues where users were getting big lists of 'uncrawled URLs' with no reason whatsoever.
• Fixed all the annoying issues with closing, opening, double opening and crashing the tool. In short, Sitebulb has an application element, which does all the work, and a separate user interface which handles the presentation. We like to be original, so we refer to these as 'the App' and 'the UI'. What we have done is basically made the App and the UI talk to each other better, so each one knows when the other is open, and when it is not.
• Fixed a pretty big issue on the Mac, where if you right clicked the app icon and clicked 'Quit', the application would no longer open again from the Applications folder. This is related to our change above to 'make the App and UI talk to each other better'.
• Fixed a few UI bugs that were throwing errors in our bug tracker, which meant we couldn't see the wood for the trees. You won't notice anything different, but rest assured we've diligently fixed them. 

Version 1.0.11 (beta)

Released on 14th July 2017.

Attention Mac Users: Don't run from the dock
We have had to temporarily disable the option to run the program from the Dock. So if you have it docked already and install the new version, it WILL NOT RUN FROM THE DOCK.

Even if you dock the new version, it won't run from there. It will just show you the spinny loader thing. We're fixing this, but it's quite complicated, so bear with us.

Updates

• Lots of users were asking for a button to export and save the crawl map (apparently some people haven't heard of Jing). Anyway, button = added (which saves the crawl map as a PNG).
• This one is very cool. You can now highlight any text in the tool (from almost anywhere) and copy it. Anything: Hint copy, URLs from URL Lists, Titles, hreflang, etc... Highlight the text, right click, then press 'Copy'.
• If you click on a Hint, then export the data from the URL List, it now passes the Hint name into the name of the exported Excel spreadsheet (e.g. sitebulb_com_has_upper_case_characters_20170714102236.xlsx). Word up to that.
• Added a natty check to see if the Project name you've entered has already been used. If so, the user is warned to buck up their ideas.
• One confused beta tester thought that Sitebulb was crawling external links and shooting off to crawl the whole entire internet. Fret not, friends, it only does a HTTP status check on external links. It does this by default, but you can switch off external links in the Advanced Settings. We've updated the copy for this to make it clearer for said confused tester.
• Within Projects, if you hover over each Audit in the Website Audit History you will now see a little box showing details of the crawl (success, error, etc...). This will help you tell one Audit apart from another, it really is quite a handy feature. I wish I'd thought of it myself to be honest.
• If you turn on 'Mobile Friendly' from the Audit options, this requires the JavaScript crawler, so it turns it on automatically. If you deign to switch the crawler over to the non-JavaScript version, this turns OFF the Mobile Friendly option that you just turned on. It still does this, don't worry, but now it actually warns you first.
• Increased the maximum view to 500 rows, in the URL lists. Apparently 250 wasn't enough for some people.
• In the Links report, the 'Top Pages' tab in the tool shows the top 50 pages based on internal links. In the export you also got 50 pages, which seemed a bit paltry. So we upped it by 24,500 (in the export only).
• Added a new global setting (Dashboard -> Settings) called 'Excluded External URLs.' This is pre-loaded with a bunch of hosts and paths that, most of the time, there is no benefit to crawling (e.g. instagram or pinterest on every single page of your site). You can delete, change or add new options, should you wish.
• Sitebulb is now saving the version number of the software against the Audit, so it will warn you if you open an Audit from an older version of the software. In case you see some mistakes which we've already fixed. You might refer to this document in fact to find out what's changed (and that way someone will actually read all this shit).
• Added some comments to Hints related to Google Tag Manager to state that they might be redundant (e.g. To stop this: "there is no GA code - yeah but there is a GA code in GTM so fuck off").
• Similarly, added some comments to some of the Performance Hints which are redundant with HTTP/2. Because people kept having a go at us for being behind the times. We are nothing if not down with the kids.
• Added the 'HTTP Error' column, and added some additional clarification, for all lists of 'Errors' in the tool. One user found some URLs that returned a 200 status code, but were being listed as errors. This is because the content had failed to download, and it wasn't clear why, or if the URLs were being incorrectly classified. They weren't, but Sitebulb was just doing a shitty job of explaining what was going on. Naughty Sitebulb.
• Changed the 'multiple H1s' Hint from 'Issue' to 'Advisory', as one user pointed out that in HTML5 you could have perfectly valid multiple H1s (for different sections, for example). Since we still think it could help indicate an issue/misconfiguration, we've kept the Hint in.
• Removed the ability to use any special characters in Project names, as they were getting cut off or causing the software to crash. So no more Projects named 'This client is a c#£$!', sorry.

Fixes

• Some Crawl Maps were coming back with a single node. While there are some legitimate reasons for this happening (e.g. homepage is canonicalised), in the examples we looked at it should NOT have been happening. Since the Crawl Map is my favourite feature, and a single node Crawl Map is as underwhelming as an England performance in a World Cup, we prioritised this fix above all others. Well, I wrote it here above the others anyway.
• There is an 'anti-sleep' function built into Sitebulb that means your computer won't sleep or hibernate while it is crawling. This was not kicking in properly on the report building element, so your computer would go to sleep. And crash Sitebulb. So you could spend 2 days crawling a 1 million URL site, then it would crash out at the last bit. Sorry if this happened to you! Anyway, 'insomnia mode' now stretches over the entire duration of the audit (note that your computer will still sleep when not running an Audit, and it still allows your display to turn off).it 
• On the Audit Overview, we had been pulling back the number of indexed pages in Google, based on a site: search. Since this uses your local machine, it does the check using your own IP address. If you happen to be a massive spammer and blocked by Google, this check will show 0 pages indexed. Since some of our beta testers are massive spammers, they were seeing 0 and thinking the tool was buggy. We can't have that, so we have removed this feature for now. This is why we can't have nice things.
• Changed the Hint text slightly for 'URL is orphaned', to clarify that is checking for internal followed links. Apparently it wasn't obvious to some people...
• Pausing Audits has been a bit of a pain in the arse. Despite previous changes, some people were still having issues, so we've done some more work to help the figures update correctly.
• Fixed a peculiar issue which occurred when you had a Sitemap index, which included XML Sitemaps that redirected (the example we had they were going from having no trailing slash, to having a trailing slash). Sitebulb would follow the redirect and tell you about it, but it would ignore all the URLs it found! Whoops.
• Imported Audits no longer have an associated Project. Well they never originally had an Associated Project, but you could try and click on 'View Project', which would crash the software.
• The XML Sitemaps export button had an identify crisis, and was claiming to be an AMP export. We gave it a dry slap, and it's come to it's senses.
• If you stop an audit early, but let XML Sitemaps complete, you won't get a Crawl Map. Don't ask me why, because I do not know, but it happened, and that is why I'm bothering telling you so. It is fixed, so don't fret.
• A few versions ago I mentioned that we'd made the UI fit better on really shitty old Windows 7 work laptops with tiny screens. Well this was true, except for one part of the onboarding, where you needed to click somewhere off the fucking screen, and would get stuck in the onboarding forever and ever. Well, we've fixed it now so the onboarding actually works on really shitty old Windows 7 work laptops with tiny screens.

Version 1.0.10 (beta)

Released on 10th July 2017.

Updates

• A biggie - we've rebuilt all the report building to be much faster, which in particular affects LARGE AUDITS. So if you think size matters, we invite you to smash Sitebulb's back doors in.
• In line with the update above, we also added an Advanced Settings option to 'not crawl parameters', which can make a regular sized website seem a lot larger than it really is (you can also specify some parameters to still be included). It's all a question of perspective, really.
• Added a cancel button to the Import Audit overlay, just in case you change your mind in those crucial few seconds while the Audit is actually being imported, as is your wont.

Fixes

• A few of you came across this issue - if you had an incomplete Audit (e.g. paused or interrupted), you'd still have the opportunity to click 'View Latest Audit' from the dashboard. Since the Audit hadn't actually been created yet, this caused Sitebulb to throw a hissy fit. Now it will say 'Incomplete Audit', and you can go to the 'Paused Audits' screen to resume or cancel them off.
• Occasionally, when running larger Audits (a few thousand URLs), the main progress screen would just freeze. In the background it was still crawling, but the numbers wouldn't move. Similarly, if you went back and forth from the Dashboard to the Crawl Progress a few times, the UI would often freeze. We felt it would be better if the UI didn't occasionally freeze, so Gareth spent all weekend rebuilding the script for this, and it seems to have done the job.
• At the same time, we also wanted to fix the issue with pause/resume (that you would know about if you actually read the emails we send), where the UI wasn't updating new crawl numbers, and would revert back to the total when you originally paused it.
• On the Mac, re-running a previous audit using the same settings, for any Audit you ran before version 1.0.9 would progress to 102% completion, and only show you half the reports. This may well have been a developer error, and has now been resolved. Any Audits that were affected should have gone back to normal as well. 
• One of our testers found a bug whereby, instead of queuing two Audits up, the tool would just crash instead. That isn't exactly the way it's supposed to work. We think this may have been caused by some slowdown to the computer or the network, and the jobs would not get queued up properly. So we've fixed THAT issue, and hope it solves the crashing thing too.
• An eagle-eyed tester spotted this: 'The "page has more than one GTM code" seems to fail on pages which only have one but use the current implementation method of <head> component and <body> component.' Fixed.
• The very same eagle-man also spotted this: 'The <body> GTM script has a frame which flags as fail for "page has no frames". If GTM is in use, it's likely this will always fail.' We deliberately left this misconfiguration in for him to find - an Easter Egg if you will. Our warg friend did not disappoint. Now we're not flagging anything with 'noscript.'
• The sample audit that we pre-load in (which most people delete without looking at) is not exportable. But we still had the button there screaming 'export me!', which, if you succumbed to its request, would duly make the software shit itself. So we took the fucking attention seeking button away now.
• In the Advanced Settings, if you were changing max depth or max URLs to crawl, you could click the down button a few too many times and set these as negative values. Now you can't go to negatives using the down button. You can still enter negative values using your keyboard, but if you do this then you are definitely a prick, and deserve everything you get.
• Fixed an annoying, rare onboarding bug that occurred when you viewed your very first Audit, if you tried to view a report that you didn't actually switch on.

Version 1.0.9 (beta)

Released on 5th July 2017.

Updates

• On a report, if you go into the Hint data and click through to look at some of the URLs, the back button will now take you back to the Hint tab you were on before (rather than just back to the report overview, which we can all agree was fucking annoying).
• Added Gravatar to our ever-growing list of automatically suppressed external links, so Sitebulb will no longer check the status of 62,000 near identical gravatar.com links. Soon we'll add this as a configurable global setting, so you can switch it back on (if you really want to check 62,000 near identical gravatar.com links).
• When the crawl has finished and all the reports are built, Sitebulb also builds a lot of the Excel exports that you find within the reports. Previously it would do a faux 'loading' screen, which was largely, if not completely, pointless. Now it just instantly gives you the file to download or open (FYI, in our dev queue, Gareth added this ticket as 'Exports are file aware', as if they are some sort of sentient being).

Fixes

• If you switched on 'International' in the Audit settings, but Sitebulb couldn't find any hreflang, when you then went to view the International report, the tool would hang forever on the spinny loading thing. Obviously we have fixed this bug, but in reality you never should have been selecting 'International' in the first place, if you knew that there was no hreflang on there (and if you didn't know this, well, you should have know this).
• On Crawl Maps, when you hovered over the homepage it would claim that it had 0 children URLs. Other than single page websites, this was utter bullshit. Suffice it to say, we had words, and it is now a more responsible parent.
• Fixed small typo on the pre-audits checks, 'Web Frameworks' had gone full Jonathon Ross (Web Fraweworks).
• In the International export, one of the columns was misaligned on the hreflang cluster matrix. No one actually noticed this issue, but it doesn't mean it wasn't wrong (if a bear shits in the woods, can the Pope hear it?).
• On the Redirect report, the headers went missing on the top graph when you switched to the 'Data View'. Fortunately, we managed to find them and return them to their graph.
• Fixed (we think) a curious issue where Google Analytics would select the wrong property, meaning the view would be wrong. Only in rather bizarre circumstances would his happen, so it won't have affected many of you.

Version 1.0.8 (beta)

Released on 3rd July 2017.

Updates

• If you clicked to start a new Audit from an existing Project, but then had a change of heart and decided you in fact did not want to do a new Audit after all, you'd end up trapped on the overlay, with the only option to start a new Audit (which you just realised you no longer want to do). We added a 'close' button, so you will not longer be trapped thus.
• Changed a few bits of the new account creation workflow, which is nothing to do with the tool anyway so does not concern you. Stop reading this bit please.

Version 1.0.7 (beta)

Released on 30th June 2017.

Updates

No updates this time, sorry to be such an enormous disappointment.

Fixes

• This one is a biggie... on the Mac, if you docked the app, when you next tried to run it from the dock, it stayed on a loading screen forever. To clarify: IT DIDN'T FUCKING WORK AT ALL. Thankfully that shit show was spotted by an eagled-eyed beta tester, and it now works as intended.
• My wife has a really shitty, old Windows 7 laptop from work, that has the tiniest screen you've ever seen. When I tried Sitebulb on there you couldn't actually click on some of the menu options from the navigation. It was an... oversight. So we optimized the user interface to still work on really shitty old Windows 7 work laptops with tiny screens.
• Some of the Site Speed statistics data was incorrect, and is now not incorrect. It was so incorrect, by the way, that nobody actually noticed. What is it we are not paying you beta testers for anyway?
• The filtered list for Broken Page Resources did not include the column 'No. URLs Referencing Resources.' This shocking omission has been duly corrected.
• One beta user complained that the Uncrawled URLs button says 'View' when it should say 'Export', so we updated this button copy to satiate his pedantry.

Version 1.0.6 (beta)

Released on 23rd June 2017.

Updates

• In URL Lists, the Add/Remove Columns selector panel was too large for smaller screens, and the bottom was getting cut off. So we made it smaller (size isn't everything y'know).
• For larger audits, when displaying Realtime URL data, there can be a delay loading and indexing the data. Previously we just had a blank screen, which looked really shit, so we added our ubiquitous loading spinner thing.
• Some unfortunate beta users found themselves in a disastrous install-restart-uninstall-restart-install experience with 1.0.5. To avoid this, we added a version number to the installer, and instructions to uninstall old versions.
• Previously, if you did an Audit on a website using a really long Start URL, when displaying Projects on the Dashboard the URL column would be so wide that you could neither see nor click any of the action buttons on the right. We didn't think this looked very good, so when we encounter long URLs now we use ellipsis instead (...).
• Sitebulb stores crawl data on your hard drive. If you don't have enough space left on your hard drive, Sitebulb has nowhere to write the data to. Up until now, Sitebulb has handled this poorly, repeatedly smashing its head against a brick wall and spamming our bug tracking software 57,000 times. Now, however, Sitebulb gracefully crashes if it runs out of space (*ahem* user messaging to follow).
• We added a new 'Realtime URL' view so you could see the crawl progress in a URL List. But we left in the Column Selector, which wasn't that smart, and would have allowed you to query the database as it was being written to. So we removed it.
• We also removed the onboarding from this Realtime URL view, because you probably don't need it.
• MAJOR change to the UI - we moved a couple of the left hand menu items around on the Dashboard, so that all the 'Audit' items were adjacent (this was our most requested update).
• In the Redirect report, we'd forgotten to put a legend on the stacked area chart (the one at the top), so that is now in.
• Also on the Redirect report, we found occasions where the two pie charts were null (had no data), and displaying a pie chart with no pie is just not cricket. So now we just don't show them when there is no data.
• One of the Advanced Settings options is called 'Included URLs'; we've just added some examples to the copy to make it clearer how you actually use this function.
• In some of the 'Hints' sections, we split up the Hint into Indexable/Not Indexable, and the buttons on the right allow you to view the associated URLs. Apparently this was not at all intuitive, so we made it marginally clearer by adding a barely perceptible hover state tooltip.
• If you did not switch on one of the Analysis options when setting up your Project (e.g. you did not tick 'XML Sitemaps'), you would not get to see the corresponding report in your Audit. However we were still producing an empty export for you to not look at, which is a massive waste of resources and entirely unnecessary. So now we don't produce the export, and we've even removed the redundant 'export' button to boot.
• For Sample Audits, link data is not processed, but Crawl Maps are. Normally Crawl Maps include some link data about each URL when you hover over a node, but since link data is not switched on, these overlays were full of 0s, which was confusing and stupid. So we've just stripped out the link data from the overlay on Sample Audits (and also on Standard Audits where 'Link Analysis' is turned off in the Advanced Settings).
• Added the text 'Redirect Loop' to URL Details page for URLs that are in a redirect loop, to make it clearer that they are in a redirect loop.

Fixes

• Some of the graphs on the International report were not hooked up to URL Lists correctly. When performing hreflang checks, Sitebulb is only looking at Indexable URLs (which filters out a lot of the noise). This 'Indexable' filter was not getting correctly applied, but now it is.
• The Pagination graph on the Internal URLs report was not hooked up to the URL List correctly. In fact it was completely broken, and is now completely unbroken.
• Ditto a bunch of graphs on the Search Traffic report (they are now unbroken too).
• Fixed issue where incorrect data was being displayed for 'Not Indexable XML Sitemap URLs' (it was showing Indexable as well, which is just plain wrong).
• In URL Lists, the Meta Description (and other fields with many characters) often displays a 'read more' eye icon, which shows the full copy when you hover over it. This hover function had stopped working, and just looked like a weird emoji character. We fixed the hover state.
• One of the Advanced Settings options is called 'Excluded URLs'; which contained a disgraceful typo in its copy ('different' instead of 'difference'). This was confusing users left right and centre, so we had to fix it pronto.
• When you close Sitebulb while a crawl is running, the modal window that displays had a button that was slightly too far left. You'll be pleased to hear this has been shifted to its rightful place (on the right).
• In the Indexation report is a warning message for when your robots.txt redirects elsewhere. This message was not firing correctly, so users were seeing it when they shouldn't have, and they were understandably up in arms about this. The problem has been resolved and order has been restored to the realm.
• Resolved obscure issue when re-auditing a site with deleted Google accounts - Search Traffic report is no longer generated (which would probably never actually come up in real life).

Version 1.0.5 (beta)

Released on 21st June 2017.

Updates

• Split Links report up by adding new 'External' section, so Links report now only contains internal link data.
• Added new Redirects report, and canonicalized data about redirects to only live in this report.
• Added option to 'View Realtime URL Data' while crawl is in progress.
• Added Filtered URL Lists, which is now the top menu item in the Report. These are pre-filtered URL lists so you can quickly jump to e.g. Internal Indexable HTML URLs.
• Added link from URL list to view all internal linking URLs to each URL (screenshot)
• Changed 'Performance' report to break out into two - 'Site Speed' and 'Mobile Rendering', now both available in the left hand menu in a Report.
• Changed Site Speed Hints to be broken into URL Hints (e.g. total page size too big) and Resource Hints (e.g. Minify JavaScript).
• For Site Speed Resource Hints, added 'View URLs' link to column 'URLs Referencing Resource', so you can see exactly which Resources are problematic (e.g. JS not minified) and which URLs they affect.
• Added Indexable/Not Indexable counts for all Hints where it makes sense (screenshot).
• Added HTTP Status for outgoing links on single URL view.
• Moved some Hints from Links Report into Indexation Report. Links Report now only contains Hints that pertain to the distribution of link equity/PageRank (or lack thereof).
• Changed Hint icons to be actual words instead of icons (Issue/Advisory).
• Moved Project List to the dashboard front and centre - you can now view the project or jump straight into the latest audit. 'Recent Audits' is still accessible via the left menu.
• Changed default columns for all Hints and Filters.
• If you have some disallowed page resources, you will see a warning message on the Indexation report. This is now hooked up to the actual offending resource files (screenshot).
• Added messaging to Crawl Comparison page to tell you when settings have changed between Audits.
• On Site Speed report, added links to URL Lists from Insights for 'Slowest TTFB' and 'Slowest Download Time'.
• Added 'Internal Linking URLs' to all filtered URL Lists (including Hints and Insights).
• Added message to tell you when you have some paused Audits.
• Added new link to Dashboard for 'Queued Audits', which displays all Audits currently queued.

 
Fixes

• Fixed issue where Sitebulb can't match up domain with GA account (now looking for UA code on page).
• Fixed issue where some resource sizes (in Kb) were not being collected properly.
• Fixed issue where the Hint 'Does not contain Google Analytics code' was being fired incorrectly.
• Fixed issue where some On Page Hints were sometimes showing false positives (H1 Not Set and Missing Meta Description).
• Fixed: Reading time now displays in mm:ss (instead of integers).
• Fixed issue on URL Lists where columns would default if you changed number of results to display.
• Fixed issue where Total Page Size was not being collected properly.
• Fixed issue on URL List where your filter resulted in only a few rows, where editing that filter was then impossible.
• Fixed issue where crawl comparison export stopped working.
• Fixed issue where Sitemaps Report filter 'Not in XML Sitemaps' was picking up non-HTML content types.
• Fixed issue where Crawl Maps data 'links from X% of URLs' wass using percentile instead of percentage.
• Fixed column alignment issue on URL Details export.
• Fixed duplicate content issue where redirects were not being checked - now URLs must be internal, HTML, status 200, indexable in order to be checked for duplicate content.
• Fixed issue where top pages would show external URLs.

Version 1.0.1 - 1.0.4 (beta)

Never publicly released, the very early beta releases are widely considered as Sitebulb's formative years.