How Barry Adams Audits Huge Websites Without Crawl Credit Anxiety

The Client

Barry Adams is a well-known freelance SEO consultant, primarily doing SEO for news publishers. Most of his client projects are site audits for news websites, which always start with a full crawl of the client's site. News websites are almost always quite large, so these initial crawls are big and time consuming. He told us all about how Sitebulb Cloud has transformed his audit process. 

The Challenge

Prior to Sitebulb Cloud, I relied mostly on Lumar (formerly DeepCrawl) for crawling large publishing sites. I used Sitebulb Desktop locally on my PC for smaller publishing sites, and Screaming Frog for small focused crawls (like URL lists and XML sitemaps). 

However, the thing about a cloud crawler like Lumar is that every URL you crawl is a credit you consume. This means that sometimes, when a crawl throws up an unexpected issue that consumes a lot of credits, you end up with a near-useless crawl report. Then I would have to reconfigure the crawl and run it again to avoid the crawl waste issue, but by that point it has cost me a lot of credits. 

This was a constant source of anxiety for me for every new project. 

The Solution

With Sitebulb Cloud, I have no more crawl credit anxiety. There are no crawl credits with Sitebulb Cloud; the pricing structure is completely different. 

Now, I can crawl any site and if there's a crawl issue I can just start a new crawl. Simple as that. I don't have to worry about wasting precious crawl credits anymore. 

I didn't realise how much stress this used to cause me until I started using Sitebulb Cloud and felt a huge load fall off my shoulders. The ability to crawl large websites as often and as intensely as I want – with no risk of incurring extra costs – is immensely liberating, and really allows me to focus on the fun bits of auditing websites.

Recently, I was running two simultaneous crawls on my cloud server, both using JavaScript rendering with Sitebulb’s Chrome Crawler, and the fact that I can do this at top speed without a) having my own PC serve as an unwanted and extravagantly loud office radiator and b) having my dedicated crawl PC pretend it's a fighter jet with its fan noises, is truly epic.

The level of technical analysis that Sitebulb provides is second to none and, for me, really elevates it as a crawler above all its rivals. Combined with the ease of use of the Cloud edition and the freedom it gives me, I can't imagine using anything else now.

The Results

Sitebulb Cloud has made my auditing process so much easier and more cost effective. No matter how many crawls I need, my monthly costs are the same; I don't have to top up credits when I run out on large site crawls. 

Fixed monthly costs for unlimited crawling. I cannot begin to express how awesome this is. Sitebulb was already my fav tech SEO crawler, it's now my favourite SEO tool for publishers, period.

I'd highly recommend using Sitebulb Cloud if you want unlimited, hassle-free crawling. You get all the advantages of legacy SaaS crawlers with none of the drawbacks!