Bigger & faster crawling

These articles provide guidance for instances where you want to crawl faster than normal. In general, this tends to coincide with also crawling larger websites.

• How to crawl really fast
• How to crawl large websites
• Limiting the Sitebulb crawler for faster and cleaner audits
• How to crawl with Sitebulb on AWS
• How to configure your machine for crawling large websites