Review the full set of meta robots directives, especially noindexed and nofollowed pages, to confirm each is intentional and appropriate. This is the reverse of the 'key pages are indexable' check: the goal is to ensure nothing is excluded from indexing or link equity flow by accident. noindex and nofollow are often applied together but either can be applied independently, so review each separately.
Review the list of noindex URLs; confirm pages that are noindexed should stay that way.
Separately review the nofollow URLs; if they match the noindex list 1:1, no further action is needed. If not, confirm the nofollow should remain.
Avoid handing off separate noindex and nofollow lists when they are duplicates.
Analyze index, noindex and nofollow pages (video instructions).
Analyze pages and resources blocked by robots.txt (video instructions).
Analyze pages and resources blocked by robots.txt (written instructions).
Review corresponding Sitebulb Hints.
noindex and nofollow are powerful exclusions applied deliberately for some pages and accidentally for others. Reviewing them catches valuable pages being quietly kept out of the index or cut off from link equity.
A noindex applied during development was never removed.
nofollow is applied site-wide by a plugin, restricting internal link equity.
Crawling & Indexing Signals for SEO by Tory Gray
Improving Crawling & Indexing with Noindex, Robots.txt & Rel Attributes by Shaikat Ray
Sitebulb
Find, fix and communicate technical issues with easy visuals, in-depth insights, & prioritized recommendations across 300+ SEO issues.
Get all the capability of Sitebulb Desktop, accessible via your web browser. Crawl at scale without project, crawl credit, or machine limits.