The Sitebulb MCP is live. Bring your crawl data into Claude & ChatGPT. The Sitebulb MCP is live. See how it works

Medium
This Hint is worth investigating further, and may warrant further attention depending on the type and quantity of URLs affected.

Review noindex and nofollow pages

What to check

Review the full set of meta robots directives, especially noindexed and nofollowed pages, to confirm each is intentional and appropriate. This is the reverse of the 'key pages are indexable' check: the goal is to ensure nothing is excluded from indexing or link equity flow by accident. noindex and nofollow are often applied together but either can be applied independently, so review each separately.

  • Review the list of noindex URLs; confirm pages that are noindexed should stay that way.

  • Separately review the nofollow URLs; if they match the noindex list 1:1, no further action is needed. If not, confirm the nofollow should remain.

  • Avoid handing off separate noindex and nofollow lists when they are duplicates.

How to check

Analyze index, noindex and nofollow pages (video instructions).

Analyze pages and resources blocked by robots.txt (video instructions).

Analyze pages and resources blocked by robots.txt (written instructions).

Review corresponding Sitebulb Hints.

Why this matters

noindex and nofollow are powerful exclusions applied deliberately for some pages and accidentally for others. Reviewing them catches valuable pages being quietly kept out of the index or cut off from link equity.

When this typically occurs

  • A noindex applied during development was never removed.

  • nofollow is applied site-wide by a plugin, restricting internal link equity.

Further reading

Tools

Sitebulb

Sitebulb Desktop

Find, fix and communicate technical issues with easy visuals, in-depth insights, & prioritized recommendations across 300+ SEO issues.

  • Ideal for SEO professionals, consultants & marketing agencies.

Sitebulb Cloud

Get all the capability of Sitebulb Desktop, accessible via your web browser. Crawl at scale without project, crawl credit, or machine limits.

  • Perfect for collaboration, remote teams & extreme scale.
;