The Sitebulb MCP is live. Bring your crawl data into Claude & ChatGPT. The Sitebulb MCP is live. See how it works

High
This Hint is very important, and definitely warrants attention.

Check for duplicate or mismatched meta robots directives

What to check

Confirm that meta robots directives are not duplicated across the page, and that instructions set in the HTML <head> and the HTTP X-Robots-Tag header do not conflict.

  • Note any pages or types where more than one meta robots directive is present; this isn't inherently a problem but creates risk of future conflicts.

  • Flag any cases where instructions conflict (e.g. one tag sets noindex while another sets index).

Note: meta robots and canonicals can also be specified in the HTTP header, which is a fine alternative.

How to check

Review crawl results for the corresponding Sitebulb Hints (listed on the right of this page).

Why this matters

When two directives disagree, search engines pick one and you lose control of the outcome. Conflicting noindex/index signals can drop pages you want indexed or index pages you don't.

When this typically occurs

  • A tag manager or plugin adds a second meta robots tag alongside the template's.

  • HTML and HTTP header directives are set independently and drift out of sync.

Further reading

Tools

Sitebulb

Sitebulb Desktop

Find, fix and communicate technical issues with easy visuals, in-depth insights, & prioritized recommendations across 300+ SEO issues.

  • Ideal for SEO professionals, consultants & marketing agencies.

Sitebulb Cloud

Get all the capability of Sitebulb Desktop, accessible via your web browser. Crawl at scale without project, crawl credit, or machine limits.

  • Perfect for collaboration, remote teams & extreme scale.
;