High This Hint is very important, and definitely warrants attention. Issue This Hint represents an error or problem that needs to be fixed.

Canonical points to a disallowed URL

This means that the URL in question has a canonical element whereby the canonical URL is disallowed in robots.txt.

Why is this important?

The point of a canonical is to explicitly and unambiguously indicate a preferred URL. However in this situation, the canonical instructions are sending conflicting messages to the search engines - which typically means they will ignore the canonicals entirely and try to make their own decisions about what 'the canonical URL' should be.

The net result is that it could lead to incorrect URLs being indexed, resulting in duplicate content issues.

What does the Hint check?

This Hint will trigger for any internal URL which contains a canonical link element that uses a disallow URL as the canonical URL.

Examples that trigger this Hint:

Consider the URL: https://example.com/page-a

The Hint would trigger for this URL if it had a canonical URL pointing at another internal page,

<link rel="canonical" href="https://example.com/page-b" />

where this canonical URL https://example/com/page-b is disallowed in robots.txt:

User-agent: *
Disallow: /page-b

How do you resolve this issue?

This is a scenario where the canonical configuration is completely broken. You will first need to establish if Page B is indeed the correct canonical for Page A, and if so remove the disallow rule from robots.txt. If in fact Page B should be disallowed, then Page A should be updated to change the canonical tag, either to become self-referential or to point to another URL.

Issues like this will typically need to be resolved on a case by case basis. 

There's a version of Sitebulb for everyone!

No project limits. No crawl credits. We save you time and we save you money.

Sitebulb Desktop

Try Sitebulb's award-winning desktop crawler for Windows or Mac:

Try our fully featured 14 day trial. No credit card required.

Try Sitebulb Desktop for Free

Sitebulb Server

Everything you love about cloud crawling, paired with everything you love about Sitebulb:

We offer fully managed cloud server plans or a DIY server license.

Explore Sitebulb Server