Duplicate Content and the Canonical Fix

There’s a persistent myth that Google runs a duplicate content penalty, a punishment meted out to sites caught saying the same thing twice. It scares people out of perfectly reasonable things, like syndicating an article or having similar product descriptions. The myth survives because it contains a shadow of truth. There’s no penalty. But there is a tax, and most sites pay it without knowing, because the commonest form of duplicate content isn’t plagiarism or laziness. It’s one page that exists at several addresses.

Consider how many URLs can serve your homepage. The bare domain and the www version. HTTP and HTTPS. With a trailing slash and without. Add an uppercase letter somewhere and many servers will still answer. Append a tracking parameter from your newsletter and the page loads fine, at yet another address. To you these are all obviously the same page. To a crawler, a URL is an identity, and each distinct string is a distinct page until proven otherwise. Your one homepage might be eight pages. An e-commerce category page with sortable columns and filter parameters might be eight hundred.

Why is this a tax rather than a mere untidiness? Two reasons. First, links. When other sites link to you, they link to whatever address they happened to have. If your page lives at four URLs, its reputation gets split four ways, and each fragment is weaker than the whole would have been. In a contest where you and a competitor have equally good pages, the one whose links all agree on a single address wins. Second, crawling. Search engines allocate finite attention to your site. Every visit spent re-crawling a duplicate is a visit not spent discovering something new. Small sites rarely feel this. Big sites feel it acutely.

The fix has been standard since 2009, and it’s one line of HTML. The canonical link element, a tag in the page’s head reading rel=“canonical” with an href pointing at the preferred URL, says: whatever address you fetched me from, this is my real one. Credit everything here. It’s a page declaring its own home address. Every variant, the tracking-parameter version, the uppercase oddity, the filtered view, carries the same tag pointing to the one true URL, and the split reputation flows back together. Even the canonical page itself should carry a tag pointing at itself, so there’s no ambiguity anywhere.

Two properties of this mechanism are worth understanding because they explain its occasional failures. First, a canonical is a hint, not a command. Search engines usually honor it, but if your other signals disagree, if your sitemap lists one URL, your internal links use another, and your canonical names a third, the crawler may overrule you and pick its own. Consistency is what makes the hint stick. Say the same thing everywhere. Second, a canonical is not a redirect. A redirect sends the visitor to the other address; a canonical leaves them where they are and only speaks to crawlers. This makes canonicals right for cases where the duplicate should keep existing for humans, like a filtered product list or a print view, and redirects right for cases where the old address should die, like an HTTP-to-HTTPS migration or a renamed page.

The failure modes are almost funny. Because canonical tags live in templates, a template mistake replicates everywhere at once. I’ve seen sites where every page’s canonical pointed at the homepage, which politely asks search engines to forget every page except one. I’ve seen canonicals pointing to staging servers, to deleted pages, to the HTTP version of an HTTPS site. The tag is powerful in proportion to how invisible it is. No human visitor ever sees it, so nobody notices when it’s wrong. It can sit there quietly misdirecting crawlers for years. This is a general truth about SEO problems, and it’s why I built GazeSite to read the actual HTML a page serves rather than trusting anyone’s memory of what the templates do. The failures aren’t subtle once you look at the head of the document. Missing canonical, wrong canonical, two canonicals that disagree. They’re only subtle because looking is rare.

What about the case people actually worry about, the same text on two truly different sites? Syndication, republishing, quoting at length? Search engines handle it undramatically: they pick one version to show, usually the one they judge original or more authoritative, and mostly ignore the rest. Your syndicated copy doesn’t hurt you; it just may not help you. If you republish elsewhere, having the copy carry a canonical pointing back to your original is the courteous, effective arrangement when you can get it.

The whole topic reduces to a principle simple enough to remember without any of the vocabulary: every page on your site should have exactly one address, and everything, your links, your sitemap, your redirects, your canonical tags, should agree on what it is. Search engines are machines for resolving identity, and you make their job easy or hard. A site that speaks about itself with one voice gets believed. A site that gives a different name every time it’s asked gets guessed at, and the guesses are never as good as the truth you could have simply told.