Comments on: Be a Normalizer – a C14N Exterminator http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/ Never Mess With a Woman Who Can Pull Rank Tue, 12 Jan 2010 20:10:47 -0600 http://wordpress.org/?v=2.8.5 hourly 1 By: httpwebwitch http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-79531 httpwebwitch Mon, 19 Jan 2009 19:27:43 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-79531 This post has been nominated for a SEMMY http://www.semmys.org/2009/search-tech-all-2009-nominees/ This post has been nominated for a SEMMY
http://www.semmys.org/2009/search-tech-all-2009-nominees/

]]> By: Mike Riley http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-78672 Mike Riley Sat, 04 Oct 2008 07:46:09 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-78672 I usually find that the best way to handle the problem of Canonicalization, as well as a slew of others, is to use a framework that just rewrites all URLs to a single file and then routes them. Ala ExpressionEngine or CodeIgniter (or Ruby!). The real beauty of doing this is that you don't really need to ever screw around with regular expressions, and definitely don't need to mess with the .htaccess or apache directives in order to reconfigure your URL rules, everything is defined in the same language you're using to actually generate the pages. It really makes for a much smoother way of handling this. I usually find that the best way to handle the problem of Canonicalization, as well as a slew of others, is to use a framework that just rewrites all URLs to a single file and then routes them. Ala ExpressionEngine or CodeIgniter (or Ruby!). The real beauty of doing this is that you don’t really need to ever screw around with regular expressions, and definitely don’t need to mess with the .htaccess or apache directives in order to reconfigure your URL rules, everything is defined in the same language you’re using to actually generate the pages. It really makes for a much smoother way of handling this.

]]> By: httpwebwitch http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60623 httpwebwitch Tue, 26 Feb 2008 23:44:48 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60623 @dan: the rewriterule for my example is something more like RewriteRule /restaurants/([0-9]+)/.* /r.php?id=$1 Validating your fluff is usually not handled with rewriting rules; they're accomplished in the code since you need to compare the "fluff" with some actual data associated with a RowID. BTW in Wordpress they call that a "post slug" used to create "pretty permalinks" I presume your fluff will be some modified and hyphenated version of a title or name in the data. Once you've figured out which restaurant is #46335, parse the URL and see if your fluff is an exact match for the string you expect to be associated with that data row. done properly, your rewriterule becomes: RewriteRule /restaurants/([0-9]+)/ (.*) /r.php?id=$1&fluff=$2 create a slug and compare it to your fluff. If they're not identical, you've already got your slug so redirect to a new URL using that. the rewriterules giveth, and the code taketh away @dan: the rewriterule for my example is something more like

RewriteRule /restaurants/([0-9]+)/.* /r.php?id=$1

Validating your fluff is usually not handled with rewriting rules; they’re accomplished in the code since you need to compare the “fluff” with some actual data associated with a RowID.

BTW in Wordpress they call that a “post slug” used to create “pretty permalinks”

I presume your fluff will be some modified and hyphenated version of a title or name in the data.
Once you’ve figured out which restaurant is #46335, parse the URL and see if your fluff is an exact match for the string you expect to be associated with that data row.

done properly, your rewriterule becomes:

RewriteRule /restaurants/([0-9]+)/
(.*) /r.php?id=$1&fluff=$2

create a slug and compare it to your fluff. If they’re not identical, you’ve already got your slug so redirect to a new URL using that.

the rewriterules giveth, and the code taketh away

]]> By: Chad Ledford http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60567 Chad Ledford Tue, 26 Feb 2008 16:14:32 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60567 Great post on canonicalization. Digg has actually just fixed one of their issues (http://www.3tailer.com/sundry/digg-implements-the-1000000-idea-non-www-301-redirect) Great post on canonicalization. Digg has actually just fixed one of their issues (http://www.3tailer.com/sundry/digg-implements-the-1000000-idea-non-www-301-redirect)

]]> By: dan http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60429 dan Mon, 25 Feb 2008 22:51:49 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60429 I've recently been working on site that uses a lot of "URL Fluffing" (great term) as in Tip #3. Fixing this I guess would just be a simple rewrite rule. Unfortunately rewrite rules are never simple for me. Using the given example C14N URL of http://www.example.com/restaurants/46335/jimmys-lunch-hamilton-ontario.php would the rewrite rule just be RewriteRule http://www.example.com/restaurants/(.*)/.* http://www.example.com/restaurants/46335/jimmys-lunch-hamilton-ontario.php [R=301,L] I believe such an approach would require a separate rule for each URL, but I guess you have to maintain your fluff somewhere. I’ve recently been working on site that uses a lot of “URL Fluffing” (great term) as in Tip #3.

Fixing this I guess would just be a simple rewrite rule. Unfortunately rewrite rules are never simple for me.

Using the given example C14N URL of
http://www.example.com/restaurants/46335/jimmys-lunch-hamilton-ontario.php

would the rewrite rule just be
RewriteRule http://www.example.com/restaurants/(.*)/.* http://www.example.com/restaurants/46335/jimmys-lunch-hamilton-ontario.php [R=301,L]

I believe such an approach would require a separate rule for each URL, but I guess you have to maintain your fluff somewhere.

]]> By: httpwebwitch http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60427 httpwebwitch Mon, 25 Feb 2008 22:29:09 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60427 Yes Soeren. I didn't delve into the "why" of C14N in this post, since that info is widely available elsewhere, but that's the crux: C14N prevents double-indexing and supplemental problems in the SERPs. Taken further, the same techniques you use for normalizing can catch and redirect IBLs that include common typos, odd punctuation or other abnormalities. I also neglected the "how", as in "how to fix it". I only covered the "what" here, which apologetically only helps you identify the problems, it doesn't offer solutions. Nonetheless I think this post is a helpful guide, if only for the first half of the journey. Cheers, hww Yes Soeren. I didn’t delve into the “why” of C14N in this post, since that info is widely available elsewhere, but that’s the crux: C14N prevents double-indexing and supplemental problems in the SERPs. Taken further, the same techniques you use for normalizing can catch and redirect IBLs that include common typos, odd punctuation or other abnormalities.

I also neglected the “how”, as in “how to fix it”. I only covered the “what” here, which apologetically only helps you identify the problems, it doesn’t offer solutions. Nonetheless I think this post is a helpful guide, if only for the first half of the journey.

Cheers, hww

]]> By: Soeren Sprogoe http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60412 Soeren Sprogoe Mon, 25 Feb 2008 20:46:15 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60412 One thing that surprised me, when I originally learned about c14n, was that Google actually treats yourdomain.com and yourdomain.com/default.aspx as two completely different pages! So page value will be divided out over the two, but one of them wil (most likely) be marked as dupe content and be removed from the index! So I can definately confirm that you need to be consequent when doing (internal) linking: - Allways use lower case. - Allways use www. (or don't). - Allways link to your frontpage with either / or default.aspx (or index.php, depending on your choice of technology). One thing that surprised me, when I originally learned about c14n, was that Google actually treats yourdomain.com and yourdomain.com/default.aspx as two completely different pages!

So page value will be divided out over the two, but one of them wil (most likely) be marked as dupe content and be removed from the index!

So I can definately confirm that you need to be consequent when doing (internal) linking:
- Allways use lower case.
- Allways use www. (or don’t).
- Allways link to your frontpage with either / or default.aspx (or index.php, depending on your choice of technology).

]]> By: Ian Ring http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60319 Ian Ring Mon, 25 Feb 2008 06:51:36 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60319 thanks Rae a lot of material was omitted from this post... watch for more installments like this one on ianring.com and at webmasterworld in the coming year. I consider it a compliment to be the only guestwhore who got dugg. :) thanks Rae
a lot of material was omitted from this post… watch for more installments like this one on ianring.com and at webmasterworld in the coming year. I consider it a compliment to be the only guestwhore who got dugg. :)

]]> By: Rae Hoffman http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60247 Rae Hoffman Sun, 24 Feb 2008 19:57:37 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60247 Ian... wow... that was a lot of info... 1. Never post in dual categories on the blog :) 2. You weren't supposed to TEACH anyone anything. Thanks and awesome job dude. ;) Ian… wow… that was a lot of info…

1. Never post in dual categories on the blog :)
2. You weren’t supposed to TEACH anyone anything.

Thanks and awesome job dude. ;)

]]> By: rdrysdale http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60108 rdrysdale Sat, 23 Feb 2008 19:20:12 +0000 http://www.sugarrae.com/be-a-normalizer-a-c14n-exterminator/#comment-60108 Alright, I finally had the time to absorb this. Nice auditing process. It takes a little while, but clearly doesn't need to be done on a weekly basis, so it's well worth the setup. My problem has always been understanding what's wrong and not having the technical know-how to fix it. Slowly getting better on that front. Thanks for the referrals. We're working with ISAPI rewrite now to counter some issues. Alright, I finally had the time to absorb this. Nice auditing process. It takes a little while, but clearly doesn’t need to be done on a weekly basis, so it’s well worth the setup. My problem has always been understanding what’s wrong and not having the technical know-how to fix it. Slowly getting better on that front. Thanks for the referrals. We’re working with ISAPI rewrite now to counter some issues.

]]>