June 20, 2008

Make a robots.txt file

Like a traveler coming to a country, search engine robots need a guide — which is, the robot.txt file. It's a specific guide, though, as it'll only tell the Robot which places he shouldn't see.

A robots.txt file will prevent the Robots from going to some pages with sensitive material, web pages that you don't want to be found through Google search (for instance, the "shopping cart"), pages that are not important or can be negative for your rankings. And, you can direct the Robot to other, keyword-rich pages, instead.

So, if there's something to hide, a robots.txt file is a must for your website. It helps you keep the Robot away from anything that's not good for your Search Engine rankings. Yep, just tell him not to go here or there — and he'll believe you.

You can make a robots.txt file yourself at www.robotstxt.org, though it's rather your webmaster's business. So ask your webmaster to write a robots.txt file for your site, or do it yourself.

After you have the robots.txt file, run it through a validator to ensure it's written correctly.
Hundreds of robots.txt validators can be found on the web. You can apply Google's tool, or this one, for instance:

As soon as the robots.txt file's correct, you needn't worry, as it will only do you a lot of good, and no harm.

Make different paths to reach a page

When you create a page, try to make links to other pages from it, where possible.

Make different pages on your website link to each other.