Question: There’s nothing on my blog that I don’t want indexed. Does that mean I do NOT need a robots file?
Answer: Your robots.txt file has two functions. First, to tell Google, Yahoo, and Bing what you do NOT want indexed. Second, to tell them the location of your XML site map. So even if you don’t care about the former, you should care about the latter. Just use your robots.txt file to point to the location of your sitemap.xml file. Be sure to put your robots.txt file in your ‘root’ or highest directory as in http://www.jasonmcdonald.org/robots.txt.
Got a question? Click this link to email Jason or call 888-993-1122. Dr. Jason McDonald is founder and Senior SEO / Social Media Director of the JM Internet Group. He teaches the SEO training classes for the group, and therefore provides most of the SEO tips for this blog. His goal with this blog is to provide an easy 'one-stop shop' for the busy marketer looking for tips, tricks, and secrets on how to get to the top of Google and Bing for free using proven SEO tactics. When not dreaming up SEO tips, Dr. McDonald lives in the San Francisco Bay Area with his wife, two dogs, a cat, four iguanas and twelve children (just kidding). Really you read down to the bio on this guy? Enjoy the blog.
on SEO | AdWords | Social Media