Question: How do you create the robots.txt and where do it go in the website?
Answer: Your robots.txt file has two functions. First, to tell Google and other search robots what NOT to index. Second, to tell them the location of your sitemap.xml file (the ‘map’ to your website). You place it in your root directory as in http://www.jm-seo.net/robots.txt.
You can follow the easy syntax by just using NOTEPAD or another editor. Or you can log into Google Webmaster tools, and use their built-in robots.txt editing tool. Then be sure to upload to your website and locate it in the ‘root’ directory. We cover this in the session on ‘website structure.’
Got a question? Click this link to email Jason or call 888-993-1122. Dr. Jason McDonald is founder and Senior SEO / Social Media Director of the JM Internet Group. He teaches the SEO training classes for the group, and therefore provides most of the SEO tips for this blog. His goal with this blog is to provide an easy 'one-stop shop' for the busy marketer looking for tips, tricks, and secrets on how to get to the top of Google and Bing for free using proven SEO tactics. When not dreaming up SEO tips, Dr. McDonald lives in the San Francisco Bay Area with his wife, two dogs, a cat, four iguanas and twelve children (just kidding). Really you read down to the bio on this guy? Enjoy the blog.
on SEO | AdWords | Social Media