Jun 28, 2012

Add multiple xml sitemap files robots.txt

robots.txt file helps to pass message to search engine bots whether to allow or disallow a webpage or a directory for index

robots.txt file look like,

User-agent: * Disallow: /abc/

In Detail: User-agent: * Means this section applies to all robots

Disallow: Means that the robot should not visit any pages in the mentioned folder.

If you have a single or multiple XML sitemap then add in robots.txt file.

Below is an example of how multiple xml sitemap can be added to robots.txt

User-agent: *

Disallow: /includes/

Disallow: /scripts/

Sitemap: http://www.yoursite.com/sitemap1.xml

Sitemap: http://www.yoursite.com/sitemap2.xml

Sitemap: http://www.yoursite.com/sitemap3.xml

No comments:

Post a Comment

Free SEO Related Top Articles, Articles on SEO & SEM, Latest Articles on SEO Tips

 
Latest Topics on SEO, SEM Updates Free SEO Directory List Free SEO | SEM | Google Top Videos