

The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site. It allows webmasters to include additional information about each URL: when it was last updated, how often it changes, and how important it is in relation to other URLs in the site. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol and complement robots.txt, a URL exclusion protocol.
Sitemaps are particularly beneficial on websites
- * where some areas of the website are not available through the browsable interface, or
The webmaster can generate a Sitemap containing all accessible URLs on the site and submit it to search engines. Since Google, MSN, Yahoo, and Ask use the same protocol now, having a Sitemap would let the biggest search engines have the updated pages information.
Sitemaps supplement and do not replace the existing crawl-based mechanisms that search engines already use to discover URLs. By submitting Sitemaps to a search engine, a webmaster is only helping that engine's crawlers to do a better job of crawling their site(s). Using this protocol does not guarantee that web pages will be included in search indexes, nor does it influence the way that pages are ranked in search results.
History of Sitemaps
- Google first introduced Sitemaps 0.84 in June 2005 so web developers could publish lists of links from across their sites.
- Google, MSN and Yahoo announced joint support for the Sitemaps protocol in November 2006. The schema version was changed to "Sitemap 0.90", but no other changes were made.
- In April 2007, Ask.com and IBM announced support for Sitemaps. Also, Google, Yahoo, MS announced auto-discovery for sitemaps through robots.txt.
- In May 2007, the state governments of Arizona, California, Utah and Virginia announced they would use Sitemaps on their web sites.
The Sitemaps protocol is based on ideas from "Crawler-friendly Web Servers".
XML Sitemap Format
The Sitemap Protocol format consists of XML tags. The file itself must be UTF-8 encoded. (Sitemaps can also be just a plain text list of URLs. They can also be compressed in .gz format.)
Sample
A sample Sitemap that contains just one URL and uses all optional tags is shown below.http://w3c-at.de 2006-11-18 daily 0.8
Search Engine Submission
If Sitemaps are submitted directly to a search engine (pinged), it will return status information and any processing errors. The details involved with submission will vary with the different search engines. The location of the Sitemap can also be included in the robots.txt file by adding the following line to robots.txt:
Sitemap:
The The following table lists the Sitemap submission URLs for several major search engines: As with all XML files, any data values (including URLs) must use entity escape codes for the characters : ampersand(&), single quote ('), double quote ("), less than (<) and greater than (>).
Search engine
Submission URL
Help page
Google
http://www.google.com/webmasters/tools/ping?sitemap=
How do I resubmit my Sitemap once it has changed?
Yahoo!
http://search.yahooapis.com/SiteExplorerService/V1/updateNotification?appid=SitemapWriter&url=
http://search.yahooapis.com/SiteExplorerService/V1/ping?sitemap=
Does Yahoo! support Sitemaps?
Ask.com
http://submissions.ask.com/ping?sitemap=
Q: Does Ask.com support sitemaps?
Moreover.com
http://api.moreover.com/ping?u=
Live Search
http://webmaster.live.com/ping.aspx?siteMap=
Webmaster Tools (beta) Sitemap limits
Sitemap files have a limit of 50,000 URLs and 10 MegaBytes per sitemap. Sitemaps can be compressed using gzip, reducing bandwidth consumption. Multiple sitemap files are supported, with a Sitemap index file serving as an entry point for a total of 1000 sitemaps.Notes
See also
External links
This article is licensed under the GNU Free Documentation License.
Last updated on Friday July 25, 2008 at 01:55:05 PDT (GMT -0700)
View this article at Wikipedia.org - Edit this article at Wikipedia.org - Donate to the Wikimedia Foundation
Copyright © 2008, Dictionary.com, LLC. All rights reserved.











