Generating a Google SiteMap File

Google provides a free service, named Google SiteMaps, that you can use to monitor and improve the way that Google indexes the pages on your website. For example, you can use Google SiteMaps to discover which Google search queries have returned pages from your website and the ranking of your pages in Google search results. You also can use Google SiteMaps to view any problems that the Google crawler encounters when indexing your site.

You can sign up for Google SiteMaps by visiting the following URL:

http://www.google.com/webmasters/sitemaps

To use Google SiteMaps, you must provide Google with the URL of a Google SiteMap file hosted on your website. The Google SiteMap file is an XML file that contains a list of URLs you want Google to index.

The Google SiteMap XML file has the following format:

<?xml version="1.0" encoding="UTF-8"?> <urlset xmlns="http://www.google.com/schemas/sitemap/0.84">   <url>     <loc>http://www.example.com/</loc>     <lastmod>2005-01-01</lastmod>   </url>   <url>     <loc>http://www.example.com/sample.html/</loc>     <lastmod>2006-03-11</lastmod>   </url> </urlset> 

The Google SiteMap file contains a simple list of <url> elements that contain <loc> elements representing the location of the URL and <lastmod> elements representing the last modified date of the URL.

Note

The Google SiteMap file also can contain <changefreq> and <priority> elements. The <changefreq> element indicates how frequently a URL changes, and the <priority> element represents the priority of a URL relative to other URLs in your site. These elements are optional and are ignored here.

You can generate a Google SiteMap file automatically from an ASP.NET SiteMap. The HTTP Handler in Listing 18.20 generates a Google SiteMap that conforms to Google's requirements for a valid SiteMap file.

Listing 18.20. PublicSiteMap.ashx

[View full width]

<%@ WebHandler Language="VB"  %> Imports System Imports System.Web Imports System.Xml Imports System.Text Imports System.IO Public Class PublicSiteMap     Implements IHttpHandler     Private _xmlWriter As XmlWriter     Public Sub ProcessRequest(ByVal context As HttpContext) Implements IHttpHandler .ProcessRequest         context.Response.ContentType = "text/xml"         Dim settings As New XmlWriterSettings()         settings.Encoding = Encoding.UTF8         settings.Indent = True         _xmlWriter = XmlWriter.Create(context.Response.OutputStream, settings)         _xmlWriter.WriteStartDocument()         _xmlWriter.WriteStartElement("urlset", "http://www.google.com/schemas/sitemap/0.84")         ' Add root node         AddUrl(SiteMap.RootNode)         ' Add all other nodes         Dim nodes As SiteMapNodeCollection = SiteMap.RootNode.GetAllNodes()         For Each node As SiteMapNode In nodes             AddUrl(node)         Next         _xmlWriter.WriteEndElement()         _xmlWriter.WriteEndDocument()         _xmlWriter.Flush()     End Sub     Private  Sub AddUrl(ByVal node As SiteMapNode)         ' Skip empty Urls         If String.IsNullOrEmpty(node.Url) Then             Return         End If         ' Skip remote nodes         If node.Url.StartsWith("http",True,Nothing) Then             Return         End If         ' Open url tag         _xmlWriter.WriteStartElement("url")         ' Write location         _xmlWriter.WriteStartElement("loc")         _xmlWriter.WriteString(GetFullUrl(node.Url))         _xmlWriter.WriteEndElement()         ' Write last modified         _xmlWriter.WriteStartElement("lastmod")         _xmlWriter.WriteString(GetLastModified(node.Url))         _xmlWriter.WriteEndElement()         ' Close url tag         _xmlWriter.WriteEndElement()     End Sub     Private Function GetFullUrl(ByVal url As String) As String         Dim context As HttpContext =  HttpContext.Current         Dim server As String =  context.Request.Url.GetComponents( UriComponents .SchemeAndServer,UriFormat.UriEscaped)         Return Combine(server,url)     End Function     Private Function Combine(ByVal baseUrl As String, ByVal url As String) As String         baseUrl = baseUrl.TrimEnd(New Char() {"/"c})         url = url.TrimStart(New Char() {"/"c})         Return baseUrl + "/" + url     End Function     Private Function GetLastModified(ByVal url As String) As String         Dim context As HttpContext =  HttpContext.Current         Dim physicalPath As String =  context.Server.MapPath(url)         Return File.GetLastWriteTimeUtc(physicalPath).ToString("s")     End Function     Public ReadOnly Property IsReusable() As Boolean Implements IHttpHandler.IsReusable         Get             Return True         End Get     End Property  End Class 

The HTTP Handler in Listing 18.20 generates an XML file by iterating through each of the nodes in an ASP.NET Site Map. The XML file is created with the help of the XmlWriter class. This class is used to generate each of the XML tags.

Note

You can think of an HTTP Handler is a lightweight ASP.NET page. You learn about HTTP Handlers in Chapter 25, "Working with the HTTP Runtime."

The file in Listing 18.21 contains the XML file returned by the PublicSiteMap.ashx handler when the Handler is called from the sample application contained on the CD that accompanies this book. (The file has been abridged for reasons of space.)

Listing 18.21. PublicSiteMap.ashx Results

<?xml version="1.0" encoding="utf-8"?> <urlset xmlns="http://www.google.com/schemas/sitemap/0.84">   <url>     <loc>http://localhost:2905/SiteMaps/Default.aspx</loc>     <lastmod>2005-10-30T03:13:58</lastmod>   </url>   <url>     <loc>http://localhost:2905/SiteMaps/Products/Default.aspx</loc>     <lastmod>2005-10-28T21:48:04</lastmod>   </url>   <url>     <loc>http://localhost:2905/SiteMaps/Services</loc>     <lastmod>2005-10-30T04:31:57</lastmod>   </url>   <url>     <loc>http://localhost:2905/SiteMaps/Employees/Default.aspx</loc>     <lastmod>1601-01-01T00:00:00</lastmod>   </url>   <url>     <loc>http://localhost:2905/SiteMaps/Products/FirstProduct.aspx</loc>     <lastmod>2005-10-30T03:43:52</lastmod>   </url> </urlset> 

When you sign up at the Google SiteMaps website, submit the URL of the PublicSiteMap.ashx file when you are asked to enter your SiteMap URL. The Google service retrieves your SiteMap from the handler automatically.