Overview of XPath, XPointer, and XQuery

An XML document represents a convenient package of related XML data. However, there are many situations where people need to work with a subset of data within a single document or an aggregation of data from several documents. To print a shipping label, they might want to grab just the address information from an order document. To print a fraud-resistant receipt, they might want to reference just the last four digits of the credit card number. To produce an upgrade notification, they might want contact information from every order document that includes the purchase of a specific product. To perform these types of operations, it helps to have the means of specifying such subsets and aggregations of document data.

XML Path Language (XPath) provides a syntax for addressing individual nodes within a document hierarchy. Developers specify a path from the document node to the target node. It includes basic selection criteria so that a particular XPath expression can differentiate among similar nodes. To grab only the billing address from orders, a developer could use XPath to specify the "Address" element within an "Order" document whose "addType" attribute is set to "ship" or "billship." XPath is a widely used W3C Recommendation.

When referring to content within a document, authors sometimes need more precision than XPath provides. XML Pointer Language (XPointer) extends the XPath syntax to include the notions of points and ranges. With points, authors can specify locations within a particular node. With ranges, they can specify a set of information that crosses node boundaries. To create a fraud-resistant receipt, an author could use XPointer to specify the last four characters of the "Number" element within the "Card" element of an "Order" document. XPointer is useful primarily for hypertext applications and is taking a long time to work its way through the W3C standards process.

As with other forms of data, people often want to search among collections of XML documents. XML Query Language (XQuery) provides a framework that enables developers to indicate the data they want, the criteria this data must satisfy , and intermediate processing steps. It uses XPath as the format for specifying many of these parameters. To create a mailing list for a product upgrade notice, a developer could use XQuery to search "Order" documents for "Address" elements with an "addType" attribute set to "ship" or "billship" where the document has a "Product" element whose value is "FooBar Version 5." The expectations for adoption of XQuery as a standard search syntax are very high, but it was still a Working Draft at the time of this writing.

How They Work

With XPath, you specify a particular node or set of nodes by indicating a navigation path. Suppose you wanted to refer to each of the "Description" elements in our example order from Example 2-6. You could use three separate XPath expressions to address each of the three desired elements individually, as shown in Example 3-3.

Example 3-3

 /Order/LineItems/LineItem[1]/Description /Order/LineItems/LineItem[2]/Description /Order/LineItems/LineItem[3]/Description 

Notice that each expression in Example 3-3 provides a full path from the document element to the target element. Each one uses an index in brackets to indicate a specific "LineItem" element based on the order in which they appear in the document. It's somewhat inconvenient to use a different expression for each "LineItem" element, especially if you don't know beforehand how many there are. Example 3-4 shows how to return all the "Description" elements in a document, regardless of how many there are.

Example 3-4

 //Description 

The use of a double slash rather than a single slash indicates that desired "Description" elements may have any path to the document element rather than the specific one indicated in Example 3-3. This single expression returns all the three "Description" elements in our example order document at the same time. When multiple elements meet the same criteria, XPath returns the set of all matching elements. However, you may not want all the elements. Consider the previously discussed scenario of accessing only the "Address" element with an "addType" attribute of either "ship" or "billship." Example 3-5 shows the appropriate XPath expression.

Example 3-5

 //Address[@addType="ship" or @addType="billship"] 

The double slash first selects all the "Address" elements. Then the qualification of these elements appears in brackets. But instead of an index corresponding to the document order of the element, Example 3-5 specifies a more dynamic constraint based on the "addType" attribute. Because a billing address could have an "addType" value of either "bill" or "billship," the expression concatenates the two possibilities with "or."

All of these XPath expressions return document nodes. When working with text documents, authors may want to specify a point that marks the beginning or end of a text selection. This feature is what we need to solve the problem of selecting the last four digits of a credit card number. The first step in solving this problem is specifying a point just after the 12th digit of a 16-digit number. Example 3-6 shows how to use XPointer for this purpose. Note that the syntax of any final XPointer Recommendation may differ somewhat from that used here.

Example 3-6

 xpointer (//Payment/Card/Number /child::text()/point()[position()=12]) 

The first part of this XPointer expression uses XPath syntax to narrow the scope of the match to the "Number" element of the "Card" element of any "Payment" element within the document. This limitation prevents an unintended match within a document that also uses a "Number" element as part of something other than a credit card number. This expression then goes beyond XPath, using the "child:text" clause to narrow further the scope of the match to the internal text content of a node. The "position()=12" clause places the point just after the 12th character of the text content.

By combining two points to specify a range, authors can use XPointer to achieve the precision necessary to indicate only the last four digits of a credit card number. Example 3-7 specifies two separate points, using the syntax from Example 3-6. The first point is just after the 12th character. The second point is just after the 16th character. Connecting these two points with the "range-to" function returns all text between the points ”in this case, the 13th, 14th, 15th, and 16th digits of the credit card number. Points can even cross node boundaries. This capability is useful for quoting passages in text documents. It is possible to grab all of the text from the 2nd paragraph of the 3rd chapter to the 5th paragraph of the 4th chapter using a similar XPointer expression.

Example 3-7

 xpointer(//Payment/Card/Number /child::text()/point()[position()=12] /range-to(//Payment/Card/Number /child::text()/point()[position()=16])) 

While XPointer provides precision for specifying matches within a single document, XQuery provides precision for specifying matches across multiple documents. The basic framework of XQuery is the "FLWR" expression ”FOR, LET, WHERE, RETURN ”although the final syntax may evolve from that used in Working Draft. We can use this framework to create a mailing list to notify customers of an upgrade opportunity from a hypothetical collection of documents that uses the "Order" DTD from Example 2-9.

Example 3-8 wraps the query expression in a "MailingList" element so that the query engine can generate a well- formed XML results document. The "FOR" clause specifies the scope of the query as "Order" documents within the "Orders" collection at Foo Company and assigns these document to the "o" variable. The "WHERE" clause constrains the matched documents to those where a "Description" element has the value "FooBar Version 5."

The "RETURN" clause indicates that the results document should wrap individually returned records in an "Addressee" element. It filters the returned information down to all child elements of billing address elements using an XPath expression like that of Example 3-5. Example 3-8 doesn't use a "LET" clause. This clause is useful for sophisticated queries where developers need to manipulate intermediate results to generate the final results.

Example 3-8

 <MailingList>  FOR $o IN document   ("http://www.foocompany.com")/Orders/Order   WHERE $o//Description = "FooBar Version 5"   RETURN   <Addressee>    $o//Address[@addType="ship" or    @addType="billship"]/*   </Addressee> </MailingList> 

At the time of this writing, the W3C was rapidly revising XPath and XQuery. XPath 2.0 and XQuery 1.0 were both Working Drafts. The purpose of these revisions is to resolve two primary issues. The first is to ensure compatibility with the XML Schema data model. It would be nice if XPath expressions could use constraints based on the basic data types defined by XML Schema. The second is to ensure compatibility between XPath and XQuery. It makes a great deal of sense to use XPath as the syntax for expressing many of the XQuery parameters. However, there are several subtle issues concerning XPath 1.0 that inhibit this compatibility. XPath 2.0 will resolve these issues to the point where XQuery 1.0 will include XPath 2.0 as a subset.

Practical Usage

Developers and authors have been using XPath for some time, but they are just beginning to experiment with XPointer and XQuery. XPointer and XQuery incorporate much of the XPath syntax. These factors make XPath the most important feature standard for accessing parts of documents. Managers should ensure that all developers working on XML projects are comfortable using XPath.

The biggest use for XPath is as a means of passing references to specific document nodes among software components . Programming languages make extensive use of pointers or handles to refer to specific locations within internal data structures. XML documents are essentially external data structures. Pointers or handles to XML document data would be meaningless to another component that did not share the same memory space. XPath provides the appropriate external reference mechanism. If one component wants to tell another to use a particular address for sending an invoice, it passes the document location and the appropriate XPath.

XPointer offers advantages over XPath primarily for hypertext applications. In these applications, much of the content is text-oriented rather than data-oriented, and documents naturally tend to refer to content within other documents. This applicability extends to boundary situations where a text-oriented document refers to parts of a data-oriented document generated by an application. XPointer is closely related to XLink, the feature standard focused on XML hypertext links and discussed later. The motivation for XPointer stems from the common need for such links to specify a particular place within a document rather than the entire document.

While XPath and XPointer address the issue of accessing data within an individual document, XQuery handles accessing data across a collection of documents. Although the standard was still under development at the time of this writing, the motivation for it is clear. Any time an application generates a significant number of XML documents containing meaningful data, it's only a matter of time before someone wants to search those documents for a specific piece of information. In cases where XML is only one representation of data actually managed within a relational database model, existing query languages and tools will be adequate. But if XML is the native representation of the data or is the common representation provided by otherwise heterogeneous data models, XQuery should see extensive use.