Convergence: XPath 2.0

During the twelve months after XSLT 1.0 and XPath 1.0 were published, it became increasingly clear that XQuery could not ignore them. All the major vendors were shipping implementations , and there was a high level of uptake by the user community. Unlike some other standards organizations, W3C likes to keep its specifications consistent and coherent : It does not like overlaps and duplications between different recommendations that offer multiple solutions to the same problem. So XQuery quickly aligned itself with XPath, adopting path expressions as part of the XQuery language that were strongly aligned with the XPath specification, though initially there were many differences in the detailed syntax and semantics (as well as many gaps, where the syntax and semantics were not fully defined).

For the management of W3C, however, a superficial similarity between the languages was not enough. If XQuery used the same syntax as XPath, but with different detailed semantics, there would be few benefits for users, indeed, there would be a great deal of confusion. However, XQuery could not easily take XPath 1.0 on board without changes, because there were fundamental differences in the data model and the type system.

At the same time, XML Schema was starting to make itself felt. XQuery and XML Schema have always had a close relationship and mutual dependency, of the same kind that data-description languages and data-manipulation languages have had since the earliest days of database technology in the mid 1960s. In the early days of XQuery development, the language had its own type system that was separate from that of XML Schema. But this clearly wasn't tenable, and the working group decided that XQuery would use XML Schema as its type system (despite the objections of language theorists like Phil Wadler, who argued cogently that it was technically incorrect to describe XML Schema as a type system).

There was also pressure, though less intense , for XSLT to align itself more closely with XML Schema. This move has aroused some fairly vocal opposition in some quarters , especially from those whose primary interest is in document processing. But many of the large corporations using XML increasingly see XML Schema as a key tool in the way they plan to manage integration of applications within and beyond the corporation, and XSLT as a key tool in handling the diversity and evolution of those applications. Although initially no one was quite sure what it meant in detail, some degree of integration of XSLT (and therefore XPath) with XML Schema was therefore seen as an important strategic goal.

The result of these pressures was the decision that the XQuery and XSL working groups would work together on the development of XPath 2.0, which would have a type system aligned with that of XML Schema, and that XQuery would be a superset of XPath 2.0. The two groups set up a joint task force to develop XPath 2.0. They decided that XSLT and XQuery meetings would in future be co-located, and they set up joint meetings to promote consistency between XSLT and XQuery in other areas, such as the formalization of the shared data model and the rules for tree construction.

As some members predicted , creation of the XPath 2.0 language was tough. There was a natural tendency for the XSLT representatives to fight for backwards compatibility with XPath 1.0, while the XQuery people argued that there were aspects of the XPath 1.0 specification that they could not live with. Gradually mechanisms were devised that encompassed both objectives, for example, by retaining the implicit type conversions of XPath 1.0 as a "fallback mode" that would be supported in XSLT but not in XQuery, and in some cases by providing new functions and operators that overlapped the old but had cleaner semantics. Some of the semantics of XPath 1.0 that had initially seemed unacceptable ”for example, the fact that the results of path expressions were always in document order ”were found to be acceptable after all, as XQuery people gradually became familiar with the strange properties of text markup structures. In some other areas, particularly where the XPath 1.0 specification had led to usability problems, compromises affecting backwards compatibility were made ”though generally only in areas where very few users were likely to be adversely affected. There was a great deal of debate over how much of the XQuery language should be included in the XPath subset, with some wanting to keep XPath extremely small, and others arguing for the inclusion of any feature that might be useful in an XSLT context.

The result is inevitably a compromise. XPath 2.0 is a substantially bigger language than XPath 1.0. Most of the growth is due to the definition of a vastly extended library of core functions, which is a good way of adding capability without increasing complexity. Aside from the growth in functions (and in operators that merely provide convenient syntax for an underlying function call), the syntax of the language has grown by a relatively modest 40 percent, and in fact some of the increased power (in particular, the generalization of path expressions) has been achieved by eliminating restrictions present in XPath 1.0, so it has actually made the language smaller. ^[4]

^[4] The number of productions in the XPath grammar has grown from 39 to 71, but only 16 of the new productions introduce basic syntactic constructs: The rest merely expand the range of binary operators available. The number of functions, however, has grown from 28 to more than 100.