 
Use TSaxon, a variant of Saxon, and TagSoup to help transform HTML. Stylesheets written in XSLT are the standard method of taking XML documents in one format and transforming them into HTML, XML documents in a different format, XHTML, or plain-text documents. There are many XSLT processors. Michael Kay's Saxon Version 6.5.3 (http://saxon.sourceforge.net/#F6.5.3) is a particularly mature and successful implementation for XSLT 1.0 and XPath 1.0. It is packaged as a Java JAR file called saxon.jar. You can download this JAR with the 6.5.3 distribution from the Saxon site on Sourceforge. Now suppose, for example, that we want to extract just the header elements (h1, h2, h3, etc.) from an XHTML document and display them as progressively indented plain text (i.e., each h1 element is unindented, each h2 element is indented by a single space, h2 by two spaces, etc.). The XSLT stylesheet outline.xsl does exactly what we want. It specifies an output method of text, and matches the h1 through h6 elements in the XHTML input, taking the content of each one and prepending the correct number of spaces. The textual content of other elements is suppressed. The following command, executed in your working directory, will process outline.xsl and the XHTML document outline.xhtml using Saxon and will display the resulting indented plain text: java -jar saxon.jar outline.xhtml outline.xsl
It so happens that outline.html contains only h1, h2, and h3 elements (borrowed from the XML specification) for the sake of brevity, but if you add other elements to it, you will see that the content of those elements is not displayed by this command. 3.20.1 Using TagSoup and TSaxon The great bulk of documents on the Web are not well-formed XHTML, but are HTML, so unmodified XSLT processors cannot be applied to them. For example, the HTML document outline.html is similar to outline.xhtml, but the end tags for the h1, h2, and h3 elements are missing. If you try to apply Saxon to outline.html, you will get no output except an error message saying the document is not well-formed. In this situation, you can use TSaxon instead of Saxon. TSaxon (http://www.ccil.org/~cowan/XML/tagsoup/tsaxon) is a variant of Saxon 6.5.3 with an HTML parser called TagSoup packaged in it. It behaves exactly like Saxon unless you give it the -H (HTML) switch, which causes it to substitute the TagSoup parser for the standard XML parser packaged with Saxon. You can download the latest version of TSaxon from the web site, or just use the version that came in the file archive. TSaxon's version of saxon.jar exists in the file archive under the subdirectory TSaxon. In the TSaxon subdirectory, issuing the following command will process outline.html using the TSaxon and the TagSoup parser and the stylesheet: java -jar saxon.jar -H outline.html outline.xsl
The reason that TSaxon is also packaged as the file saxon.jar is that it is completely backward-compatible and can be used as a drop-in replacement for Saxon. Saxon is stable, but TSaxon is not yet stable, because TagSoup (http://www.ccil.org/~cowan/XML/tagsoup) is still under active development. Nevertheless, TSaxon is still quite useful with even very messy HTML: less than one percent of a sample of over a thousand HTML files (downloaded from the Web at random) could not be correctly processed with the current version. John Cowan | 