Sam Spade, Black Widow, and Teleport Pro

Sam Spade, Black Widow, and Teleport Pro

The wget retriever and grep are powerful tools for automated source sifting. At times, we also use GUI-driven tools such as Sam Spade, Black Widow, and Teleport Pro for crawling and analyzing Web sites. The main drawback of wget is that it isn't multithreaded. Web crawlers such as Black Widow from SoftByteLabs (http://www.softbytelabs.com) and Teleport Pro by Tennyson Maxwell (http://www.tenmax.com) are excellent multithreaded crawler programs that run on Windows. However, neither have the capability to search mirrored HTML code. For that, we use the Windows' findstr utility.

Sam Spade v1.14 from Blighty Design, Inc. (http://www.samspade.org) features a Web crawler tool along with options to search for patterns and elements such as e-mail addresses within downloaded HTML code. Figure 7-4 shows Sam Spade's Web crawler options.

Figure 7-4. Sam Spade Web crawler options

Figure 7-5 shows the output produced by Sam Spade: hyperlinks, e-mail addresses, and hidden fields extracted from http://www.acme-art.com.

Figure 7-5. Sam Spade run against www.acme-art.com