One of the best Web crawling engines for Windows, Teleport Pro is remarkably robust and fast. It can take a URL and pull down all the files on or related to a Web server in a multithreaded fashion. The product is simple to use and can copy an entire Web server to a local computer (all the client-side content anyway).
As shown in Figure 15-25, Teleport Pro has a clean, well-organized interface, allowing you to view the entire directory and file structure of the target Web site quickly and easily.
Of the Web crawlers available, including wget for UNIX, few are faster or easier to manage than Teleport Pro.
With a mirrored Web site, we can parse all the client-side code for a number of security vulnerabilities, including the following.
1. Inappropriate comments Comments may contain sensitive information, such as company department names or phone extensions. Even worse, the authors of this book have found usernames and passwords in the comments fields of a Web site.
2. Form identification Once an attacker knows all the form pages on a Web site, he can launch a variety of attacks, including denial of service attempts, password brute forcing, and input validation.
3. Script identification Once an attacker knows the pages with client-side script on them, she can launch a variety of attacks, including input validation.
4. Applet identification Once an attacker knows the pages with Java applets in them, he can attempt to decompile each applet, looking for sensitive information such as passwords.
Prohibiting a user from mirroring your entire Web site isn't a trivial task. That's why an attacker can use tools such as those discussed here to break into a Web site.