We had some good fun testing our controllers and database application using Nose, and we've taken advantage of testutil. So far, all our tests took place directly on the server.
The good news is that Python has an easy-to-use solution to all these problems. Mechanize makes browser simulation testing in Python so easy, writing tests is almost fun.
Mechanize can impersonate a specific user agent and thus enable you to test the code generated for each browser. It operates just like any browser through HTTP, so if it works in Mechanize, it will work in the browser. Of course, Mechanize doesn't simulate browser rendering, so you still need to look at your pages in the various browsers you support and see if they look okay!
If you decide to use Mechanize, you'll need to install it, because it isn't part of TurboGears. We suggest you use the following command:
easy_install -Z mechanize
If you drop the -Z, Mechanize will be installed as a zipped egg and you will not be able to debug it or browse its code. Mechanize depends on ClientForm, which allows you to process forms easily.
Let's see what Mechanize can do for you. Run the bookmarker application and execute the following code:
from mechanize import Browser b = Browser() b.open("http://localhost:8080/") # Just get all bookmarks r = b.follow_link(text=r"All Bookmarks") assert b.viewing_html() print r.geturl() print r.info() # headers print r.read() # body
You should see some metadata followed by the actual HTML page returned from the "All Bookmarks" link. Mechanize a wrapper around urllib2. If you are familiar with that module, you know that you can use most of its methods from within a Mechanize script.
The Browser class is a glorified urllib2.OpenerDirector wrapped in some thick layers of Mechanize code. This means that the urllib2 Request and Response objects are there (not to be confused with the CherryPy Request and Response objects), and that the entire API is very urllib2 like. Having said that, you'll likely run into some trouble if you mix plain urllib2 objects with mechanize objects.
Mechanize provides very fine-grained ways to navigate your application using regular expressions to follow links and analyze the responses. You can reload the current page, go back, set and control proxies, handle cookies, and logging levels of specific operations like redirects and refreshes.
Let's see how Mechanize can handle forms. The following test function is trying to add a new bookmark using the bookmark/add form. First, it verifies that the All bookmarks page doesn't contain the word 'Mechanize'. Then it navigates to the bookmark/add page, gets the form object, populates the object with proper value, and submits it. Then it verifies that 'Mechanize' appears in the new response and even validates that the actual URL appears in the links. Note that we use a regular expression to locate the link, which is one of the amenities that Mechanize offers. The code feels very natural and Python-like.
from mechanize import Browser import ClientForm def test_add_link(): # Just get all bookmarks b = Browser() b.open("http://localhost:8080/") # follow link to All bookmarks page r = b.follow_link(text=r"All Bookmarks") assert b.viewing_html() assert not 'Mechanize' in r.read() # Open the add bookmark page b.open("http://localhost:8080/bookmark/add") # get the form forms = ClientForm.ParseResponse(b.response(), backwards_compat=False) form = forms # Fill the form form["bookmarkName"] = "Mechanize" form["link"] = "http://wwwsearch.sourceforge.net/" form["description"] = "The best browser simulator ever!!!" # Submit the form r = b.open(form.click()) # Make sure it's in there assert 'Mechanize' in r.read() links = list(b.links(url_regex=".*wwwsearch.*")) assert len(links) == 1 assert links.url == 'http://wwwsearch.sourceforge.net/'
Mechanize and its ClientForm sidekick have a lot to offer, but unfortunately the documentation (on the site and on the web) is somewhat scant. If you really want to utilize Mechanize be prepared to dig into the source.
Another option to make writing mechanized-based testing easier is the fantastic twill library, which offers an easy-to-use layer on top of mechanizing to write tests from Python, or in a domain specific browser emulation language. Twill is well documented, and available at: http://twill.idyll.org/.