One potential weakness in the examples presented so far in this chapter is that there hasn't been a way to monitor a download in progress. Sure, it's nice that a Deferred will pass you the results of a page once it's completely downloaded, but sometimes what you really need is to keep an eye on the download as it's happening.
3.5.1. How Do I Do That?
Again, the utility functions provided by twisted.web.client don't give you quite enough control. Define a subclass of client.HTTPDownloader, the factory class used for downloading a web page to a file. By overriding a couple of methods, you can keep track of a download in progress. The webdownload.py script in Example 3-6 shows how.
Example 3-6. webdownload.py
from twisted.web import client class HTTPProgressDownloader(client.HTTPDownloader): def gotHeaders(self, headers): if self.status == '200': # page data is on the way if headers.has_key('content-length'): self.totalLength = int(headers['content-length']) else: self.totalLength = 0 self.currentLength = 0.0 print '' return client.HTTPDownloader.gotHeaders(self, headers) def pagePart(self, data): if self.status == '200': self.currentLength += len(data) if self.totalLength: percent = "%i%%" % ( (self.currentLength/self.totalLength)*100) else: percent = '%dK' % (self.currentLength/1000) print "