Listing the Newsgroups on a Server

This lab demonstrates how to use the twisted.news.nntp module to perform one of the most common tasks in Usenet: retrieving a list of the available newsgroups on a server.

9.1.1. How Do I Do That?

Use a subclass of twisted.news.nntp.NNTPClient as your Protocol. NNTPClient is one of the older protocol implementations in Twisted, and it's starting to show its age. It was developed before the best practices for writing Twisted client protocols had really been understood. In particular, you'll notice that it doesn't make use of Deferreds. Instead, NNTPClient has an API that provides methods for sending commands to the server, and you write handler methods for receiving the server's reply.

To get the list of newsgroups from the server, call the fetchGroups method of NNTPClient. Use the gotAllGroups method as shown in Example 9-1 to handle the result, and getAllGroupsFailed to handle an error.

Example 9-1. nntplist.py

from twisted.news import nntp

from twisted.internet import protocol, defer



class NNTPGroupListerProtocol(nntp.NNTPClient):



 def connectionMade(self):

 nntp.NNTPClient.connectionMade(self)

 self.fetchGroups( )



 def gotAllGroups(self, groups):

 groupnames = [groupInfo[0] for groupInfo in groups]

 self.factory.deferred.callback(groupnames)

 self.quit( )



 def getAllGroupsFailed(self, error):

 self.factory.deferred.errback(error)



 def connectionLost(self, error):

 if not self.factory.deferred.called:

 self.factory.deferred.errback(error)



class NNTPGroupListerFactory(protocol.ClientFactory):

 protocol = NNTPGroupListerProtocol



 def _ _init_ _(self):

 self.deferred = defer.Deferred( )



if __name__ == "_ _main_ _":

 from twisted.internet import reactor



 def printGroups(groups):

 for group in groups:

 print group

 reactor.stop( )



 def handleError(error):

 print >> sys.stderr, error.getErrorMessage( )

 reactor.stop( )



 factory = NNTPGroupListerFactory( )

 factory.deferred.addCallback(printGroups).addErrback(handleError)



 reactor.connectTCP('news-server.maine.rr.com', 119, factory)

 reactor.run( )

Run nntplist.py with the hostname of an NNTP server as the only argument. It will download and print the list of available newsgroups:

 $ python nntplist.py freetext.usenetserver.com

 aaaustin.jobs

 aacc.general

 aapg.general

 ab.agriculture

 ...

 zipworld.help

 zoom.cheesedog

 zoomnet.general

 zz.unity.chat

 zz.unity.netlink

 zzz.3

 

9.1.2. How Does That Work?

NNTP doesn't require authentication, so NNTPGroupListerProtocol only has to call one method: self.fetchGroups( ). When the server responds with a list of newsgroups, the gotAllGroups method is called, with the newsgroup list as an argument. This list is a set of tuples in the form (groupName, articleCount). NNTPGroupListerProtocol's gotAllGroups method extracts the group names and uses them to call back the Factory's Deferred object.

The getAllGroupsFailed method is the error handler for fetchGroups. If an error occurs, it will be passed to the factory's Deferred.