Managing a Hierarchy of Resources

The paths in a web application usually imply a hierarchy of resources. For example, look at these URIs:

 http://example.com/people

 http://example.com/people/charles

 http://example.com/people/charles/contact

It's easy to see the hierarchy here. The page /people/charles is a child of /people, and the page /people/charles/contact is a child of /people/charles. Each page in the hierarchy is more specific: /people/charles is one specific person, and /people/charles/contact is one specific type of data (in this case, contact information) related to charles.

The default behavior for most web servers is to map request paths to a hierarchy of files and folders on disk. Each time a client requests the resource at a certain path, the web server tries to find a file at the corresponding path on disk, and responds with either the content of the file itself or (as in the case of a CGI script) the output created by executing the file. But in web applications, it can be artificially constraining to have to have a file on disk for every path that might be requested. For example, the data in your application might not be stored on disk, but in a relational database in another server. Or you might want to create resources on demand when they are requested. In cases like this, it's useful to be able to write your own logic for navigating a hierarchy of resources.

Writing your own logic for managing resources can also help you to manage security. Rather than opening up an entire directory to web access, you can selectively control which files are made available.

4.4.1. How Do I Do That?

The twisted.web.resource, twisted.web.static, and twisted.web.server modules provide classes for working with requests at a higher level than twisted.web.http.Resource, which you can use to set up a web server that combines several different kinds of resources into a logical hierarchy. Example 4-4 uses these classes to build an application for testing hexadecimal color codes. Request the resource /colors/hex, where hex is a hexadecimal color code, and you'll get a page with the background color #hex. Rather than trying to generate a page for every possible color in advance, this server creates resources on demand.

Example 4-4. resourcetree.py

from twisted.web import resource, static, server



class ColorPage(resource.Resource):

 def _ _init_ _(self, color):

 self.color = color



 def render(self, request):

 return """

 

Color: %s

This is #%s.

<a href="/color/">Back</a>

""" % (self.color, self.color, self.color) class ColorRoot(resource.Resource): def _ _init_ _(self): resource.Resource._ _init_ _(self) self.requestedColors = [] self.putChild('', ColorIndexPage(self.requestedColors)) def render(self, request): # redirect /color -> /color/ request.redirect(request.path + '/') return "Please use /colors/ instead." def getChild(self, path, request): if path not in self.requestedColors: self.requestedColors.append(path) return ColorPage(path) class ColorIndexPage(resource.Resource): def _ _init_ _(self, requestedColorsList): resource.Resource._ _init_ _(self) self.requestedColors = requestedColorsList def render(self, request): request.write("""Colors

Colors

To see a color, enter a url like <a href="/color/ff0000">/color/ff0000</a>.
Colors viewed so far:

• """) for color in self.requestedColors: request.write( "
• <a href="%s">%s</a>
• " % ( color, color, color)) request.write("""

""") return "" class HomePage(resource.Resource): def render(self, request): return """Colors

Colors Demo

What's here:

• <a href="/color">Color viewer</a>

""" if __name__ == "_ _main_ _": from twisted.internet import reactor root = resource.Resource( ) root.putChild('', HomePage( )) root.putChild('color', ColorRoot( )) root.putChild('styles.css', static.File('styles.css')) site = server.Site(root) reactor.listenTCP(8000, site) reactor.run( )

Example 4-4 requires one static file, a CSS stylesheet. Create a file in the same directory as resourcetree.py called styles.css, with the content shown in Example 4-5.

Example 4-5. styles.css

body {

 font-family: Georgia, Times, serif;

 font-size: 11pt;

}



h1 {

 margin: 10px 0;

 padding: 5px;

 background-color: black;

 color: white;

}



a {

 font-family: monospace;

}



p {

 padding: 10px;

}

 

Run resourcetree.py to start a web server on port 8000. Here's a complete list of resources that the server provides:

/	Home page
/css	Virtual resource for holding CSS stylesheets
/css/styles.css	The static file styles.css
/colors/	Index of colors people have viewed so far
/colors/hexcolor	A page with the background color #hexcolor

 

Try going to the URI http://localhost:8000/colors/00abef, and you'll see the page in Figure 4-6 with the background color #00abef (which may not look like much printed in monochrome, but in real life is a bright shade of blue).

Figure 4-6. Viewing a hexadecimal color

 

Feel free to try other colors as well. You can also go to the page http://localhost:8000/colors/, shown in Figure 4-7, to see a list of the colors you've viewed so far.

Figure 4-7. The /colors/ page showing a list of viewed colors

 

4.4.2. How Does That Work?

Example 4-4 introduces several new classes from the twisted.web package: resource.Resource, static.File, and server.Site. Each resource.Resource object does two things. First, it defines how requests for that resource should be handled. Second, it defines which Resource objects should be used for requests for child resources resources whose path puts them below this resource in the hierarchy.

For example, take a look at the class ColorRoot in Example 4-4. An instance of this class will later be inserted into the resource hierarchy at the path /colors. When initialized, ColorRoot uses the putChild method to insert a ColorIndexPage Resource as the child resource ''. What does that mean? It means that requests for /colors/ (the path of ColorRoot plus a trailing slash) should be handled by the ColorIndexPage object.

You might think of them as being equivalent, but the URIs http://example.com/stuff and http://example.com/stuff/ (note the trailing slash) are not the same. They are different URIs identifying different resources. Browsers will expand relative links differently depending on whether the trailing slash is part of the URI. In the first example, a link to "otherpage" will expand to http://example.com/otherpage; in the second example, it will expand to http://example.com/stuff/otherpage.

If you're not explicit in your server code, this problem can come back to bite you. It's a good idea to decide whether you want to have trailing slashes in your URIs, and redirect requests from one form to the other. The Resource class makes this easy to do. If you set the attribute addSlash to true, a Resource will automatically add a trailing slash to any requests that don't already have them and redirect those requests to the updated URI.

 

The render method defines what happens after a Resource has been found that matches the path of a request. Resource.render works basically the same way as the request handler methods in Example 4-3: it takes a Request object as its only argument, and is responsible for handing the request and sending a response to the client. Resource.render has a few caveats that you'll need to keep in mind, however. First, it expects you to return a string. This is a useful shortcut in many cases: you can just return the data you want to send as the body of the response, and the Resource will send it to the client and end the response. But even when you choose to use request.write to write the response body yourself, render still expects you to return a string. You can return an empty string to make it happy without adding anything to the response.

At times, you might want to start a deferred operation inside a render method. In this case, you won't be ready to write the response until your Deferred calls back. You might wonder, "Can I just return a Deferred that calls back with a string?" Well, sadly, you can't. (This is one of many deficiencies in the Resource object; see the note at the end of this lab for a discussion of why you shouldn't use the classes discussed here for major web development projects.) Instead, you return the magic value twisted.web.server.NOT_DONE_YET, which tells the Resource that you've started something asynchronous and aren't done with this request yet, so it shouldn't call request.finish( ). Then you can call request.finish( ) yourself later after you're done writing the response. (See Example 4-5 for an example of this technique.)

The ColorRoot Resource will be used to render requests for the path /colors. In reality, though, ColorRoot is just a container for child resources. ColorRoot.render calls request.redirect, a helper function that sets the HTTP status code of the response to 302 ("Moved Temporarily") and writes a Location: header directing the client to request a page from another location, in this case /colors/ (with a trailing slash). Note that even though it's told the client to go somewhere else, render still has to return a string.

Resource offers an alternative to the render method. You can write separate methods to handle different HTTP methods: render_GET, render_POST, and so on. This approach is discussed in detail in Example 5-1 in Chapter 5.

 

ColorRoot has one more method, getChild. Here the possibilities for doing interesting things with resource hierarchies start to expand. The getChild method is designed for dynamically managing child resources. A Resource's getChild method is called when the client has sent a request for a path beneath the Resource in the hierarchy, and no matching path has been registered using putChild. By default, getChild will send a 404 ("Not Found") response. But you can override it, as ColorRoot does. ColorRoot's getChild method takes the child path and uses it to initialize a ColorPage object. The ColorPage can then respond to the request, using the last part of the path as a hexadecimal color code.

The static.File class is a subclass of Resource that serves the contents of a file or directory on disk. Initialize a static.File object with a filename as an argument. Using static.File is better than loading files from disk yourself because static.File is smart about handling large files: it won't take up too much memory, and it won't cause your entire server process to become unresponsive while it reads data from disk. If you initialize a static.File with the path of a directory, it will serve all the files and subdirectories under that directory.

Even if you're not using server.Site and a tree of Resource objects to manage your web server, you can still use a static.File object to handle a request. You can use a temporary static.File to push the contents of a file as the response to a request like this:

 static.File('file.txt').render(request)

 

You can change the MIME type static.File uses when serving files by making changes to the contentTypes attribute. That attribute functions as a dictionary for the purpose of mapping file extensions (such as .png) to MIME types.

 

The server.Site class is a Factory that you initialize with a Resource object. It will handle HTTP requests by splitting the requested path into segments and then walking the tree of Resource objects to find the Resource that it should use to handle the request.