Section 18.2. Zope: A Web Application Framework

18.2. Zope: A Web Application Framework

Zope is an open source web application server and toolkit, written in and customizable with Python. It is a server-side technology that allows web designers to implement sites and applications by publishing Python object hierarchies on the Web. With Zope, programmers can focus on writing objects and can let Zope handle most of the underlying Hypertext Transfer Protocol (HTTP) and Common Gateway Interface (CGI) details. In short, Zope is a popular and easy way to build enterprise-level web sites, whose basic structure is object oriented and whose dynamic content is scripted in Python.

Sometimes compared to commercial web toolkits such as ColdFusion, Zope is made freely available over the Internet and enjoys a large and very active development community. Indeed, many attendees at recent Python conferences were attracted by Zope, which has its own conference tracks. The use of Zope has spread so quickly that many Pythonistas have looked to it as a Python killer applicationa system so good that it naturally pushes Python into the development spotlight.^[*]

^[*] Over the years, observers have also pointed to other systems as possible Python "killer applications," including Grail, Python's COM support on Windows, and Jython. I hope they're all correct, and I fully expect new killers to arise after this edition is published. At the time of this writing, the IronPython implementation for .NET, Python's role in systems such as Google and BitTorrent, and other developments in the Python world seem to be helping drive the buzz too.

Although useful by itself, Zope is also the basis of the popular Plone content management systema way to build web sites that delegate content responsibilities to content producers, thus reducing the webmaster bottleneck. Under Plone, users may extend web site content using a workflow model. Plone is built on top of Zope and is something of a prepackaged and highly customizable Zope site (in fact, Plone sites are generally customized in the Zope ZMI user interface). Because Zope is in turn based on Python, both Plone and Zope are Python-based systems.

If you are interested in implementing more complex web sites than the form-based interactions we've seen in the preceding two chapters, you should investigate Zope.Once its learning curve is mastered, it can obviate many of the tasks that web scripters wrestle with on a daily basis. Zope offers a higher-level way of developing sites for the Web, above and beyond raw CGI scripting.

18.2.1. Zope Overview

Zope began life as a set of tools (part of which was named "Bobo") placed in the public domain by Digital Creations (now known as Zope Corporation). Since then, it has grown into a large system with many components, a growing body of add-ons (called "products" in Zope parlance), and a fairly steep learning curve. If you take the time to learn the Zope way of thinking about a web site, though, building and reusing site objects in Zope is quick and avoids much of the complexity in lower-level techniques such as CGI.

Due to the scope of this system, we can't do it any sort of justice in this book. See Zope- and Plone-specific texts for more on these systems. However, because Zope is a popular Python-based application, and because it is a prime example of Python use on the Internet, a quick overview is in order here.

18.2.1.1. Zope hierarchy model

The key to understanding Zope is its hierarchical site modelin a nutshell, Zope web sites are constructed as a hierarchy of objects, which reflect site content and acquire attributes from high objects in the web site tree. Every object in a Zope web site is ultimately a Python object, and Zope uses inheritance and acquisition (a containment relationship) to allow behavior to be shared among site objects.

The effect is much like the code reuse possible with Python's own class inheritance model. For instance, all pages in a web site tree can acquire and reuse common header and footer objects higher in the tree, to implement a common look-and-feel. Similarly, methods in the tree have implied context much like the "self" argument in Python classes, and they may be applied in the context of other tree objectsa directory lister, for instance, can be run on any folder in the tree.

Just as important, every object in a Zope web site tree may be addressed and viewed by its direct URL. Zope maps a URL path to the folder structure of the web site. Executable script code, for example, may be either invoked by other code in the web site or called directly through the Web by its URL.

18.2.1.2. Zope scripting

In Zope-based sites, Python code provides dynamic interaction in a variety of forms, including:

Expressions: Inline Python expressions, useful for trivial number and string manipulations, passing parameters to script and method calls, and so on
Scripts: Small function-like blocks of code designed for simple tasks that run in a limited secure context, useful for math and text processing, error checking, and so on
External methods: Modules with registered functions and arbitrary code, useful for full-blown programming tasks such as image processing, XML parsing, FTP, database, and so on
Products: Classes that extend Zope with new stateful and persistent objects that can be added to web sites, useful for address books, photo albums, and so on

As we'll see, Python code in a Zope web site may generally be invoked by URL or from other objects in the web site, including Zope's templating languages. The combination of Python code and templating objects provides for both logic and presentationthe presentation languages run Python code to process requests and create parts of the reply.

18.2.1.3. Zope components

Besides its scriptability, Zope features a variety of components, some of which are designed for enterprise-level sites and many of which would be difficult to implement from scratch in CGI scripting:

An object request broker that maps URLs to Python objects or code on the server (the ZPublisher ORB)
An over-the-web-site development paradigm, in which sites are designed and managed from any web browser (the ZMI user interface)
Two server-side templating languages, which are evaluated and expanded on the server, to render an HTML reply stream (DTML and ZPT/TAL)
An object-oriented database, for storing site content objects (ZODB, described in Chapter 19)
Enterprise-level tools such as transaction rollback and server load-balancing tools (ZEO, and more)

Some of its components underscore Zope's object-based model. For instance:

Zope Object Request Broker (ORB): At the heart of Zope, the ZPublisher ORB dispatches incoming HTTP requests to Python objects and returns results to the requestor, working as a perpetually running middleman between the HTTP CGI world and your Python objects. The Zope ORB is described further in the next section.
HTML document templates: Zope provides a simple way to define web pages as templates, with values automatically inserted from Python objects and calls. Templates allow an object's HTML representation to be defined independently of the object's implementation, and they allow for a strong separation of display format and business logic. For instance, values of attributes in a class instance object may be automatically plugged into a template's text by name. Template coders need not be Python coders, and vice versa.
Object database (ZODB): To record data persistently, Zope comes with a full object-oriented database system for storing Python objects, called ZODB. The Zope object database is based on the Python pickle serialization module that we'll meet in the next part of this book, but it adds support for transactions, write-through to disk on in-memory object attribute changes, concurrent access, and more. The database takes the form of a persistent dictionary of persistent objects where objects are stored and retrieved by key, much as they are with Python's standard shelve module. However, classes must subclass an imported Persistent superclass to take full advantage of the system. Zope automatically starts and commits transactions at the start and end of HTTP requests. Because ZODB is a separate component that is also useful in nonweb applications, we will study it in detail in the next part of this book.

Zope also includes a security model, the ZServer web server, the ZClasses system for development of components, and more. Zope ships its components integrated into a whole system, but many parts can be used on their own as well. For instance, Zope's ZODB object database can be used in arbitrary Python applications by itself.

18.2.2. Zope Object Publishing

If you're like me, the concept of publishing objects on the Web may be a bit vague at first glance, but it's fairly simple in Zope. The Zope ORB automatically maps URLs requested by HTTP into calls on Python objects. Consider the Python module and function in Example 18-1.

Example 18-1. PP3E\Internet\Other\Zope\messages.py

 "A Python module published on the Web by Zope" def greeting(size='brief', topic='zope'):     "a published Python function"     return 'A %s %s introduction' % (size, topic)

This is normal Python code, of course, and apart from its documentation, it says nothing about Zope, CGI, or the Internet at large. We may call the function it defines from the interactive prompt as usual:

 C:\...\PP3E\Internet\Other\Zope>python >>> import messages >>> messages.greeting( ) 'A brief zope introduction' >>> messages.greeting(size='short') 'A short zope introduction' >>> messages.greeting(size='tiny', topic='ORB') 'A tiny ORB introduction'

But if we place this module file in the appropriate directory on a server machine running Zope and register it to Zope as part of our site, it automatically becomes visible on the Web. That is, the function becomes a published objectit can be invoked through a URL, and its return value becomes a response page.

For instance, if our web site and Zope are installed on a server called myserver.net, and the module in Example 18-1 is placed in a Zope folder called "messages" at the top of our web site's object tree, the following URLs are equivalent to the three earlier calls:

 http://www.myserver.net/messages/greeting http://www.myserver.net/messages/greeting?size=short http://www.myserver.net/messages/greeting?size=tiny&topic=ORB

When our function is accessed as a URL over the Web this way, the Zope ORB performs two feats of magic:

The URL is automatically translated into a call to the Python function. The first part of the URL after the directory path (messages) names the Zope site folder (which happens to be the same as the module name but doesn't have to be); the second part (greeting) names a function or other callable object within that module; and any parameters after the ? become keyword arguments passed to the named function. URL query parameters are matched to argument names in the called Python object.
After the function runs, its return value automatically appears in a new page in your web browser. Zope does all the work of formatting the result as a valid HTTP response.

In other words, URLs in Zope become remote function calls, not just script invocations. The functions (and methods) called by accessing URLs are coded in Python and may live at arbitrary places on the Net. It's as if the Internet itself becomes Python namespaces, with one namespace per server and site.

Zope is a server-side technology based on objects, not text streams; the main advantage of this scheme is that the details of CGI input and output are handled by Zope, while programmers focus on writing domain objects, not on text generation. When our function is accessed with a URL, Zope automatically finds the referenced object, translates incoming parameters to function call arguments, runs the function, and uses its return value to generate an HTTP response. In general, a URL like:

 http://servername/folderpath/object1/object2/method?arg1=val1&arg2=val2

is mapped by the Zope ORB running on servername into a call to a Python object in a Python module file of the form:

 folderpath.object1.object2.method(arg1=val1, arg2=val2)

The return value is formatted into an HTML response page sent back to the client requestor (typically a browser). By using longer paths, programs can publish complete hierarchies of objects; Zope simply uses Python's generic object-access protocols to fetch objects along the path.

As usual, a URL like those listed here can appear as the text of a hyperlink, typed manually into a web browser, or used in an HTTP request generated by a program (e.g., using Python's urllib module in a client-side script). Parameters are listed at the end of these URLs directly, but if you post information to this URL with a form instead, it works the same way:

 <form action="http://www.myserver.net/messages/greeting" method=POST>     Size:  <input type=text name=size>     Topic: <input type=text name=topic value=zope>     <input type=submit> </form>

Here, the action tag references our function's URL again; when the user fills out this form and presses its Submit button, inputs from the form sent by the browser magically show up as arguments to the function again. These inputs are typed by the user, not hardcoded at the end of a URL, but our published function doesn't need to care. In fact, Zope recognizes a variety of parameter sources and translates them all into Python function or method arguments: form inputs, parameters at the end of URLs, HTTP headers and cookies, CGI environment variables, and more.

Although this example describes an external method in Zope, the same concepts apply, whether the referenced object is Python code in a module, or a method implemented within Zope using a page templating language. In fact, under Zope, every component becomes an object in the web site's object tree and can be addressed by direct URL or can be invoked from other program components in the tree.

18.2.3. A Zope External Method

As a more complete example, and to illustrate some of the last section's concepts, consider the module in Example 18-2.

Example 18-2. PP3E\Internet\Other\Zope\webtools.py

 """ fetch a web page or FTP file, escape it for embedding in HTML; functions here become Zope external methods and may be invoked by URLs or other Zope objects, but can be used outside Zope too """ import urllib, ftplib, StringIO, cgi def fetchWebPage(self, url, sizelimit=None):     """     fetch reply from a web page URL     will also work for ftp:// URLs, script parameters     """     site = urllib.urlopen(url)     text = site.read( )     if not sizelimit: sizelimit = len(text)     return cgi.escape(text[:sizelimit]) def fetchFtpFile(self, host, directory, file, userinfo=( ), sizelimit=None):     """     fetch a file from an FTP site     assume binary mode, anonymous"     """     buff = StringIO.StringIO( )     site = ftplib.FTP(host)     site.login(*userinfo)     site.cwd(directory)     site.retrbinary('RETR ' + file, buff.write)     site.quit( )     text = buff.getvalue( )     if not sizelimit: sizelimit = len(text)     return cgi.escape(text[:sizelimit]) def selftest( ):     X = '-'*40     import getpass     login = raw_input('user?'), getpass.getpass('pswd?')     print fetchWebPage(None, 'http://www.python.org', 193), X     print fetchFtpFile(None, 'home.rmi.net', '.', 'mytrain.html', login, 72), X     print fetchWebPage(None, 'http://www.rmi.net/~lutz/mytrain.html', 72), X if _ _name_ _ == '_ _main_ _': selftest( )

The self argument here, if included, can be used to access other objects in the Zope web site tree context (roughly, the URL parent). However, nothing else is Zope specific in this code. In fact, when run standalone without Zope, its self-test code fetches two web pages by HTTP, and one by FTP, using tools we met in Chapter 14:

 C:\...\PP3E\Internet\Other\Zope>webtools.py user?lutz pswd? &lt;!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"                       "http://www.w3.org/TR/html4/loose.dtd" &gt; &lt;?xml-stylesheet href="./css/ht2html.css" type="text/css"?&gt; &lt;html&gt; ---------------------------------------- &lt;HTML&gt; &lt;HEAD&gt; &lt;TITLE&gt;Mark Lutz's Python Training Page&lt;/TITLE&gt; &lt;/HEAD&gt; ---------------------------------------- &lt;HTML&gt; &lt;HEAD&gt; &lt;TITLE&gt;Mark Lutz's Python Training Page&lt;/TITLE&gt; &lt;/HEAD&gt; ----------------------------------------

Now, to make the code in Example 18-2 part of a Zope web site as an external method:

Create, copy, or move the module in the Zope Extensions directory. On Windows, put it in C:\Zope-Instance\Extensions.
Add its functions as "External Method" objects to your web site in the Zope ZMI (e.g., add it to the "/" root folder to make it visible across the entire site). To add both functions in the module, add two external methods.

Once added in the ZMI interface, the functions are external method objects in the web site tree and will be acquired (roughly, inherited) by objects lower in the tree, as well as by paths in URLs that name the methods to operate on the path context. The end result will be that the two functions in Example 18-2 will become callable through the Web via URLs and from other Zope objects such as DTML template language code and other Python code.

Figure 18-1 shows one of the two functions being added in the Zope ZMIthe web-based interface used to build sites.

Figure 18-1. The Zope ZMI

The Zope site tree built in the ZMI is separate from the filesystem where the external method's module lives; here, we're adding the method to the root of the Zope site tree (the "/" folder). In Zope, your entire site is designed and maintained in the ZMI interface, and every object added in the ZMI becomes a persistent Python object in the ZODB database used to store your site. However, some components, such as external method module files, also live on the filesystem; as such, they have access to the machine at large.

18.2.3.1. Calling through the Web

Once added to the site tree, your methods are callable through the Web, using URLs that name the Zope server's hostname and port, any nested folder paths, the name of the external method as registered to Zope in the ZMI, and URL query parameters to provide inputs. Here is a URL that runs the web page fetch function in the module directly; Zope listens for HTTP requests on port number 8080 by default and is running on the local machine ("localhost") here:

 http://localhost:8080/fetchWebPage?url=http://www.rmi.net/~lutz

As mentioned, Zope uses information entered in the ZMI to map URLs that reference external methods of the form:

 server/method?arg1=va11&arg2=val2

into calls to Python functions in Python modules on the server of the following form:

 method(arg1=val1, arg2=val2)

In our example, the ZPublisher ORB matches request inputs to method parameters by name, and a call of this form is invoked for the localhost URL:

 webtools.fetchWebPage(url='http://www.rmi.net/~lutz')

Because this function returns raw text, Zope automatically renders it in the reply page stream (default reply formatting uses the Python str function). For example, Figure 18-2 shows the reply page returned by Zope for the Python home page, using the following URL in a web browser's address field (technically, the url parameter's value string should probably be escaped with urllib.quote_plus, but it works in all browsers tested as is):

 http://localhost:8080/fetchWebPage?url=http://www.python.org

Figure 18-2. Python home page fetched by a Zope external method

The HTML is escaped in the reply in Figure 18-2 because it is not wrapped in enclosing HTML yet; it is taken to be a string when fetched from the method directly. To make this display nicely, we need to move on to the next section.

18.2.3.2. Calling from other objects

Besides such direct URLs, Python external methods can also be referenced and called from other types of Zope objects, including Python scripts and DTML templating language code. When referenced, Zope finds the method object by acquisition (web site tree search); calls the Python function in the module file, passing in any arguments; and renders and inserts the returned result into the HTML reply stream.

For instance, the following Zope Python script, fetchscript, is a Script object added in the ZMI to the site's /scripts101 folder (it can also be uploaded to the ZMI from an external file). The script becomes a persistent object in the ZODB database used by Zope; it is not stored in the Extensions directory in the filesystem. Assuming this is stored lower in the site tree than the external method, when run, it locates and invokes the code in Example 18-2:

 # called from DTML or URL, calls external method # gets external method in "/" by acquisition context # uses FTP, returned string inserted into HTML reply site = 'home.rmi.net' directory = '.' login = ('lutz', 'XXXXXXXX') reply = context.fetchFtpFile(context, site, directory, 'mytrain.html', login,72) return reply

Zope Python scripts are small bits of Python code, designed for running calculations that are too complex for templating languages such as DTML, but are not complex enough to warrant an external method or other construct. Scripts generally perform simple numeric or string manipulations. Unlike external methods, scripts run in a limited secure environment and are stored in the Zope site tree. In scripts, the context variable gives access to the Zope acquisition context in which the script is being run, and other variables give access to request inputs and reply output interfaces.

Similarly, the following DTML templating language method object, named fetchdtml and created in the same /scripts101 ZMI web site folder, invokes both the external method directly and the script of the prior listing. Both the script and the DTML objects themselves become addressable by direct URL or by other objects in the web site tree.

 <dtml-var standard_html_header> <h2>External Method call (urllib)</h2> <pre> <dtml-var expr="fetchWebPage('http://www.python.org')[:277]"> </pre> <h2>Python script to External Method call (ftplib)</h2> <pre> <dtml-var fetchscript> </pre>

DTLM combines normal HTML with DTML tags that are evaluated and expanded on the server by Zope when the enclosing page is fetched. The results of DTML tags are inserted into the HTML reply stream. The dtml-var tag, for instance, can name inline Python code to be run (expr=) in the context of the web site tree, or name another object to be looked up in the tree and calledthe expression or object's result text is rendered and inserted into the reply stream HTML, replacing the entire dtml-var tag.

The object called from a dtml-var tag can be another DTML templating language object, a Python script or external method object, or other object types such as images. For example, the standard_html_header in this code references another DTML method object higher in the object tree, which in turn references an image object in the tree; by listing this in each page lower in the tree, it provides a common page header.

Figure 18-3 captures the reply generated when we visit the DTML code in a web browserthe original Python external method is run twice along the way. This page is addressed by the following URL; replace the last component of this URL with fetchscript to access the Python script by direct URL (it is also run by the DTML method):

 http://localhost:8080/scripts101/fetchdtml

Figure 18-3. Running DTML code that calls Python methods

In a sense, DTML embeds Python in HTMLit runs Python code in response to tags embedded in the reply page. This is essentially the opposite of the CGI scripts we met earlier which embed HTML in Python, and it is similar to the ASP and PSP systems we'll meet later in this chapter.

More important, DTML, as well as Zope's other templating language, ZPT (TAL), encourages separation of presentation and business logic. DTML presents the results of Python method and script invocations in HTML, but it doesn't know about their operation. The Python code of the script and external method objects referenced by DTML implements more complex programming tasks, but it doesn't know about display formatting of the context in which it may be used. Where appropriate, the display and logic components can be implemented by different specialists.

18.2.4. A Simple Zope Interactive Web Site

As a final example, consider the following Zope-based web site. It consists of three Zope objects, all created and edited in the ZMI: an input page, a reply page, and a Python script used for calculations. Its input page form references the reply page object, and the reply page calls a Python script from a DTML expression.

The input page is a DTML method object, created and stored as the Zope tree object /scripts101/salaryInput in the ZMI. Its form input parameters are automatically converted to float and integer objects by Zope:

 <dtml-var standard_html_header> <form action="salaryResult" method=POST> <h2>Enter job data:</h2> <table>   <tr><td>Hours worked: <td><input name="hours:float"><br>   <tr><td>Pay per hour: <td><input name="rate:int"><br> </table><br> <input type="submit" value="Compute"><br> </form> <dtml-var standard_html_footer>

The reply page, the web tree object /scripts101/salaryResult, is also a Zope DTML method object, invoked by the salaryInput page:

 <dtml-var standard_html_header> <p>Your pay this week:     <b><dtml-var expr="calculateSalary(hours, rate)"> </p> <dtml-var standard_html_footer>

Finally, the Python script object, added as /scripts101/calculateSalary in the ZMI, performs numeric calculations required by the reply page, which are outside the scope of DTML display code. Input parameters to this script come automatically from DTML namespaces; their names (hours, rate) may be listed in the ZMI when the script is created or by special comments at the start of the script's code. When run, this script's return value is automatically rendered by Zope and inserted in the HTML reply stream, replacing the dtml-var tag that calls the script by name.

 import math if hours < 0:     hours = 0 else:     hours = math.floor(hours) return hours * rate

As before, this fosters a separation of presentation and business logic: the DTML salaryResult presents the result of Python calculateSalary, but the DTML code doesn't know about salary calculation and the Python code doesn't know about presentation. Ideally, the two parts can be worked on independently, by people with different skill sets.

This separation is especially striking when compared with classic CGI scripts, which embed and mix HTML reply code with Python codein the Zope model, salaryResult display is independent of the Python calculateSalary logic. In practice, more complex pages may require additional formatting logic in the templating language code (e.g., loops and tests), but the general separation still applies.

Figure 18-4 captures this site's input page (it can also be displayed with the View tab in the ZMI) at the URL http://localhost:8080/scripts101/salaryInput.

Figure 18-4. Input page

Figure 18-5 shows the reply page returned when the input page is submitted. The reply page reflects the DTML code that presents the result returned by the Python script.

Figure 18-5. Reply page

We can also call the calculateSalary Python script directly by its URL, though we have to take care to convert the input arguments to their expected datatypes by using type codes after their namesZope uses these to perform from-string conversions before the values are passed into the called object. We use these in the input fields of salaryInput as well. Alternatively, we could restructure the script to convert from strings to the expected types itself by using the REQUEST inputs object rather than declared parameters. As is, the following URL produces a page that displays just the text "5200.0"the default str rendering of the returned Python floating-point number:

 http://localhost:8080/scripts101/calculateSalary?hours:float=65&rate:int=80

The salaryResult DTML page object can be called directly by a similar URL (replace the Python script's name), though the reply is a complete web page produced by the DTML code.

In fact, as seen in Figure 18-6, the Python script can also be tested within the ZMI itselfclick the test tab, and input the parameters manually. Objects can be tested this way in the ZMI, without having to type the corresponding URL in another browser window.

Figure 18-6. Testing scripts in the ZMI

As you can probably tell, in this introduction we're just scratching the surface of what Zope can do. For instance, we haven't introduced the other templating language in Zope, Zope Page Templates (ZPT), coded in Template Attribute Language (TAL). ZPT is an alternative way to describe presentation based on attributes of normal HTML tags, rather than embedded DTML tags. As such, ZPT code may be more easily handled by some HTML editors when edited outside the context of Zope.

Moreover, published functions and methods can use the Zope object database to save state permanently; there are more advanced Python constructs in Zope, including Zope products; URLs can provide method context using reference paths in ways we have not mentioned here; and Zope provides additional tools such as debugging support, precoded HTTP servers for use with the ORB, and finer-grained control over responses to URL requestors.

For all things Zope, visit http://www.zope.org. There, you'll find up-to-date releases, as well as documentation ranging from tutorials to references to full-blown Zope example sites.

During the lifespan of the second edition of this book, Python creator Guido van Rossum and his PythonLabs team of core Python developers were located at the Zope Corporation, home of the Zope framework introduced here. As I write this third edition, Guido has just been hired by Google, but many of the original PythonLabs team members are still at Zope.