10.1 An Automated Complaint System

The scenario we use for this example is that of a startup company, Joe's Toothpaste, Inc., which sells the latest in 100% organic, cruelty-free tofu-based toothpaste. Since there is only one employee, and that employee is quite busy shopping for the best tofu he can find, the tube doesn't say "For customer complaints or comments, call 1-800-TOFTOOT," but instead, says "If you have a complaint or wish to make a comment, visit our web site at www.toftoot.com ." The web site has all the usual glossy pictures and an area where the customer can enter a complaint or comment. This page looks like that in Figure 10.1.

Figure 10.1. What the customer finds at http://www.toftoot.com/comment.html

The key parts of the HTML that generated this page are displayed in the sidebar Excerpt From the HTML File. As this is not a book about CGI, HTML, or any of that, ^[1] we just assume that you know enough about these technologies to follow this discussion. The important parts of the HTML code in the sidebar are in bold: here's a brief description:

^[1] If you've never heard of these acronyms: CGI stands for the Common Gateway Interface and is a protocol for having web browsers call programs on web servers; HTML stands for HyperText Markup Language, which is the format that encodes web pages.

Excerpt From the HTML File

This is the important part of the code that generates the web page shown in Figure 10.1:

 <  FORM METHOD=POST ACTION="http://toftoot.com/cgi-bin/feedback.py"  > <UL><I>Please fill out the entire form:</I></UL> <CENTER><TABLE WIDTH="100%" > <TR><TD ALIGN=RIGHT WIDTH="20%">Name:</TD>     <TD><  INPUT TYPE=text NAME=name  SIZE=50 VALUE=""></TD></TR> <TR><TD ALIGN=RIGHT>Email Address:</TD>     <TD><  INPUT TYPE=text NAME=email  SIZE=50 VALUE=""></TD></TR> <TR><TD ALIGN=RIGHT>Mailing Address:</TD>     <TD><  INPUT TYPE=text NAME=address  SIZE=50 VALUE=""></TD></TR> <TR><TD ALIGN=RIGHT>Type of Message:</TD> <TD><  INPUT TYPE=radio NAME=type CHECKED VALUE=comment  >comment&nbsp;     <  INPUT TYPE=radio NAME=type VALUE=complaint  >complaint</TD></TR> <TR><TD ALIGN=RIGHT VALIGN=TOP>Enter the text in here:</TD>     <TD><  TEXTAREA NAME=text ROWS=5, COLS=50  VALUE=""  >         <  /TEXTAREA>  </TD></TR> <TR><TD></TD> <TD><  INPUT type=submit name=send value="Send the feedback!"  ></TD></TR> </TABLE></CENTER> </FORM>

The FORM line specifies what CGI program should be invoked when the form is submitted; specifically , the URL points to a script called feedback.py , which we'll cover in detail.
The INPUT tags indicate the names of the fields in the form ( name , address , email , and text , as well as type ). The values of those fields are whatever the user enters, except for the case of type , which takes either the value 'comment' or 'complaint' , depending on which radio button the user checked.
Finally, the INPUT TYPE=SUBMIT tag is for the submission button, which actually calls the CGI script.

We now get to the interesting part as far as Python is concerned : the processing of the request. Here is the entire feedback.py program:

 import cgi, os, sys, string def gush(data):  print "Content-type: text/html\n"  print "<h3>Thanks, %(name)s!</h3>" % vars(data)     print "Our customer's comments are always appreciated."     print "They drive our business directions, as well as"     print "help us with our karma."     print "<p>Thanks again for the feedback!<p>"     print "And feel free to enter more comments if you wish."     print "<p>"+10*"&nbsp;"+"--Joe." def whimper(data):  print "Content-type: text/html\n"  print "<h3>Sorry, %(name)s!</h3>" % vars(data)     print "We're very sorry to read that you had a complaint"     print "regarding our product__We'll read your comments"     print "carefully and will be in touch with you."     print "<p>Nevertheless, thanks for the feedback.<p>"     print "<p>"+10*"&nbsp;"+"--Joe." def bail():     print "<H3>Error filling out form</H3>"     print "Please fill in all the fields in the form.<P>"     print '<a href="http://localhost/comment.html">'     print 'Go back to the form</a>'     sys.exit() class FormData:     """ A repository for information gleaned from a CGI form """     def __init__(self, form):         for fieldname in self.fieldnames:             if not form.has_key(fieldname) or form[fieldname].value == "":                 bail()             else:                 setattr(self, fieldname, form[fieldname].value)          class FeedbackData(FormData):     """ A FormData generated by the comment.html form. """     fieldnames = ('name', 'address', 'email', 'type', 'text')     def __repr__(self):         return "%(type)s from %(name)s on %(time)s" % vars(self) DIRECTORY = r'C:\complaintdir' if __name__ == '__main__':     sys.stderr = sys.stdout  form = cgi.FieldStorage()  data = FeedbackData(form)     if data.type == 'comment':         gush(data)     else:         whimper(data)     # save the data to file     import tempfile, pickle, time     tempfile.tempdir = DIRECTORY     data.time = time.asctime(time.localtime(time.time()))     pickle.dump(data, open(tempfile.mktemp(), 'w'))

The output of this script clearly depends on the input, but the output with the form filled out with the parameters shown in Figure 10.1 is displayed in Figure 10.2.

Figure 10.2. What the user sees after hitting the "send the feedback" button

How does the feedback.py script work? There are a few aspects of the script common to all CGI programs, and those are highlighted in bold. To start, the first line of the program needs to refer to the Python executable. This is a requirement of the web server we're using here, and it might not apply in your case; even if it does, the specific location of your Python program is likely to be different from this. The second line includes import cgi , which, appropriately enough, loads a module called cgi that deals with the hard part of CGI, such as parsing the environment variables and handling escaped characters . (If you've never had to do these things by hand, consider yourself lucky.) The documentation for the cgi module describes a very straightforward and easy way to use it. For this example, however, mostly because we're going to build on it, the script is somewhat more complicated than strictly necessary.

Let's just go through the code in the if _ _ name __ == ' _ _ main __ ' block one statement at a time. ^[2] The first statement redirects the sys.stderr stream to whatever standard out is. This is done for debugging because the output of the stdout stream in a CGI program goes back to the web browser, and the stderr stream goes to the server's error log, which can be harder to read than simply looking at the web page. This way, if a runtime exception occurs, we can see it on the web page, as opposed to having to guess what it was. The second line is crucial and does all of the hard CGI work: it returns a dictionary-like object (called a FieldStorage object) whose keys are the names of the variables filled out in the form, and whose value can be obtained by asking for the value attribute of the entries in the FieldStorage object. Sounds complicated, but all it means is that for our form, the form object has keys 'name' , 'type' , 'email' , 'address' , and 'text' , and that to find out what the user entered in the Name field of the web form, we need to look at form['name'].value .

^[2] You'll remember that this if statement is true only when the program is run as a script, not when it's imported. CGI programs qualify as scripts, so the code in the if block runs when this program is called by the web server. We use it later as an imported script, so keep your eyes peeled.

The third line in the if block creates an instance of our user-defined class FeedbackData , passing it the form object as an argument. If you now look at the definition of the FeedbackData class, you'll see that it's a very simple subclass of FormData , which is also a user-defined class. All we've defined in the FeedbackData subclass is a class attribute fieldnames and a _ _ repr __ function used by the print statement, among others. Clearly, the _ _ init __ method of the FormData class must do something with the FieldStorage argument. Indeed, it looks at each of the field names defined in the fieldnames class attribute of the instance (that's what self.fieldnames refers to), and for each field name, checks to see that the FieldStorage object has a corresponding nonempty key. If it does, it sets an attribute with the same name as the field in the instance, giving it as value the text entered by the user. If it doesn't, it calls the bail function.

We'll get to what bail does in a minute, but first, let's walk through the usual case, when the user dutifully enters all of the required data. In those cases, FieldStorage has all of the keys ( 'name' , 'type' , etc.) which the FeedbackData class says it needs. The FormData class __ init __ method in turn sets attributes for each field name in the instance. So, when the data = FeedbackData(form) call returns, data is guaranteed to be an instance of FeedbackData , which is a subclass of FormData , and data has the attributes name , type , email , etc., with the corresponding values the user entered.

A similar effect could have been gotten with code like:

 form = cgi.FieldStorage() form_ok = 1 if not form.has_key("name") or form["name"].value == "":     form_ok = 0 else:     data_name = form["name"].value if not form.has_key("email") or form["email"].value == "":     form_ok = 0 else:     data_email = form["email"].value ...

but it should be clear that this kind of programming can get very tedious , repetitive, and error-prone (thanks to the curse of cut and paste). With our scheme, when Joe changes the set of field names in the web page, all we need to change is the fieldnames attribute of the FeedbackData class. Also, we can use the same FormData class in any other CGI script, and thus reuse code.

What if the user didn't enter all of the required fields? Either the FieldStorage dictionary will be missing a key, or its value will be the empty string. The FormData. __ init __ method then calls the bail function, which displays a polite error message and exits the script. Control never returns back to the main program, so there is no need to test the validity of the data variable; if we got something back from FeedbackData() , it's a valid instance.

With the data instance, we check to see if the feedback type was a comment, in which case we thank the user for their input. If the feedback type was a complaint, we apologize profusely and promise to get back in touch with them.

We now have a basic CGI infrastructure in place. To save the data to file is remarkably easy. First, we define the DIRECTORY variable outside the if test because we'll use it from another script that will import this one, so we wish it to be defined even if this script is not run as a program.

Stepping through the last few lines of feedback.py :

Import the tempfile , pickle , and time modules. The tempfile module, as we've seen in previous chapters, comes up with filenames currently not in use; that way we don't need to worry about "collisions" in any filename generation scheme. The pickle module allows the serialization, or saving, of any Python object. The time module lets us find the current time, which Joe judges to be an important aspect of the feedback.
The next line sets the tempdir attribute of the tempfile module to the value of the DIRECTORY variable, which is where we want our data to be saved. This is an example of customizing an existing module by directly modifying its namespace, just as we modified the stderr attribute of the sys module earlier.
The next line uses several functions in the time module to provide a string representation of the current date and time (something like 'Sat Jul 04 18:09:00 1998' , which is precise enough for Joe's needs), and creates a new attribute called time in the data instance. It is therefore saved along with data .
The last line does the actual saving; it opens the file with a name generated by the tempfile module in write mode and dumps the instance data into it. That's it! Now the specified file contains a so-called "pickled" instance.