CGI Programming in OmniMark | XML and SOAP Programming for BizTalk(TM) Servers (DV-MPS Programming)

[Previous] [Next]

You invoke an OmniMark program from the command line by using the following basic syntax, where Program.xom is the filename of the program to run:

 omnimark -sb Program.xom

Contrast this to Perl's command-line interface. To run a Perl program you use the following syntax, where program.pl is the filename of the program to run:

 perl program.pl

The free OmniMark C/VM (compiler/virtual machine) is analogous to the Perl or Python interpreter.

You can also pass OmniMark C/VM an argument file or an IDE project file. (I'll cover OmniMark argument files later in the chapter.)

Hash-Bang Notation

Hash-bang notation (sometimes known as she-bang or #!) is a way to allow a source file to specify a command line to use to run it.

Hash-bang notation is used primarily in UNIX-type operating systems. Microsoft Internet Information Services (IIS), for example, does not use this method. IIS runs CGI programs based on an associated file extension. Apache for Win32 uses hash-bang notation to configure programs to run as CGI programs.

Hash-bang notation is required for scripts run under the Apache Web server on any platform. When Apache processes a script, the Web server reads the first line of the program. If the first two characters of the first line of the program are #!, the Web server uses the rest of the first line (whatever follows #!) as the command line to use to run the script, placing the name of the script at the end of the command line.

For example, for the program Hello.xom:

 #!C:\OmniMark\omnimark.exe -sb process output "Content-Type: text/html%13#%10#%13#%10#" || "Hello, OmniMark world!"

the Web server will run the following command:

 C:\OmniMark\omnimark.exe -sb Hello.xom

For the corresponding Perl program Hello.pl :

 #!C:\Perl\bin\Perl.exe print "Content-Type: text/html\r\n\r\n"; print "Hello, Perl world!";

the Web server will run the following command:

 C:\Perl\bin\Perl.exe Hello.pl

You'll find the comment character # in many of the UNIX-world scripting languages, such as Perl. Therefore, the #! line is ignored when the program is run. OmniMark behaves the same way. Even though OmniMark's comment character is a semicolon (;), OmniMark won't object to the #! if, and only if, it is in the very first line of the program.

NOTE
On UNIX-type systems, hash-bang programs must have the executable bit set to indicate to the Web server that the file can be executed; otherwise, the Web server will report an error when it is asked to invoke the CGI program. This bit is set using the chmod command.
For example, to allow a UNIX Web server to run the Hello.xom program mentioned earlier, you have to make the program executable by entering:
 chmod a+x Hello.xom 
at the terminal prompt.

Configuring Web Servers for OmniMark CGI

This section describes how to configure IIS and Apache Web servers to run OmniMark CGI programs.

NOTE
You will need to log on to your machine with administrative rights to perform these steps.

Internet Information Services

To configure IIS on Microsoft Windows 2000 so that you can run OmniMark programs as CGIs, follow these steps:

Log on to your machine with administrative privileges.

Start the Microsoft Management Console by right-clicking the My Computer icon on the desktop and selecting Manage from the pop-up menu.

Expand Services and Applications and Internet Information Services in the Tree window of the console. Right-click on the Web site you want to add a CGI directory to, and select Virtual Directory under the New pop-up menu item. This will run the Virtual Directory Setup Wizard. Create a new virtual directory, and make sure that the executable flag is set. You'll use this directory to hold your OmniMark source files.

Right-click on the new CGI directory, and select Properties. A Properties dialog box will appear.

Click the Configuration button on the Virtual Directory tab. The Application Configuration dialog box will appear.

Click the Add button on the App Mappings tab. The Add/Edit Application Extensions Mapping dialog box will appear.

In the Executable text box, enter the command line to start the OmniMark C/VM, substituting %s for the filename of your script. Typically it will look like this:

 C:\OmniMark\omnimark.exe -noexpand -sb %s

Type .xom in the Extension text box.

Ensure that Script Engine is checked, and click OK.

Repeat steps 7 through 9 for the extension .xop, using the following command line:

 C:\OmniMark\omnimark.exe -noexpand -f %s

Repeat steps 7 through 9 for the extension .xar, using the following command line:

 C:\OmniMark\omnimark.exe -noexpand -f %s

Repeat steps 7 through 9 for the extension .pmo, using the following command line:

 C:\OmniMark\omnimark.exe -noexpand -s pmo.xom %s

Click OK to accept the settings.

In the Properties dialog box, make sure that Execute Permissions is set to Scripts and Executables, and click OK.

Personal Web Server on Windows 95/98

To configure Microsoft Personal Web Server (PWS) on Microsoft Windows 95/98 to run OmniMark programs as CGI, follow these steps:

As a safety measure, back up your registry.

Click Start, and select Run to open the Run dialog box. In the drop-down combo box, type regedit and click OK.

In the Registry Editor, navigate to HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\w3svc\parameters\ ScriptMap.

From the Edit menu, select New and then click String Value. This will create a new string value icon and allow you to set its name.

Type .xom as the name of the entry.

Double-click on the new entry. The Edit String dialog box will appear.

In the Value data text box, type the command line for running .xom files, typically:

 C:\OmniMark\OmniMark.exe -noexpand -sb %s

Click OK.

Repeat steps 4 through 8 for the extension .xop, using the following command line:

 C:\OmniMark\OmniMark.exe -noexpand -f %s

Repeat steps 4 through 8 for the extension .xar, using the following command line:

 C:\OmniMark\OmniMark.exe -noexpand -f %s

Repeat steps 4 through 8 for the extension .pmo, using the following command line:

 C:\OmniMark\OmniMark.exe -noexpand -s pmo.xom %s

Close the Registry Editor.

Make sure you have a directory in your server configuration that is set up for running CGI programs. (See the PWS documentation for information on setting up and configuring such a directory.)

The Apache Web Server

The simplest way to make CGI scripts work in any version of Apache is to use the ScriptAlias directive in the Web server's configuration file. This directive maps a virtual directory to a physical directory and tells the Web server that any files in this directory will be run as CGI programs.

Notice that this directive only creates a virtual-to-physical directory mapping; it does not map file extensions to interpreters. Apache will use the hash-bang line at the top of each script to determine the appropriate command line to use.

Apache on Windows 2000

Add the following line to Apache's configuration file. This file is named httpd.conf in the directory in which you install Apache.

 ScriptAlias /virtdir/ "C:/www/root/cgi-bin/"

In this line, virtdir is the name of the virtual directory you want to create and C:/www/root/cgi-bin/ is the name of the physical directory that your CGI scripts reside in. Note the forward slashes on the physical path.

All our examples are going to assume that virtdir is cgi-bin, so the command should be this:

 ScriptAlias /cgi-bin/ "C:/www/root/cgi-bin/"

Stop and restart the Apache server by selecting Start, Settings, Control Panel, and then Services to force the server to reread its configuration file.

Now a request to the Web server for http://hostname/cgi-bin/Hello.xom will cause the server to run the file C:\www\root\cgi-bin\Hello.xom.

Apache on UNIX/Linux

To configure Apache on UNIX or Linux machines to run OmniMark CGI programs, add the following line to Apache's configuration file.

 ScriptAlias /virtdir/ "/www/root/cgi-bin/"

In this line, virtdir is the name of the virtual directory you want to create and /www/root/cgi-bin/ is the name of the physical directory your CGI scripts reside in. Notice the forward slashes on the physical path.

All our examples are going to assume that /virtdir is /www/root/cgi-bin, so the command should be this:

 ScriptAlias /cgi-bin/ "/www/root/cgi-bin/"

Stop and restart the Apache server by running the following line to force the server to reread its configuration file:

 apachectl restart

Now a request to the Web server for http://hostname/cgi-bin/Hello.xom will cause the server to run OmniMark from the path specified at the top of the Hello.xom file with the source file as a parameter, as shown in the following code:

 line:/usr/local/bin/omnimark -sb /www/root/cgi-bin/Hello.xom

Locating OmniMark's System Libraries

OmniMark uses system environment variables to find its system libraries and include files. On Windows platforms, the OmniMark installation program sets the environment variables. On UNIX-type systems, you will need to ensure that the following environment variables are set. Assuming the location of OmniMark is /usr/local/bin/omnimark, the following paths should be set up as environment variables:

 OMNIMARK_INCLUDE /usr/local/bin/omnimark/xin/ OMNIMARK_XFLPATH /usr/local/bin/omnimark/lib/=L.so

Each UNIX system has its own procedure for setting environment variables.

NOTE
These OmniMark system environment variables are separate and distinct from CGI environment variables.

Writing an OmniMark CGI Program

This section describes the basics of writing a CGI program in OmniMark, including how to work with forms and CGI environment variables.

OmniMark's CGI Library

The OmniMark library (omcgi.xin) provides helper functions for writing CGI programs. To take advantage of these functions, add the following code to every OmniMark CGI program:

 include "omcgi.xin"

Retrieving Input from the Web Server

CGI programs retrieve their input—such as HTML form fields and values, script name, and the IP address of the client—from both environment variables that the Web server sets and from standard input.

The GET and POST Request Methods

HTTP defines several different types of client requests, named methods. The two most common methods are GET and POST. You might have seen or written HTML form markup that looks like this:

 <form action="Commit.xom" method="post">...

When a browser makes a GET request, all request information is encoded in the URL itself. For instance, output from a form with two fields named user and age might look like this:

 http://localhost/Commit.xom?user=Randall+McMurtry&age=29

The part of the URL to the right of the question mark is named the query string. It contains the information the client sends to the Web server for processing.

When the client makes a POST request, the query string is not appended to the end of the URL. Instead, it is sent in the body of the HTTP request.

Whichever method is used—GET or POST—you can use the cgiGetQuery function to access the form data in the same way.

Accessing GET and POST Information

The cgiGetQuery function (defined in omcgi.xin) determines which method (GET or POST) was used to send form data, decode the data, and copy the key/value pairs into a stream shelf that you provide.

For instance, the following CGI program displays the names and values of every form field in the request:

 #!C:\Omnimark\omnimark -sb declare #process-input has unbuffered declare #process-output has binary-mode include "omcgi.xin" process local stream query variable cgiGetQuery into query output "Content-Type: text/html" || CRLF || CRLF repeat over query output key of query || ": " || query || "<br>" again

Assuming this program is named Commit.xom, run it with the following URL:

 http://localhost/cgi-bin/Commit.xom?user=Randall+McMurtry&age=29

The browser will display the following information:

 user: Randall McMurtry age: 29

This type of code—which iterates over the values passed in the query string—is useful for CGI debugging.

Accessing CGI Environment Variables

The cgiGetEnv function (defined in omcgi.xin) copies the names/value pairs of a predefined set of CGI environment variables into a stream shelf that you supply. The set includes any environment variables that the Web server uses to pass information to the CGI program.

For instance, you can modify the Commit.xom program to display the unparsed query string before displaying the parsed values. The query string is passed to a CGI program through the QUERY_STRING environment variable:

 #!/usr/local/bin/omnimark -sb declare #process-input has unbuffered declare #process-output has binary-mode include "omcgi.xin" process local stream query variable local stream env variable cgiGetQuery into query cgiGetEnv into env output "Content-Type: text/html" || CRLF || CRLF output "Raw query string: " || env key "QUERY_STRING" || "<br>" when env has key "QUERY_STRING" repeat over query output key of query || ": " || query || "<br>" again

Assuming this program is named Commit2.xom, run it with this URL:

 http://localhost/cgi-bin/Commit2.xom?user=Randall+McMurtry&age=29

The browser will display the following information:

 Raw query string: user=Randall+McMurtry&age=29 user: Randall McMurtry age: 29

Notice the guard on the query string output statement in the OmniMark program. The Web server will create environment variables only when there is some appropriate data available. The following URL will result in the QUERY_STRING environment variable not being set:

 http://localhost/cgi-bin/Commit2.xom

Sending Output to the Web Server

Returning output to the Web server is straightforward: a CGI program writes a simple header followed by some content information to standard output. OmniMark output statements write to standard output (stream #process-output) by default.

Forming the HTTP header

An HTTP header must be output before the HTTP body. These headers are processed by the Web server and help determine the content that is sent to the browser.

Most CGI programs send HTML to the browser, as shown in the following common form:

 output "Content-Type: text/html" ; <- header || CRLF || CRLF ; <- separator || "Hello, OmniMark world!" ; <- body

Note that you must use the CRLF macro twice to create a blank line between the header and the body. Do not use OmniMark's "%n" to do this.

While a CGI program can send several other headers, only a few are generally used. For example, if you want to redirect the browser to another URL, you can use the following header instead of a Content-Type header:

 output "Location: http://www.omnimark.com" || CRLF || CRLF ; no body required

The Web server will interpret this directive and output the correct headers to the browser (for example: HTTP Status 302 -- Page moved). The browser will then be redirected to the desired page.

Copying binary files

Occasionally you will need to copy a binary file, such as a GIF, to the browser. To ensure that OmniMark does not affect any of the internals of the binary file, you must use OmniMark's binary-mode modifier:

 #!/usr/local/bin/omnimark -sb declare #process-input has unbuffered declare #process-output has binary-mode include "omcgi.xin" process output "Content-Type: image/gif" || CRLF || CRLF || binary-mode file "C:\TravisBook\AppxA\Duck.gif "

Notice that you must declare the output stream and the GIF file as binary.

Unbuffering standard input and output

OmniMark streams, including #process-input and #process-output, are normally buffered by OmniMark for efficiency. However, buffered input can cause problems with CGI programs. An OmniMark CGI program might wait indefinitely for the rest of its input data because of OmniMark stream buffering. For this reason, the following declaration must appear at the top of all OmniMark CGI programs that receive input from the browser:

 declare #process-input has unbuffered

Optionally, you can also unbuffer #process-output:

 declare #process-output has unbuffered

This declaration might make responses appear quicker to the browser, particularly for CGI programs that intersperse output with significant periods of processing.

Bulletproofing Your CGI Program

OmniMark provides the programmer with several techniques for writing reliable programs. In a CGI environment, perhaps the most important technique is catching errors that can abort processing.

Catching program errors

OmniMark will halt with interpreter error messages if an illegal action occurs. Examples of illegal actions are setting a keyed value for which no key exists, dividing by zero, accessing a file that doesn't exist, or even running out of disk space.

To ensure that OmniMark doesn't attempt to write interpreter errors back to the browser, use OmniMark's catch facility to catch #program-error:

 #!/usr/local/bin/omnimark -sb declare #process-output has unbuffered declare #process-output has binary-mode include "omcgi.xin" process local stream query variable local stream env variable cgiGetQuery into query cgiGetEnv into env output "Content-Type: text/html" || CRLF || CRLF output "Raw query string: " || env key "QUERY_STRING" || "<br>" repeat over query output key of query || ": " || query || "<br>" again catch #program-error put #error "<br><b>An error occured in this CGI program.</b><br>"

Note that there is no guard on the use of the env shelf. Assuming this program is Commit3.xom, the following URL

 /localhost/cgi-bin/Commit3.xom

will display the following output:

 Raw query string: An error occured in this CGI program.

Preventing source-code snooping

OmniMark's macros can be expanded using the -expand command-line option. Anyone invoking an OmniMark CGI program with this option could cause the entire program to be sent back to the browser, even if the program has no macros. This presents an undesirable security problem. Always configure your Web server to invoke OmniMark with the -noexpand option, which will override any outside attempt to use the -expand option.

Pulling It All Together

Now you have all the pieces to write OmniMark CGI programs. In this section I've included two programs that illustrate the concepts discussed in this appendix.

Using a simple form

The first example displays a simple form that asks for your name and returns the number of vowels in your name.

To invoke this program, simply place it in your cgi-bin directory and enter the URL localhost/cgi-bin/Vowel.xom:

 #!/usr/local/bin/omnimark -noexpand -sb ; ; URL localhost/cgi-bin/Vowel.xom ; include "omcgi.xin" declare #process-input has unbuffered declare #process-output has unbuffered declare #process-output has binary-mode global stream query variable global stream env variable define stream function get-script-name as return env^"SCRIPT_NAME" when env has key "SCRIPT_NAME" return "" process cgiGetQuery into query cgiGetEnv into env output "Content-type: text/html" || CRLF || CRLF || "<html><head><title>OmniMark CGI Test</title></head><body>%n" || "<h1>OmniMark Vowel Counter</h1>%n" do when query has key "name" local counter vowels initial {0} do when length of query^"name" > 0 repeat scan query^"name" match unanchored ul ["aeiou"] increment vowels again output "<p>" || query^"name" || ", you have %d(vowels) vowel" output "s" unless vowels = 1 output " in your name.</p>%n" else output "<p><b>You didn't enter a name.</b></p>%n" done done output "<form action='" || get-script-name || "' method='get'>Your name: %n<input type='text' name='name'>%n" || "<input type='submit' value='Count Vowels'></form>" || "</body></html>%n"

Accessing external data sources

The second example is an OmniMark CGI program that gets the current time from the National Institute of Standards and Technologies atomic clock in Boulder, Colorado. The default refresh rate is 60 seconds; however, you can set it in the URL. For instance, localhost/cgi-bin/atomclock?refresh=10 will set a 10-second refresh rate.

This program uses a local guard where the value of the refresh CGI query string is set. If the value isn't provided in the URL—or if it is non-numeric—the initial value will be used. The final #program-error trap will catch any other errors that are not anticipated but could still occur.

Finally, the always block ensures that the correct HTML ending sequence is issued regardless of what else occurs in the program.

 #! /usr/local/bin/omnimark -noexpand -sb ; ; URL localhost/cgi-bin/atomclock?refresh=NN ; declare #main-output has binary-mode declare #main-output has unbuffered include "omioprot.xin" include "omtcp.xin" include "omdate.xin" include "omcgi.xin" process local counter refresh-rate initial {60} local stream query variable output "Content-type: text/html" || CRLF ||*2 || "<html><head><title>The Exact Local Time" || "</title></head>%n" || '<body bgcolor="ffffff" text="000000">%n' cgiGetQuery into query do ; establish a guard for existance and type set refresh-rate to query^"refresh" catch #program-error ; just use the initial value done do scan TCPConnectionGetCharacters TCPConnectionOpen on "132.163.135.130" at 14 match white-space* digit+ blank+ digit{2} => y "-" digit{2} => m "-" digit{2} => d blank+ digit{2} => h ":" digit{2} => min ":" digit{2} => s output '<meta http-equiv="refresh" content="' || "d" % refresh-rate || '">%n' || '<font face="Comic Sans MS">%n' || '<p align="center">The local time %n' || "according to the ATOMIC CLOCK in%n" || "<br>Boulder Colorado<br><br><b>%n" || format-ymdhms "=xh:=m:=s =a.m. on =W, =n =xD, =xY" with-date ymdhms-adjust-time-zone "20%x(y)%x(m)%x(d)%x(h)%x(min)%x(s)+0000" to-be date "=t" || "%n</b></font><br><br></p>" else output "<h1>Date in unexpected format!</h1>%n" done catch #program-error message the-problem output "<p>" || the-problem || "</p>%n" || "<p><b>An error occurred in this CGI application.</b></p>%n" always output "</body></html>"

Using OmniMark's Argument and Project Files

The OmniMark C/VM supports several command-line options that you can examine by entering the following code from the command line:

 omnimark -help

Often, Web servers only allow the substitution of a single option—the name of the program. You might want to include more options, the most important of which is a log file to capture compiler and interpreter errors. You can do this with an OmniMark arguments file or a project file. The two files accomplish the same thing; however, the project file is more up-to-date.

OmniMark's Argument File

By convention, an OmniMark argument file has the suffix .xar. So the program Hello.xom should have the argument file Hello.xar.

An argument file is a file containing the command-line arguments to be passed to an OmniMark program. Use the -f option to pass an argument file to OmniMark:

 C:\omnimark\omnimark -noexpand -f Hello.xar

Assuming hello.xar contains the following two lines:

 -sb hello.xom -alog Hello.log

it is equivalent to the following command:

 C:\omnimark\omnimark -noexpand -sb Hello.xom -alog Hello.log

OmniMark reads the argument file, ignores comments and extra white space, and treats the file's contents exactly like command-line options.

Hash-Bang Notation in Argument Files

UNIX-like systems do not have a file extension to key on for program invocation. They rely heavily on the hash-bang notation. You can place the hash-bang line at the top of the argument file:

 #!/usr/local/bin/omnimark -f -sb Hello.xom -alog /www/root/log/Hello.log

NOTE
On a UNIX-type system, the .xar file must also be made executable.

Assuming the file Hello.xar is /www/root/cgi-bin/, the following URL will invoke hello.xom and append any log messages to Hello.log:

 /localhost/cgi-bin/hello.xar

OmniMark's Project File

OmniMark has another, newer, variation on the arguments file named a project file. By convention, a project file has the suffix .xop. Thus the OmniMark program Hello.xom should have the project file Hello.xop.

A project file is an XML file that performs the same function as an arguments file and is invoked in exactly the same way. It, too, can have a hash-bang line as the first line (although XML doesn't strictly provide for this).

The OmniMark IDE generates project files directly, and the capabilities of project files will expand over the next few years.

NOTE
When you develop CGI programs with the IDE, be sure to add -brief under the Extra tab in Project Options. If you don't do this, OmniMark will emit its banner information before your program can output its HTTP headers, the Web server will become confused, and your CGI program will break.

Compiling CGI Programs

OmniMark offers a separate byte-code compiler available through the OmniMark Developers Network. For details, visit http://www.omnimark.com. Compiling CGI applications offers several key advantages:

Better response time Larger CGI applications can be more responsive because they don't need to be compiled each time they are executed.

Better source-code control You don't need to distribute source code across several machines or departments running the same CGI script.

Better application security OmniMark byte-code cannot be reverse-compiled. OmniMark's application strings do not show up in byte-code.

Better business security Your business logic will not be compromised if an outsider gains access to your Web server.

Better intellectual property protection CGI-based commercial applications can ship royalty-free in byte-code form, thereby protecting your intellectual property.

By convention, OmniMark byte-code files use the .xvc convention. Byte-code programs execute in a similar manner to source-code programs. The extension/executable pair that must be set up for the Web server should look like this:

 .xvc = C:\OmniMark\omnivm -load %s

You don't need to use the -noexpand option. The VM does not have access to the macro source.

You can also use the VM with an arguments file. A VM arguments file uses the .xva convention. As with source code, the first line of a VM arguments file can be a hash-bang line.

The extension/executable pair that you must set up for the Web server should look like this:

 .xva = C:\OmniMark\omnivm -f %s

For More Information on OmniMark
This appendix has been a brief introduction to the OmniMark programming language. For more information about specifics in the language, check out my book, OmniMark At Work, Volume 1: Getting Started, which is included on the companion CD. Even though the book is a few years old and does not have much in the way of network programming concepts, OmniMark At Work is a good place to start to learn about the basics of the language, including data types, control structures, referents, structured text processing, and so on.

The OmniMark Technologies Web site (http://www.omnimark.com) also has a rich collection of articles, tips, and developer support. The site's OmniMark Developers Network is a subscription service that gets you access to the developers and provides very fast turnaround for problems. Support is also available by e-mail and mailing list.

If you are working in a network programming environment, you owe it to yourself to learn about how OmniMark can provide a viable alternative to Perl for cross-platform, high-performance, scalable Web architectures.

NOTE
This material is adapted from an article written by OmniMark. Copyright © OmniMark Technologies Corporation, 1988-2000. For OmniMark support, go to http://www.omnimark.com. (The praise for this great product is all mine, however!)