The Software Specification Process

When finally agreed upon and documented, the requirements for a new software application emerge in the form of a specification. This document tells exactly what will be expected of the software's functionality after it is completed and meets the specification. Until now, all software development methodologies depended on a multistage process involving specialists with different educational backgrounds, different work experience, and different development responsibilities who had very little contact with other stages in the development process. These multiple stages with high technological separation have made software development rather error-prone. Product and application opportunities are developed by marketing specialists. Requirements are then developed by functional domain experts called application or system analysts to produce a functional specification. This document is mapped into a technical design document by software architects, then is detailed component by component by software engineers, and finally is coded, tested, and certified by programmers. From the earliest days of the application generator, invented by Betty Holberton, and the compiler, invented by Dr. Grace Hopper, the possibility of specification-based programming has been the Holy Grail of the application programming craft. Reaching this elusive goal would entail domain expertssay, accountants or supply-chain specialistswriting a precise functional specification for an accounting or supply-chain application. This would then be fed into a sort of meta-compiler to produce a high-level language program that could be compiled to machine language by a conventional compiler and tested. Kristin Nygaard and Ole-Johann Dahl, the Norwegian inventors of Object-Oriented Programming, spent years trying to take application development to this next level. They used a generic technology that would ultimately be available to all members of the consortium funding its development. Unfortunately, both inventors passed away before their plans were fully realized. Richard Lawson and Richard Patton of Lawson Software in St. Paul, Minnesota have been working on a similar but proprietary middleware technology for more than 20 years. They recently announced their latest version of such a system, called Landmark™. The proprietary Landmark™ development system represents the ultimate upstreaming of software application development design decisions. These decisions will now be made as domain decisions by empowered domain experts, thereby eliminating the multiple layers of (mis)communication involved in traditional software development. The announcement of this new technology created quite a buzz on the Internet and in the MIS press. For example, a lead article in Information Week noted the following:

Lawson Software Inc. last week unveiled a domain-specific language aimed at increasing software quality by reducing the massive amounts of code required in an application. Code-named Landmark, the technology has been under development for more than three years by founder Richard Lawson and chief architect Richard Patton. Lawson software architects will spend the next several years using Landmark to rewrite the company's product line, which was originally written in COBOL. Unlike a general-purpose computer language, a domain-specific one is aimed at a particular kind of business need. Based on a services-oriented architecture, Landmark generates application-specific code in Java.^[1]

Figure 16.1a illustrates the many steps of conventional application software development, with its implied technical language "translation" between each stage. Figure 16.1b shows the idealized specification-based development approach, with domain experts writing precise specifications in a specification language and domain experts evaluating and testing the results. Programmers are needed, but they are not directly involved in the development process. They are the very high-level system programmers who wrote and will later maintain the application software generator. Such a program has been called a "meta-compiler," but this term is not really appropriate. A meta-compiler is a compiler that produces a new compiler, such as Generalized ALGOL at Stanford^[2] and ARGMAT at Stuttgart.^[3] The best nomenclature for this kind of application generator would be "super-compiler," because it transforms one high-level language to another, which is then compiled by a conventional compiler.

Figure 16.1A. Traditional Software Development Processes

Figure 16.1B. Specification-Based Software Development

Comparing the two diagrammed processes does not fully indicate the difference between their final stages. In Figure 16.1b the testing is done by domain experts solely to determine whether the functional capabilities of the resulting application fulfill the specified requirements. In conventional development processes, the testing stage involves straightening out a tangled web of design errors, technical errors, coding errors, human miscommunications, and mistakes. This is best done following a disciplined DFTS approach like that shown in Figure 2.1 in Chapter 2.

It is the contention of the authors of this book that the future of application programming as a profession will include two groups: those who learn enough about the domain to write in the forthcoming novel specification languages by Lawson, Wescorp, and others, and those who learn enough about system programming to write the application generator programs. Application programming as we know it today will either disappear or be outsourced. Between now and then, however, we will have to invoke Taguchi and QFD methods to move design tasks upstream in the development process to achieve robust software architecture.

While the silver bullets are being molded by Lawson Software, Wescorp is getting its patents, and the Norwegian consortium left behind by Nygaard and Dahl is proceeding to a consensus, we must still write a lot of robust enterprise software applications in Java using a dependable multistage software development methodology.

Sidebar 16.1: A Precise Functional Specification

In the late 1960s, one of the authors was general manager of one of the first software companies. Its president had declared that the firm could no longer enter fixed-price contracts for programming due to runaway "scope" problems the firm had with some application development contracts with General Motors. They decided that the solution to this problem was writing more precise functional specifications and "attaching" such a customer-approved specification to the application development contract. The author soon got a call from the head of MIS for a major Minnesota-based railroad for which they had written numerous applications to run on her Univac III in COBOL. She needed a new application and had budgeted $60,000 for it. When he told her we no longer did fixed-price work, she exploded. Margaret was a size-40 scorpion with a vocabulary that would embarrass a Duluth stevedore. She exploded at his reluctance, which she termed a business relationship "betrayal." He proposed a compromise whereby they would write a precise functional specification for $15,000 and she could shop it out. If any other software firm would contract to write the COBOL application for less than $45,000, she would be under budget, and he would have no hard feelings. She agreed. However, the firm won the final bid. The approved new type of specification was attached to a contract calling for three $15,000 progress payments, all of which pleased the firm's president.

Some weeks later, the author visited Margaret at the railroad MIS center for the show-and-tell. They entered the Univac III machine room, which had "no smoking" signs posted; Margaret kept her cigarette in her mouth. They went to the line printer, where proud programmers from both groups were printing out the test report from Margaret's test data. She leaned her ample frame over the printer to get a closer look through her bifocals, dropping cigarette ashes into the printer. She said: "Not bad, Patton. Now, if your team will just move this column over here and that column back there, and change this caption to read like this, etc., etc., before I release the last progress payment." He replied, "But, Margaret, we agreed on a precise functional specification, and this program fully and accurately implements that specification." She looked at him incredulously over her bifocals, which had now slipped down her nose a bit, and replied, "Patton, who the *%$& ever reads a two-volume functional spec?" He realized that Margaret was just exercising her managerial prerogative of "marking territory." However, he could not agree to make the changes without the president's approval, because this was their first use of what was to become a new business model for the firm.

Naturally, the president was furious. He called in the firm's general counsel to discuss suing the railroad to make them live up to the contract and give us the final progress payment without making any changes. The lawyer asked the author how much it would cost to make the changes, and he estimated $300 to $400. He told the boss it would cost several times that to file a lawsuit, so they should just go ahead and please the customer.

Margaret's question has remained unanswered in the author's mind over the ensuing 36 years of application development experience: "Who the *%$& ever reads a two-volume functional spec?" Now we know the answer to her question: When it is "read," it is read by an application generator program!