1.8 Performance metrics and evaluation criteria

< Free Open Study >

Selecting a computer system architecture and system support software requires performance metrics and evaluation criteria. In order to generate such information, a user must follow a methodology of selection that defines the user needs, the motivations, and the environmental and technological boundaries. As with the purchase of any product, the purchaser should identify how the product (in this case a computer system) will be used. This first element of the selection process is the most important, since if we don't define the needs and uses properly, the remaining tasks will have a predefined built-in error. Therefore, the prospective buyer should compile a wish list of all potential uses. For example, the list may include the following:

Multiple processors
Distributed file server
Redundant disk drives
Word processing
Spreadsheet analysis
Electronic mail
Remote job entry
Real-time control
Interactive log on and execution or results
Physical installation layouts
Maximum node count and types
Reliability considerations
Network management
Factory automation
Computer types
Video, audio, or both
Interconnection to existing MANs or WANs
Resource sharing
Distributed computing
Very large database

From this wish list the user must generate processing requirements, communications transfer, and management requirements. For example, given that we have N computers, which must be able to simultaneously transfer data to other sites, we have given a requirement for bandwidth (or an I/O rate maximum) and concurrency of access, both of which affect protocols, topology, and media requirements, to name a few. This set of processing requirements, communications transfer, and management requirements can now be used to aid us in the other phases. The second portion of the methodology is to develop a motivational purpose for the computer system: to define why we want one in the first place. For example, we may want to compete with our competitors, who are offering better or extended service to their customers by the use of an enhanced backplane, to have an edge in information availability to enhance the corporation's decision-making ability, or to provide better control or use of the company's computing resources. The motivation for computing system selection will also provide our prospective buyer or designer with more performance and evaluation criteria upon which to base a decision.

The next phase within the computer systems evaluation methodology is to assess the environmental and technological aspects in which the computer system must fit. For example, is the computer system and its interconnection subsystem intended for implementation in a dirty, hot, cold, or varying environment? Will the computer system or some of its components be subjected to stress and strain from natural elements such as wind, rain, snow, or lightning? Will the computer system or its components be put in an air-conditioned computer room or be spread out throughout a building? Is the building new construction or old construction? Will computer systems interconnects need to penetrate floors and go up risers? If so, what is the prevailing fire code? Will the computer system link many buildings together? If so, will interconnections be strung overhead or be poled from building to building? Will wiring be buried? Will it go under water or within water-carrying pipes?

From a technological viewpoint, the computer system may need to interconnect to a diverse set of present company assets and also be able to link planned new resources. These resources have their own peculiarities in terms of electrical specifications, pin count, and makeup. These peculiarities will also map into requirements on the interface equipment and software. The computer system's interconnect components must be able to interface these devices directly or via an intermediate device, which should be an off-the-shelf component if possible.

Once all these initial analyses have been completed and their data compiled, the prospective purchasers or designers should have a large volume of data from which to drive the computer systems requirements.

The next question is: How to use these data to assist in the selection? Do you compile these data into a model of a prospective computer system and use this information to derive analytical and qualitative analysis of the prospective computing system and then compare these results to other known product parameters? Or is a simulation model more in line? In any case, a means of evaluating these data must be provided and must be able to use data that have been collected.

The collected data can be divided into quantitative and qualitative classes. That is, there is one set of data from which specific performance measures can be derived and another from which only subjective measures can be derived. The quantitative data sets should be used to build a model of the proposed system and derive composite measures to evaluate given prospective computer systems architectures and configurations. The methods used for this analysis are analytical and simulation models. The testbed and operational analysis methods may not be viable to test alternatives early on in systems analysis.

< Free Open Study >