| Quality is a multifaceted concept in software, just as in hardware. In fact, taking a multisided view of quality is essential to understanding and creating product value and to satisfy an assortment of stated and unstated customer requirements. The producer also has to meet the requirements of a multitude of other stakeholders, such as regulators, suppliers, trade associations, media, and other interest groups. Last, but not least, every producer has to ensure that its products are cost-effective and competitive to respond to the needs of the enterprise's owners. Quality encompasses all such needs and requirements. One often hears that quality issues are different in software, but that has to be taken with a grain of salt. Generally, quality principles, systems, and methodologies applicable to manufactured products are equally valid in software, hardware, and other products and services. However, software products have their own design and development environments that must be understood. Furthermore, you must recognize the specific challenges associated with the discrete nature of digital systems and the complexities associated with many software products given the novelty and difficulties of the tasks they are often designed to handle. We believe that for software enterprises, and in organizations where software development is an important activity, the issues of software architecture and design are central to the enterprises' long-term viability and success. In all such organizations, these tasks are too important to be left to software engineers only. The task of producing trustworthy software is truly challenging and calls for total management involvement. This book presents a philosophical base, a management system, and a technology for managing software quality in both large and small enterprises, with particular reference to enterprise software. Similarities and Differences Between Software and Manufactured Products Understanding similarities between software and manufactured products is essential in designing a robust software quality management system. Only then can you adapt systems and methodologies that have made such a remarkable contribution to improving quality of all kinds of manufactured products, as well as other products and services, over the last 50 years. Important differences, however, must be taken into account to develop valid software quality systems. The following are some pertinent issues: Software and manufactured products differ regarding the significance of various phases in their development process (see Figures 2.1 and 2.2). Software development is characterized by the centrality of design. Software is in fact a case of pure design in which all activities are design-related. Thus, its quality and reliability issues are always the result of design faults that are, in turn, the deficiencies often caused by some cognitive failure. Figure 2.1. Basic Steps in the Development of Manufactured Goods  Figure 2.2. Basic Steps in Software Development  In software, nothing is comparable to manufacturing and assembly, in which design lapses can be compensated for or even corrected. Essentially, software has no fabrication; the program code itself is but a further level of design. Furthermore, software has nothing like a product's "done and delivered." It can always be redesigned, modified, or upgraded and thus changed. This flexibility in software often breeds an attitude of "Let's deliver now; we can always fix it later." Unlike a manufactured product, software does not have any equivalent design analysis. Most analysis (testing) has to be done on program code. Hence, design problems or lapses may not be detected until late in the development process or during usage.
It may also be noted that today many hardware products are increasingly dependent on software for their functioning. These complex systems present another level of challenge for software design. Comparing Software and Hardware Reliability Hardware unreliability is dominated by random physical failure of individual components, which are often a consequence of the perversity of nature. Software, on the other hand, is characterized by the discrete behavior of digital systems. Hardware design and engineering knowledge are more easily documented to prevent failure than is the case in software. Furthermore, reliability theories for hardware have been developed over the years and have enabled very high reliability in manufactured products.[1] By comparison, very little knowledge base is available for software reliabilitythus the inherent difficulty in software reliability assurance. Software is always accompanied by hardware. However, when you know the reliability of the hardware component in the system, you can optimize the system's reliability by including only software components.[2] Any system consisting of software and hardware may fail due to the incapability of the software executed by external commands. A software failure is defined as a departure from the expected external result or as an output of the program that differs from the stated requirements. Software may fail when it is used in an unexpected situation. Generally, a failure may be caused by a software fault or by some other cause in a new user environment or an unanticipated usage condition. In other words, the program must be run for a failure to occur. The current hardware/software system cost trend is approaching software domination disproportionately. A typical software life-cycle cost (LCC) exceeds hardware cost, with 80 to 90% of these costs going into software maintenance to fix, adapt, and expand the delivered program to meet the users' changing and growing needs. Some 40% of software development cost is spent on testing to remove errors and to ensure high quality.[3] The software-to-hardware failure frequency could be as high as 100:1 for typical integrated-circuit computers.[4] For more complicated chips, it may be even higher and thus carries critical cost and quality imperatives. Table 2.1 summarizes the major differences in hardware and software reliability. Table 2.1. Differences and Similarities Between Hardware and Software Reliability[5]Category | Hardware Reliability | Software Reliability |
|---|
Fundamental concept | Due to physical effects | Due to programmer errors (or program defects or faults) | Life-cycle causes | | | Analysis Feasibility Design Development Operation | Incorrect customer understanding Incorrect user requirements Incorrect physical design Quality control problems Degradation and failure | Incorrect customer understanding Incorrect user requirements Incorrect program design Incorrect program coding Program errors (or remaining defects or faults) | Use effects | | | Function of design | | | Domains Time relationships | Hardware wears out and then fails | Software does not wear out but fails from unknown defects or faults | Math models | Physics of failure Time (t) | Programmer skill Time and data | Time domain | Bathtub curve | | | | Theory well-established and accepted | Decreasing function Theory well-established but not well-accepted | Functions | R = f(λ,t), λ = failure rate Exponential (constant λ) Weibull (increasing λ) | R = f (failure [or defects or faults], t) | Data domain | No meaning | No agreement among the various time function models that have been proposed Failures = f (data tests) | Growth models | Several models exist | Several models exist | Metric | λ, MTBF (mean time between failures) MTTF (mean time to failure) | Failure rate, number of defects (or faults) detected or remaining | Growth application | Design, prediction | Prediction | Prediction techniques | Block diagram, fault trees | Path analysis (actual analysis of all paths is an unsolvable problem, because the number of possible dynamic paths for even simple programs can be shown to be infinite), complexity, simulation | Test and evaluation | Design and production acceptance | Design acceptance | Design | MIL-STD-781C (exponential) Other methods (nonexponential) | Path testing, simulation, error, seeding Bayesian | Operation | MIL-STD-781C | None | Use of redundancy | | | Parallel | Can improve reliability | Need to consider common cause | Standby | Automatic error detection and correction, automatic fault detection and switching | Automatic error detection and correction, automatic audit software and software reinitializing | Majority logic | m-out-of-n | Impractical |
Reproduced with permission from W. Kuo, V. Rajendra Prasad, F. A. Tillman, and Ching-Lai Wang. Optimal Reliability Design. Cambridge University Press, Cambridge, 2001, section 13.5.1, p. 4.
Causes of Software Unreliability Software reliability is a major societal issue. In fact, it is assuming global significance, and considerable resources are being committed to address it. The following are major causes of software unreliability: Lack of management commitment: The most common causes of quality problems are lack of management commitment, involvement, and support. Deming[6] once estimated that causes attributable to management comprised some 85% of the overall quality problems in an organization; he later revised that estimate upward to 94%. This is applicable to both software and manufactured goods. Inadequate interaction with users: The user environment and requirements are not properly understood. It is implied that the voice of customer should be sought and that their stated and unstated requirements are adequately understood and interpreted. Unfortunately, software development often entails little user involvement after specifications are written and agreed upon. This lack of ongoing interaction with the users and their changing and evolving requirements is not always recognized in the software development process. This is a major cause of software quality problems and must be addressed. Increasing complexity: Software systems are being called upon to handle increasingly difficult problems. Often, no equivalent manual solutions are available to help you understand the nature of the difficulty involved. Software empowers the designer to undertake unsurpassed levels of difficulty and to provide additional features and convenience that may or may not create bona fide value. Complex software systems used for automatic flight control, massive search engines, e-commerce, and global multicurrency fund-transfer management have no manual equivalents for comparison. Furthermore, such novel applications may have no prior experience to compare with. Both difficulty and novelty lead to complexity and consequent design and cognitive challenges, with resultant reliability, safety, and security challenges in software development. Lack of agreed-upon criteria: In novel complex systems, the developer often ends up setting reliability criteria that may or may not meet user requirements. Competitive time pressure: The competitive need for short development cycle time ("We can fix it later, but not deliver later") almost inevitably results in design and other faults. Very often, the time needed for comprehensive testing and debugging is just not made available. Limited scope of automation: Software development and usage are intensely human-interactive. Automation tools such as CASE and Object-Oriented Design have certainly helped but the scope of automation in software is limited compared to manufactured products. Linkage with the Internet: The increasing use and integration of software systems with the Internet makes the system vulnerable to accidental and malicious risks. Such risks can be enormous. They range from identity theft to massive financial fraud to national security threats. Discrete behavior of digital systems: Given the inherently discrete behavior of digital systems, it is difficult to provide quality assurance in a software product. Lack of adequate incentives: Often, there are inadequate market and regulatory incentives for software reliability, but there are plenty of inducements for innovative features, convenience of use, and rapid development cycle time.
|
