| 1: | Given that it's impossible to test a program completely, what information do you think should be considered when deciding whether it's time to stop testing? |
| A1: | There is no universal answer for when to stop testing. Each project is different. Examples of the information that might go into the decision would be: Are lots of bugs still being found? Is the team satisfied with the number and types of tests that have been run? Have the reported bugs been evaluated to decide which ones to fix and which ones not to fix? Has the product been validated against the user's requirements? |
| 2: | Start the Windows Calculator. Type 5,000-5= (the comma is important). Look at the result. Is this a bug? Why or why not? |
| A2: | The answer you get is 0, not 4095 as you would expect. The reason is that the , (comma) was automatically converted to a . (decimal point). What you calculated was 5.000-5=0, not 5,000-5=4995. To determine if this is a bug, verify this operation against the product specification. It might state that commas are to be converted to decimal points. You should also validate this "feature" against the user requirements. Find out if most users want this to occur or if it would be confusing. |
| 3: | If you were testing a simulation game such as a flight simulator or a city simulator, what do you think would be more important to testits accuracy or its precision? |
| A3: | The purpose of a simulation game is to put the player into an artificial environment that mimics a real-world scenario. Flying a flight simulator should look and feel like flying a real plane. Running a city simulator should reflect what happens in a real city. What's most important is how accurately the simulator reflects reality. Does the plane fly like a Boeing 757 or Piper Cub? Does the skyline of the city look like the city it represents? Once you know the software is accurate, you can then test for precision. This is where you start caring about whether the windows in the buildings are in their precise locations and whether the plane movement perfectly maps to the joystick motion. |
| 4: | Is it possible to have a high-quality and low-reliability product? What might an example be? |
| A4: | Yes, but it depends on the customer's expectations for quality. Many people buy high-performance sports cars that are considered to be high quality for their acceleration and speed, their style, and their fit and finish. Often these same cars are notoriously unreliable, frequently breaking down and being expensive to repair. The owners have their own perception of quality and don't see low-reliability as an important issue. |
| 5: | Why is it impossible to test a program completely? |
| A5: | With any software other than the smallest and simplest program, there are too many inputs, too many outputs, and too many path combinations to fully test. Also, software specs can be subjective and be interpreted in different ways. |
| 6: | If you were testing a feature of your software on Monday and finding a new bug every hour, at what rate would you expect to find bugs on Tuesday? |
| A6: | Two axioms come into play here. The firstthat the number of bugs remaining is proportional to the number of bugs you've already foundmeans that you won't come in on Tuesday and miraculously find the software to be perfect. The pesticide paradox tells you that if you continue to run the same tests over and over that you eventually won't find new and different bugs until you add more tests. Given these two axioms, you will probably continue to find bugs at the same rate or slightly less. |
