Recipe 7.7. Interpreting the Output of Test::Unit

Problem

You've diligently created unit tests for your application, but you're puzzled by what happens when you run the tests. How do you understand the output of Test::Unit? How do you find which tests passed and failed?

Solution

Here are the results from running a small test case. The output shows that two tests passed, one failed, and one produced an error:

$ ruby test/unit/employee_test.rb  Loaded suite unit/employee_test Started .F.E Finished in 0.126504 seconds.   1) Failure: test_employee_names_are_long(EmployeeTest) [unit/employee_test.rb:7]: <"Nancy"> expected to be =~ </^\w{12}/>.   2) Error: test_employee_is_from_mars(EmployeeTest): NoMethodError: undefined method 'from_mars?' for #<Employee:0x40763418>     /usr/lib/ruby/gems/1.8/gems/activerecord-1.13.2/lib/active_record/base.rb:1498:in                                                                        'method_missing'     unit/employee_test.rb:22:in 'test_employees_is_from_mars' 4 tests, 5 assertions, 1 failures, 1 errors

Even without seeing the test suite, we can get a feel for what happened. The .F.E message summarizes what happened when the tests ran: the dots indicate tests passed, F indicates a failure, and E indicates an error. The failure is simple: application should reject employees with a first name less than 12 character long, but it evidently doesn't. We can also tell what the error was: the test suite evidently expected the Employee class to have a from_mars? method, which wasn't there. (It's an error rather than a failure because the application itself encountered an errora call to a missing methodinstead of returning an incorrect result.) With this knowledge, it should be easy to debug the application.

Discussion

The order of the results in .F.E don't correspond to the order of the test method definitions in the class. Tests run in alphabetical order; the order in which tests run should have no effect on the results. Each test should be independent of the others. If they're not (if the tests only succeed if they run in a certain order), the test suite is poorly constructed. The output of one test should not impact the results of any other tests.

Notice that using descriptive test names makes it a lot easier to analyze the output. A name that shows just what a test is trying to accomplish can only help later, when the memory of writing the test fades.

See Also

• Section 7.24"