Hack 53. Debug with Test Cases

Make exploratory code reusable.

Many programmers have subdirectories full of little test snippets; it's common to write a few programs to explore a feature of the language or a new library. It's also common to do this with false laziness, eyeballing output and tweaking an idea here or there.

Usually that's okay, but occasionally you know you wrote code to explore something you need to know right nowif only you could find it and decipher what you were thinking.

If you know how to write test cases with Perl's standard testing tools, you can end this madness and make even your experiments reusable and maintainable.

The Hack

Suppose you've just learned that Perl's default sorting algorithm changed from unstable to stable for Perl 5.8.0. The Internet reveals that, with a stable sorting algorithm, elements that have the same position in the sorting order will retain the positions relative to each other that they had in the input.

Writing test code

What does that really mean in practice? It's time to write some code:

my @elements = (     [ 2, 2 ], [ 2, 1 ], [ 2, 0 ],     [ 1, 0 ], [ 1, 1 ], [ 1, 2 ], ); my @sorted   = sort { $a->[0] <=> $b->[0] } @elements; local $"     = ', '; print "[ @$_ ]\\n" for @sorted;

A stable sorting algorithm should produce the output:

[ 1, 0 ] [ 1, 1 ] [ 1, 2 ] [ 2, 2 ] [ 2, 1 ] [ 2, 0 ]

Because the algorithm sorts only on the first element, all of the ones should come before the twos. Because the algorithm is stable, all of the second values of the ones should increase and all of the second values of the twos should decrease.

From test code to test cases

Of course, six months later that code may be somewhat impenetrable. It has decent variable names, but it's quick and dirty and likely uncommented. What does it prove? Why? Even worse, the first time it ran it has no debugging informationit's easy to misread the output when flipping back and forth between it and the code to recreate the algorithm in your head.

That's the point of test cases: removing tedium and making expectations clear, unambiguous, and automatable. Here's the same file rewritten as executable tests:

use Test::More tests => 4; my @elements = (     [ 2, 2 ], [ 2, 1 ], [ 2, 0 ],     [ 1, 0 ], [ 1, 1 ], [ 1, 2 ], ); my @sorted   = sort { $a->[0] <=> $b->[0] } @elements; is( $sorted[0][0], 1, 'numeric sort should put 1 before 2'     ); is( $sorted[0][1], 0, '... keeping stability of original list' ); is( $sorted[2][1], 2, '... through all elements'               ); is( $sorted[3][1], 2, '... not accidentally sorting them'      );

With a little more work up front, your expectations are clearer. If there's a failure, you see where it fails without having to trace the algorithm in your head again. You can also see which part of your assumptions (or code) failed in detail as fine-grained as you care to test. Even better, you can add more tests to check further behavior, such as mingling the definition of the ones and twos further.

In case an upgrade changes the behavior of your production code, you can also run the test cases to narrow down the problem.

Hacking the Hack

Ideally, someone's already tested this sort of codethe Perl 5 porters. If you have access to the source code of Perl (in this case) or the library you're testing, you can skim the test suite for examples to borrow and modify or learn from outright. In this case, code in t/op/sort.t tests Perl's stable sort. Even just skimming the test descriptions can reveal a lot of information about the ideas behind the implementation.