Hack 64. See Test Failure Diagnosticsin Color


Hack 64. See Test Failure Diagnostics in Color!

Highlight the unexpected.

If you follow good development practices and write comprehensive unit tests for your programs, you'll be able to develop faster and more reliably. You'll also eventually run into the problem of too many successes hiding the failuresthat is, if you keep your tests always succeeding, you only need to know about the tests that fail.

Why not make them stand out?

Perl's standard testing harness, Test::Harness is actually a nice wrapper around Test::Harness::Straps, which is a parser for the TAP format that standard tests follow. If and when the report that Test::Harness writes isn't sufficient, use Test::Harness::Straps to write your own.

The Hack

There are two barriers to this approach. First, the default behavior of Perl's standard testing tools is to write diagnostic output to STDERR. Test::Harness doesn't capture this. Second, Test::Harness goes to a bit of trouble to set up the appropriate command line paths to run tests appropriately.

The first problem is tractable, at least if you can use a module such as IPC::Open3 to capture STDERR and STDOUT. The second problem is a little trickier. The current version of Test::Harness::Straps, which ships with Test::Harness 2.48, doesn't quite provide everything publicly that you need to run tests well. Hopefully a future version will correct this, but for now, the hack is to use a private method, _command_line( ), to generate the appropriate command for running the test.

Adding color is very easy, at least on platforms where Term::ANSIColor works. The code can be as simple as:

#!/usr/bin/perl use strict; use warnings; use IPC::Open3; use Term::ANSIColor; use Test::Harness::Straps; my $strap = Test::Harness::Straps->new( ); for my $file (@ARGV) {     next unless -f $file;     my $output;     my $command = $strap->_command_line( $file );     my $pid     = open3( undef, $output, $output, $command );     my %results = $strap->analyze( $file, $output );     print $_->{output} for @{ process_results( $file, \\%results ) }; } sub process_results {     my ( $file, $results ) = @_;     my $count              = 0;     my @results;     for my $test ( @{ $results->{details} } )     {         $count++;         next if $test->{ok};         push @results =>         {             test   => $test,             output => create_test_result(                 $test->{ok}, $count, @{$test}{qw( name reason diagnostics )}             )         };     }     return \\@results; } sub create_test_result {     my ( $ok, $number, $name, $reason, $diag ) = @_;     $ok       = $ok ? 'ok' : 'not ok';     $reason ||= '';     $reason   = " ($reason)" if $reason;     $diag   ||= '';     return color( 'bold red' ) .            sprintf "%6s %4d %s%s\\n%s\\n", $ok, $number, $name, $reason,             color( 'clear yellow' ) . $diag . color( 'reset' ); }

The code loops through every test file given on the command line, running it through IPC::Open3::open3( ) to collect the output from both STDERR and STDOUT into $output. It uses Test::Harness::Straps's analyze( ) method to turn the TAP output into a data structure representing the tests, and then processes each result.

Passing tests are uninteresting; only failures with diagnostics are useful, so the process_results( ) function filters out everything else. Test numbers and names print out in bold red and test diagnostics print in clear yellow.

Hacking the Hack

Test::Harness::Straps actually provides much more information, such as the number of total tests expected and actually run, as well as special information about TODO and SKIP tests. It's easy to provide a Test::Harness-style summary of each test file run as well as the total tests.

It's also possible to write a harness that collects output over a network or from other sources. Perl Testing: A Developer's Notebook (O'Reilly, 2005) has other examples and suggestions.



Perl Hacks
Perl Hacks: Tips & Tools for Programming, Debugging, and Surviving
ISBN: 0596526741
EAN: 2147483647
Year: 2004
Pages: 141

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net