Hack 62. Segregate Developer and User Tests

Run only the tests you need when you need them.

In general, the more tests you have for a system the better. In specific, the more tests you have, the more time it takes to test your code. In very specific, some tests are more valuable than others. You may reach 95% confidence by running a few test files (user tests) and that may be good enough for day-to-day operations. You may also have a few deeper tests (developer tests) that use external resources or take a long time to explore all of the potential possibilities for failureand you don't necessarily want to make everyone run them all at once.

If your project uses Perl's standard module-building tools (at least, Module::Build, which comes very highly recommended), you can segregate developer and user tests very easily, running the time-consuming tests only when you need to, perhaps right before a release.

The Hack

To customize Module::Build's behavior, almost all you have to know is to define your own subclass and override the appropriate ACTION_* method. How about running only the user tests with the normal perl ./Build test invocation and all tests with perl ./Build disttest?

It's helpful to skim the source of Module::Build::Base [Hack #2] when overriding an action. That's where you can learn that the next method to override is find_test_files( ). The ACTION_test( ) method calls this to figure out which tests to run. Override the test finder method to filter out developer tests. Easy!

Don't celebrate just yet, though: ACTION_disttest( ) launches another Perl process to run Build.PL and, eventually, perl ./Build test. Because this is another process, there's no easy way to set a flag or a Perl variable to tell the second invocation of ACTION_test( ) to run all tests. Fortunately, you can set an environment variable, perhaps PERL_RUN_ALL_TESTS, that both the parent and child can see.

All that's left is to decide where the developer tests are; how about in t/developer/*.t? That's enough to write a Module::Build subclass:

package Module::Build::FilterTests; use base 'Module::Build'; use SUPER; use File::Spec::Functions; sub ACTION_disttest {     local $ENV{PERL_RUN_ALL_TESTS} = 1;     super( ); } sub find_test_files {     my $self  = shift;     my $tests = super( );     return $tests unless $ENV{PERL_RUN_ALL_TESTS};     my $test_pattern = catfile(qw( t developer *.t ) );     push @$tests, <$test_pattern>;     return $tests; } 1;

The SUPER module makes calling the parent implementations of overridden methods a little cleaner syntactically. The only other notable feature of the test is the use of the glob operator to find all tests in the t/developer/ directory.

Running the Hack

In your own Build.PL file, load and instantiate a Module::Build::FilterTests object instead of a Module::Build object. Everything else should proceed as normal.

The easiest way to distribute a custom Module::Build subclass is to distribute it by storing it in build_lib/ or another directory and to use it from Build.PL with a use lib line.

$ perl Build.PL  Checking whether your kit is complete... Looks good Deleting Build Removed previous script 'Build' Creating new 'Build' script for 'SomeModule ' version '1.28' $ perl ./Build  $ perl ./Build test  ... user tests run ...

To run the developer tests, either set the PERL_RUN_ALL_TESTS environment variable before running perl ./Build test or run the distribution tests with perl ./Build disttests.

Hacking the Hack

Could you do the same by checking for the presence of the environment variable in each test file? Absolutelybut consider that adding a new developer test is as easy as putting it in the t/developer/ directory without adding any extra magic to the test file. Also you get the test and disttest targets to behave correctly almost for free.

You can further customize the tests, running specific subsets for different test targets. perl ./Build networktests could try to connect to a server, for example. See the Module::Build documentation for more information.

The author particularly recommends that CPAN authors who want to raise their Kwalitee scores skip the POD tests for non-developers.

In theory, you can achieve the same effect with ExtUtils::MakeMaker. Yet every time the author spelunks into the module's documentation, or worsecode, he wakes up fully clothed and shivering in the shower several hours later.