Hack 68. Run Test Suites Persistently


Speed up your tests.

Large Perl applications with many interconnected modules can take a long time to start up. Perl needs to load, compile, and initialize all of the modules before it can start running your application.

Tests for a large system can be particularly slow. A test suite typically contains lots of small short-lived scripts, each of which pulls in lots of module code at start up. A few seconds of delay per script can add up to a lot of time spent waiting for your test suite to finish.

The cure for long startup times within web-based applications is to run under a persistent environment such as mod_perl or PersistentPerl. PersistentPerl works for command-line programs as well. It's usually as simple as changing the shebang line from #!/usr/bin/perl to #!/usr/bin/perperl.

Running your test suite persistently is slightly more complicated, and doesn't work for every test, but the benefit is a huge speed increase for most of your tests. Running your test suite persistently can speed up your tests by a factor of five on a slow machine.

The Hack

The first step of the hack is to make Test::Builder-based scripts compatible with PersistentPerl. There are several parts to this:

  • The script has to reset the Test::Builder counter on startup.

  • The script needs to prevent Test::Builder from duplicating STDOUT and STDERR, as this seems to be incompatible with PersistentPerl.

  • Scripts with no_plan have to register a PersistentPerl cleanup handler to display the final 1..X line.

Test::PerPerlHelper does all of this for you:

package Test::PerPerlHelper; use strict; use warnings; use base 'Test::Builder'; require Test::More; sub import {     my $class = shift;     if (eval {require PersistentPerl} && PersistentPerl->i_am_perperl( ))     {         # rebless the Test::Builder singleton into our class         # so that we can override the plan and _dup_stdhandles methods         my $Test = Test::Builder->new( );         bless $Test, __PACKAGE__;     }     $class->plan(@_); } sub plan {     my $class = shift;     return unless @_;     my $Test  = Test::Builder->new( );     if (eval {require PersistentPerl} && PersistentPerl->i_am_perperl( ))     {         $Test->reset( );         Test::Builder::_autoflush(*STDOUT);         Test::Builder::_autoflush(*STDERR);         $Test->output(*STDOUT);         $Test->failure_output(*STDERR);         $Test->todo_output(*STDOUT);         $Test->no_ending(1);         my $pp   = PersistentPerl->new( );         $pp->register_cleanup(sub { $Test->_ending });     }     $Test->SUPER::plan(@_); } # Duplicating STDERR and STDOUT doesn't work under perperl # so override it with a no-op sub _dup_stdhandles {  } 1;

Under the hood, Test::Builder uses a singleton $Test object to maintain state. No matter how many times you call Test::Builder->new( ), it always returns a reference to the same $Test object. It does this so that all the various CPAN test modules can all share the same test state (especially the test counter).

Test::PerPerlHelper makes itself a subclass of Test::Builder, and then sneakily reblesses the Test::Builder singleton so that it is a Test::PerPerlHelper instead of a Test::Builder. In this way Test::PerPerlHelper can make itself compatible with all of the CPAN Test::* modules by customizing the singleton $Test object.

Test::PerPerlHelper only does this and other PersistentPerl-related compatibility tricks if the test script is running under PersistentPerl, so you can safely run the same test script normally as well.

The only change that you should have to make to your test scripts is make them end in a true value:

use Test::More 'no_plan'; ok(1); ok(2); ok(3); 1;  # persistent tests need to end in a true value!

Creating a wrapper around prove

Next you need to make all of your test scripts run in the same shared PersistentPerl interpreter.

Normally when you run a program under PersistentPerl, the Perl interpreter stays running in the background after your program terminates. The next time you run the program, PersistentPerl will reuse the same backend interpreter. Typically, each program gets its own private interpreter.

However, for test suites, this policy of one interpreter per program causes a problem. When you use Test::Harness's prove program to run your tests, you don't want to make prove itself persistent; you want to make all of your test scripts persistentand you want them all to share a single interpreter.

The first step is to create a wrapper script called perperl-runscript which you will use to run every test script:

#!/usr/bin/perperl -- -M1 use strict; use Test::PerPerlHelper; my $script; while (my $arg = shift) {     # if the arg is a -I switch, add the directory to @INC     # unless it already exists     if ($arg =~ /^-I(.*)/ and -d $1)     {         unshift @INC, $1 unless grep { $_ eq $1 } @INC;     }     else     {         $script = $arg;     } } do $script or die $@;

Place perperl-runscript somewhere in your $PATH.

Running the Hack

Set the HARNESS_PERL environment variable to perperl-runscript to cause Test::Harness to run every test through this script instead of through Perl. Because the name of this script never changes, PersistentPerl will always use the same backend interpreter to run every test. The -M1 switch on the shebang line tells perperl to only spawn one backend interpreter.

You can set HARNESS_PERL on the same line as prove:

$ HARNESS_PERL=perperl-runscript prove -Ilib t/             

Better still, create a wrapper script around prove called perperl-prove:

#!/bin/sh export HARNESS_PERL=perperl-runscript prove $*

Now you have the choice of running your test suite persistently or non-persistently:

$ perperl-prove -Ilib t/ $ prove -Ilib t/             

To "restart" PersistentPerl, you must kill its backend processes:

$ killall perperl_backend             

You have to restart PersistentPerl if any code outside of the test script itself has changed. However you don't have to restart PersistentPerl if only the test script has changed.

Hacking the Hack

There are some limitations with running tests persistently. In particular:

  • Scripts that muck about with STDIN, STDOUT, or STDERR will have problems.

  • The usual persistent environment caveats apply: be careful with redefined subs, global variables, and so on; required code only gets loaded on the first request, and so forth.

  • Test scripts have to end in a true value.

Expect some scripts to cause problems. When you find a script that does not play nicely with PersistentPerl, you can configure the script to skip all its tests when run persistently:

use Test::More; use Test::PerPerlHelper; if (eval { require PersistentPerl } and PersistentPerl->i_am_perperl( ) ) {     Test::PerPerlHelper->plan(         'skip_all',         'Redirecting STDIN doesn't work under perperl' ); } else {     plan "no_plan"; }

You can also use the "Reload Modified Modules" [Hack #30] hack to reload modules without restarting PersistentPerl.

Add the following lines to your test script:

use Module::Reloader; Module::Reloader::reload( ) if $ENV{'RELOAD_MODULES'};

Now you can reload modules by setting the RELOAD_MODULES environment variable to a true value:

$ RELOAD_MODULES=1 perperl-prove t/             

If you don't set the environment variable, then the modules will not be reloaded:

$ perperl-prove t/             

Note that for some reason Module::Reloader doesn't work on the first run of a script; it only starts working on the second run. The first run fails with an error, Too late to run INIT block.



Perl Hacks
Perl Hacks: Tips & Tools for Programming, Debugging, and Surviving
ISBN: 0596526741
EAN: 2147483647
Year: 2004
Pages: 141

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net