| Writing tests is easy. Managing tests well is more difficult. Having complete test coverage is worthless if running the complete test suite is so difficult that no one ever does it. Making your tests easy to run without user intervention and making it easy to interpret their results will pay off over and over again. Using the standard testing tools that understand the Test Anything Protocol is just one part of the process. Organizing your tests sensibly is another. How do I do that? Consider the tests for the Test::Harness module. Download the latest distribution from the CPAN and extract it. Change into the newly created directory, run Makefile.PL , and build and test the module:
Note: Look for the Download link at http://search.cpan.org/dist/Test-Harnes/ .
$ perl Makefile.PL Checking if your kit is complete... Looks good Writing Makefile for Test::Harness $ make cp lib/Test/Harness/TAP.pod blib/lib/Test/Harness/TAP.pod cp lib/Test/Harness/Iterator.pm blib/lib/Test/Harness/Iterator.pm cp lib/Test/Harness/Assert.pm blib/lib/Test/Harness/Assert.pm cp lib/Test/Harness.pm blib/lib/Test/Harness.pm cp lib/Test/Harness/Straps.pm blib/lib/Test/Harness/Straps.pm cp bin/prove blib/script/prove /usr/bin/perl5.8.6 "-MExtUtils::MY" -e "MY->fixin(shift)" blib/script/prove <output snipped> $ make test PERL_DL_NONLAZY=1 /usr/bin/perl5.8.6 "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t <output snipped>
What just happened ? Until now, all of the examples have mixed code and tests in the same file. That's fine for teaching, but it won't work as well in production code. There's no technical reason to keep all of the tests for a particular program or module in a single file, so create as many test files as you need, organizing them by features, bugs , modules, or any other criteria. The only technical requirement when using separate test files is that the files must be able to load the modules they test. That means you must manage Perl's library paths appropriately. Fortunately, most good CPAN modules handle this. The magic of making these tests work is the magic of Perl module installation tools such as ExtUtils::MakeMaker and Module::Build . Test::Harness uses the former, as the presence of Makefile.PL implies. By convention, CPAN modules follow a standard directory hierarchy: $ ls -l total 52 drwxr-xr-x 2 chromatic wheel 4096 Jan 20 09:59 bin -rw-rr-- 1 chromatic wheel 19110 Jan 20 09:51 Changes drwxr-xr-x 2 chromatic wheel 4096 Jan 20 09:59 examples drwxr-xr-x 3 chromatic wheel 4096 Jan 20 09:59 lib -rw-rr-- 1 chromatic wheel 950 Dec 31 13:28 Makefile.PL -rw-rr-- 1 chromatic wheel 1262 Dec 31 13:28 MANIFEST -rw-rr-- 1 chromatic wheel 347 Jan 20 09:49 META.yml -rw-rr-- 1 chromatic wheel 434 Dec 31 13:28 NOTES drwxr-xr-x 4 chromatic wheel 4096 Jan 20 09:59 t
Note: The -R flag causes ls to recurse into subdirectories, listing all of their files .
The modules themselves live in various subdirectories under the lib/ directory: $ ls -lR lib/ lib: total 4 drwxr-xr-x 3 chromatic wheel 4096 Jan 20 09:59 Test lib/Test: total 36 drwxr-xr-x 2 chromatic wheel 4096 Jan 20 09:59 Harness -rw-rr-- 1 chromatic wheel 29682 Jan 20 09:35 Harness.pm lib/Test/Harness: total 36 -rw-rr-- 1 chromatic wheel 958 Dec 31 13:28 Assert.pm -rw-rr-- 1 chromatic wheel 1230 Dec 31 13:28 Iterator.pm -rw-rr-- 1 chromatic wheel 18375 Dec 31 13:28 Straps.pm -rw-rr-- 1 chromatic wheel 5206 Dec 31 13:28 TAP.pod
All of the test files live under the t/ directory: $ ls -l t/ total 112 -rw-r--r-- 1 chromatic wheel 541 Dec 31 13:28 00compile.t -rw-r--r-- 1 chromatic wheel 656 Dec 31 13:28 assert.t -rw-r--r-- 1 chromatic wheel 198 Dec 31 13:28 base.t -rw-r--r-- 1 chromatic wheel 2280 Dec 31 13:28 callback.t -rw-r--r-- 1 chromatic wheel 328 Dec 31 13:28 harness.t -rw-r--r-- 1 chromatic wheel 539 Dec 31 13:28 inc_taint.t drwxr-xr-x 4 chromatic wheel 4096 Jan 20 09:59 lib -rw-r--r-- 1 chromatic wheel 151 Dec 31 13:28 nonumbers.t -rw-r--r-- 1 chromatic wheel 71 Dec 31 13:28 ok.t -rw-r--r-- 1 chromatic wheel 275 Dec 31 13:28 pod.t -rw-r--r-- 1 chromatic wheel 755 Dec 31 13:28 prove-globbing.t -rw-r--r-- 1 chromatic wheel 2143 Dec 31 13:28 prove-switches.t drwxr-xr-x 2 chromatic wheel 4096 Jan 20 09:59 sample-tests -rw-r--r-- 1 chromatic wheel 17301 Dec 31 13:28 strap-analyze.t -rw-r--r-- 1 chromatic wheel 8913 Dec 31 13:28 strap.t -rw-r--r-- 1 chromatic wheel 26307 Dec 31 13:28 test-harness.t
Note: This is output from a Unix-like system. It will look different on other platforms .
Running Makefile.PL or Build.PL (in the case of Module::Build ) writes out either a Makefile or a Build file, respectively, that knows how to build the module and its documentation as well as how to run the tests. The default behavior is to run everything in the t/ directory that ends in .t . The full command that make test ran earlier shows more details: PERL_DL_NONLAZY=1 /usr/bin/perl5.8.6 "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
The most important part of this command is the shell pattern at the end, t/*.t . The shell expands it to include all of the files in the t/ directory in sorted order. If you've never installed this module before, how can the tests find the module files? The preceding command-line invocation includes the blib/ subdirectories created during the make stage. Tests can also include a little magic at the beginning to set up their working environment appropriately: BEGIN { chdir 't' if -d 't' } use lib '../lib'; use blib;
Note: You can omit the blib line if you have pure-Perl modules that rely on nothing tricky during the building process .
The contents of the BEGIN block change the current directory to the t/ directory immediately after Perl encounters it. This is important for the next command, which loads the lib module to add the ../lib directory (a sibling of t/ ) to @INC . Finally, the blib module adds the blib/lib and blib/arch directories to @INC . All together, this set of commands allows you to run your tests with perl itself, not just prove , make test , or perl Build test . As long as you follow the convention of storing modules under lib/ and tests under t/ and add the appropriate path manipulations to the start of the test files, you can run and distribute your tests automatically with the standard Perl tools. What about... | Q: | How can I run tests in a specific order? | | A: | Both ExtUtils::MakeMaker and Module::Build run tests in a predictable order ( alphabetically , with numbers coming before names ). You can control this order yourself by prepending numbers to the test names. For example, 00-first.t will run before 99-last.t .
Note: If you need even more customization, subclass Module::Build to override the ACTION_test() method. It's painful to override ExtUtils:: MakeMaker behavior, so avoid it if possible .
If a directory full of flat files isn't enough organization for you, you can put your tests in as many subdirectories of t/ as you like. Remember to tell your build process about the change, though! See the test attribute for Makefile.PL in the ExtUtils::MakeMaker documentation or the test_files parameter for Build.PL in the Module::Build documentation. | | Q: | Do I need that magic BEGIN block? It looks complicated. | | A: | Not all tests need it. It's useful if you need to know that you're in a specific directory ”to create temporary files under t/ or to load testing modules from t/lib/ , for example. If your test file does neither , you can safely omit it. | | 