Hack 33. Presolve Module Paths

Make programs on complex installations start more quickly.

In certain circumstances, one of Perl's major strengths can be a weakness. Even though you can manipulate where Perl looks for modules (@INC) at runtime according to your needs [Hack #29], and even though you can use thousands of modules from the CPAN, your system has to find and load these modules.

For a short-running, repeated program, this can be expensive, especially if you have many paths in @INC from custom testing paths, sitewide paths, staging servers, business-wide repositories, and the like. Fortunately, there's more than one way to solve this. One approach is to resolve all of the paths just once, and then use your program as normal.

The Hack

"Trace All Used Modules" [Hack #74] shows how putting a code reference into @INC allows you to execute code every time you use or require a module. That works here, too.

package Devel::Presolve; use strict; use warnings; my @track; BEGIN { unshift @INC, \\&resolve_path } sub resolve_path {     my ($code, $module) = @_;     push @track, $module;     return; } INIT {     print "BEGIN\\n{\\n";     for my $tracked (@track)     {         print "\\trequire( \\$INC{'$tracked'} = '$INC{$tracked}' );\\n";     }     print "}\\n1;\\n";     exit; } 1;

Devel::Presolve's resolve_path( ) captures every request to load a module, stores the module name, and returns. Thus Perl attempts to load the module as normal. After the entire program has finished compiling, but before it starts to run [Hack #70], it prints to STDOUT a BEGIN block that loads all of the modules by absolute filepath then exits the program.

Running the Hack

Put Devel::Presolve somewhere in your path. Then run your slow-starting program while loading the module. Redirect the output to a file of your choosing:

$ perl -MDevel::Preload slow_program.pl > preload.pm             

preload.pm will contain something similar to:

BEGIN {     require( $INC{'CGI.pm'}      = '/usr/lib/perl5/5.8.7/CGI.pm' );     require( $INC{'CGI/Util.pm'} = '/usr/lib/perl5/5.8.7/CGI/Util.pm' );     require( $INC{'vars.pm'}     = '/usr/lib/perl5/5.8.7/vars.pm' );     require( $INC{'constant.pm'} = '/usr/lib/perl5/5.8.7/constant.pm' );     require( $INC{'overload.pm'} = '/usr/lib/perl5/5.8.7/overload.pm' ); } 1;

You can either include the contents of this file at the start of slow_program.pl or load it as the first module. If you do the latter, put the file in a directory at the front of @INC, lest you erase any performance gains.

Note that the trick of assigning to %INC within the require avoids a potentially nasty module-reloading bug, where Perl doesn't see require '/usr/lib/perl5/5.8.7./CGI.pm' as loading the same file as use CGI; does.

Hacking the Hack

Pre-resolving paths likely won't help long-running programs. For short-running programs where startup time can dwarf calculation time, it may, depending on how complex your @INC is. Be especially careful that upgrading Perl or installing new versions of modules may invalidate this cacheit is a cacheand cause strange errors. This technique may work better only when you want to deploy a program to a production system, but likely not when you're merely developing or testing.