XSUB Techniques

Chapter 9 - Writing C Modules with XS
by?Sam Tregar?
Apress ? 2002
Companion Web Site

The XSUB shown earlier is about as simple as an XSUB can be. However, there are many useful changes that can be made to enhance the usability and functionality of XSUBs.

Types and Typemaps

You might have noticed that the types used in the gnome_mime_type() XSUB are subtly different from those used in the actual function signature. In the gnome_mime_type() function, the return type is a const char * and the filename argument type is const gchar *, whereas the XSUB used char * for both. This was done because XS comes with a typemap for char *, but doesn't know anything about const char * and const gchar *. A typemap is a description of how to map Perl types to and from C types.

If you try to use the real types in an XSUB as follows:

 const char * gnome_mime_type(filename)      const gchar *filename 

you'll receive this compilation error:

 Error: 'const gchar *' not in typemap in MIME.xs, line 10 Error: 'const char *' not in typemap in MIME.xs, line 10 

To use types that XS doesn't natively support, you need to create a new file called typemap in your module directory that contains code to translate to and from the new types. In this case, only two lines are required:

 const char  *    T_PV const gchar *    T_PV 

This tells XS that const char * and const gchar * are both to be treated as T_PV, which is the supplied typemap for char *. The T_PV typemap is defined in the system-wide typemap file installed with Perl. You can find it in under the module directory for the ExtUtils modules in your Perl library. I'll explore typemaps in more detail later in this chapter.

The preceding code still doesn't work, though; it produces a syntax error because it doesn't recognize Gnome's gchar type. To fix this problem, I need to add an #include line below the three #includes already in the file:

 #include "EXTERN.h" #include "perl.h" #include "XSUB.h" #include <gnome.h> 

This is part of the section before the MODULE command that is passed through to the generated MIME.c file verbatim.

Modifying the XSUB Name with PREFIX

Calling gnome_mime_type() from Perl with the XSUB shown earlier is done using a line that looks like this one:

 my $type = Gnome::MIME::gnome_mime_type($filename); 

This works just fine, but it's awfully verbose; the user is forced to type the words "Gnome" and "MIME" twice. One solution would be to export the function as-is, but XS offers a simpler solution. By modifying the MODULE line, shown here:

 MODULE = Gnome::MIME    PACKAGE = Gnome::MIME 

to include a new directive, PREFIX:

 MODULE = Gnome::MIME PACKAGE = Gnome::MIME PREFIX = gnome_mime_ 

XS will automatically remove the specified prefix, gnome_mime_, from the front of the Perl interface. After this change, test.pl needs changes to use the new interface:

 # test some simple mime-type recognitions is(Gnome::MIME::type("foo.gif"),  'image/gif',  "recognizes .gif"); is(Gnome::MIME::type("foo.jpg"),  'image/jpeg', "recognizes .jpg"); is(Gnome::MIME::type("foo.html"), 'text/html',  "recognizes .html"); 

Writing Your Own CODE

Changing the XSUB name from gnome_mime_type() to type() with PREFIX is certainly an improvement, but it isn't very flexible. Modifying the XSUB name beyond removing a fixed prefix will require a new technique-writing the actual code to call the underlying C function with the CODE keyword. For example, to rename the XSUB to file_type(), I could use this XS code:

 const char * file_type(filename)      const gchar * filename CODE:      RETVAL = gnome_mime_type(filename); OUTPUT:      RETVAL 

This example shows a new wrinkle in the XS syntax: keywords that come after the function definition and declare blocks of C code. This is a pattern that you'll see repeated by most of XS. In this case, two keywords are used, CODE and OUTPUT. The CODE keyword allows you to override the default XSUB call with your own custom call.

The CODE block shown previously makes use of the automatic RETVAL variable. RETVAL is an automatic variable with the same type as the return type in the XSUB definition. In this case, RETVAL is a const char * variable. The CODE block simply calls gnome_mime_type() and places the return value in RETVAL.

The OUTPUT block tells xsubpp which variable (or variables) should be returned back to Perl. In most cases, your CODE blocks will be immediately followed by an OUTPUT block exactly like the one shown earlier.

After this change, the tests would need to be updated to reflect the new function name, but underneath the call is still going to gnome_mime_type():

 # test some simple mime-type recognitions is(Gnome::MIME::file_type("foo.gif"),  'image/gif',  "recognizes .gif"); is(Gnome::MIME::file_type("foo.jpg"),  'image/jpeg', "recognizes .jpg"); is(Gnome::MIME::file_type("foo.html"), 'text/html',  "recognizes .html"); 

Managing Memory Usage

As it turns out, the preceding XSUB does not leak memory. This came as a surprise to me-I assumed that the gnome_mime_type() function returned a dynamically allocated string that I would need to clean up. If you look at the generated code in MIME.c (Listing 9-5), you'll see this line at the end of the function:

 sv_setpv(TARG, RETVAL); XSprePUSH; PUSHTARG; 

This line copies the string pointed to by RETVAL into TARG. TARG is the SV that will actually be returned by the subroutine. After that, TARG is pushed onto the stack and the function returns. My expectation was that this would result in a leak since the pointer stored in RETVAL wouldn't be freed before going out of scope. As it turns out, this pointer doesn't need to be freed because it comes from an internal pool managed by the Gnome API.

But, for the sake of the example, what would I need to do if the return value from gnome_mime_type() did need to be freed? My first draft might have been as follows:

 const char * file_type(filename)      const gchar * filename CODE:      RETVAL = gnome_mime_type(filename); OUTPUT:      RETVAL CLEANUP:      Safefree(RETVAL); 

The CLEANUP block specifies code to be run at the end of the generated function. This might work fine, but it's incorrect. The problem is that Gnome and Perl might be using different memory allocators. Thus, calling Perl's Safefree() function on memory allocated by Gnome is not guaranteed to yield the expected results. Instead, I would need to use the same call that Gnome uses, g_free():

 CLEANUP:      g_free(RETVAL); 

The moral of this story is that managing memory usage in C modules is rarely simple. It requires you to think carefully about the way the underlying library allocates memory. Often the only way to get some of the information you need is to test. For example, the only way I could find out that gnome_mime_type() doesn't dynamically allocate its return value was to run my wrapped version in a loop and watch the system with top in another window:

 $ perl -Mblib -MGnome::MIME -e 'while(1) { Gnome::MIME::file_type("foo.gif"); }' 

It's a good idea to do this sort of testing on all your XS functions-at least until someone finds a way to write ExtUtils::LeakDetector!