Using Shared Libraries

Most modern operating systems use shared libraries, also called dynamic libraries. These libraries are not linked into a program at compile time but rather are loaded when the program starts (or later in some cases). The names of files housing shared libraries have filename extensions of .dylib (dynamic library)for example, libc.dylib. Sometimes libaaa.dylib is a symbolic link to libaaa.dylib.x, where x is a small number representing the version of the library. Many of these libraries are kept in /usr/lib: A typical Mac OS X installation has more than 150 shared libraries in /usr/lib and more than 30 in /usr/X11R6/lib. Applications can have their own shared libraries. For example, the gcc compiler might keep its libraries in /usr/lib/gcc-lib/powerpc-apple-darwin8/4.0.0.

Archived libraries

In contrast to shared libraries are the older, statically linked libraries (with a .a filename extension), also called archived libraries. Archived libraries are added to the executable file during the last (linking) phase of compilation. Their addition can make a program run slightly faster the first time it is run, albeit at the expense of program maintainability and size. Taken together, the combined size of several executables that use a shared library and the size of the shared library are smaller than the combined size of the same executables with static libraries. When a running program has already loaded a dynamic library, a second program that requires the same dynamic library starts slightly faster.

Reducing memory usage and increasing maintainability are the primary reasons for using shared object libraries; thanks to these advantages, they have largely replaced statically linked libraries as the library type of choice. Consider what happens when you discover an error in a library. With a static library, you need to relink every program that uses the library once the library has been fixed and recompiled. With a dynamic library, you need to fix and recompile only the library itself.

Shared object libraries also make dynamic loading of program libraries on the fly possible (for example, perl, python, and tcl extensions and modules). The Apache (HTTP) Web server specifies modules in the httpd.conf file and loads them as needed.

otool

The otool (object tool) utility with the L option tells you which shared libraries a program needs. The following example shows that cp uses libSystem.B.dylib, one of the system libraries:

$ otool -L /bin/cp /bin/cp:         /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.0.0)

Refer to page 805 for more information on otool.

The compiler needs to see the shared libraries at link time to make sure that the needed functions and procedures are present as promised by the header (.h) files. Use the L option to tell the compile-time linker to look in the directory mylib for shared or static libraries: L mylib. Unlike the search path, L can use relative pathnames such as L ../liba handy feature when a program builds its own shared library. Each shared library contains a pathname telling the linker where to look for it; this path is recorded in the executable. When the program is run, dyld (the dynamic link editor) looks for each library in the recorded location. If dyld does not find a library in that location, dyld searches for it; see "Fixing Broken Binaries" on page 488. You can repeat the L option multiple times on the link line.

Frameworks

Mac OS X uses frameworks, which are similar to shared libraries. Frameworks differ from shared libraries in that they can contain resources, such as headers, in addition to code. Shared libraries are typically found in /usr/lib, while frameworks are usually found in /System/Library/Frameworks. Frameworks are specified with the framework option to the compiler; shared libraries are specified with l.

Fixing Broken Binaries

The command line search path is a fairly new idea. The search path was traditionally specified by the DYLD_LIBRARY_PATH environment variable. This variable has the same format as PATH (page 285). The directories in DYLD_LIBRARY_PATH are normally searched before the usual library locations. A number of other variables can affect the behavior of dyld. See the dyld man page for details.

The use of DYLD_LIBRARY_PATH brings up several problems. Because only one environment variable exists, it must be shared among all programs. If two programs have the same name for a library or use different, incompatible versions of the same library, only the first will be found. As a result one of the programs will not run oreven worsewill not run correctly.

Security: DYLD_LIBRARY_PATH

Under certain circumstances a malicious user can create a Trojan horse named libc.dylib and place it in a directory that is searched before /usr/lib (any directory that appears in DYLD_LIBRARY_PATH before /usr/lib). The fake libc will then be used instead of the real libc.

Wrappers

DYLD_LIBRARY_PATH still has its place in the scripts, called wrappers, that are used to fix broken binaries. Suppose that the broken binary bb uses the shared library libbb.so, which you want to put in /opt/bb/lib and not in /usr/lib, as the bb programmer requested. The command otool L bb will tell you which libraries are missing. Not a problem: Rename bb to bb.broken, and create a /bin/sh wrapper named bb.

$ cat bb #!/bin/sh DYLD_LIBRARY_PATH=/opt/bb/lib export DYLD_LIBRARY_PATH exec bb.broken "$@"

(Using $@ rather than $* preserves SPACEs in the parameters; see page 566.) A wrapper can also allow you to install programs in arbitrary locations.

If dyld does not find a library anywhere in DYLD_LIBRARY_PATH, dyld searches DYLD_FALLBACK_LIBRARY_PATH ($HOME/lib:/usr/local/lib:/lib:/usr/lib is the default). Frameworks are searched for in DYLD_FRAMEWORK_PATH and DYLD_FALLBACK_FRAMEWORK_PATH.

Creating Shared Libraries

Building a dynamically loadable shared library is not a trivial matter: It involves setting up reentrant function calls, defining a library entrance routine, and performing other tasks. When you want to create a shared object library, you must compile the source files with the dynamiclib option to gcc. Shared libraries can depend on other shared libraries. The best resource for investigating shared library construction and usage is existing code on the Internet. For example, you can look at the source files for zlib at www.gzip.org/zlib.

C++

C++ files have special needs, and libraries (shared or not) often have to be made by the compiler rather than ld or ar.