Using Shared Libraries

Most modern operating systems use shared libraries, also called dynamic libraries. These libraries are not linked into a program at compile time but rather are loaded when the program starts (or later in some cases). The names of files housing shared libraries have filename extensions of .so (shared object) for example libc.so. Usually libaaa.so is a symbolic link to libaaa.so.x, where x is a small number representing the version of the library. Many of these libraries are kept in /usr/lib: A typical Linux installation has more than 300 shared libraries in /usr/lib and more than 30 in /usr/X11R6/lib. Applications can have their own shared libraries. For example, the gcc compiler might keep its libraries in /usr/lib/gcc-lib/i386-redhat-linux/3.4.0.

Archived libraries

In contrast to shared libraries are the older, statically linked libraries (with a .a filename extension), also called archived libraries. Archived libraries are added to the executable file during the last (link) phase of compilation. This addition can make a program run slightly faster the first time it is run, albeit at the expense of program maintainability and size. Taken together, the combined size of several executables that use a shared library and the size of the shared library are smaller than the combined size of the same executables with static libraries. When a running program has already loaded a dynamic library, a second program that requires the same dynamic library starts slightly faster.

Reducing memory usage and increasing maintainability are the primary reasons for using shared object libraries; they have largely replaced statically linked libraries as the library type of choice. Consider what happens when you discover an error in a library. With a static library, you need to relink every program that uses the library once the library has been fixed and recompiled. With a dynamic library, you need to fix and recompile only the library itself.

Shared object libraries also make dynamic loading of program libraries on the fly possible (for example, perl, python, and tcl extensions and modules). The Apache (HTTP) Web server specifies modules in the httpd.conf file and loads them as needed.

ldd

The ldd (list dynamic dependencies) utility tells you which shared libraries a program needs. The following example shows that cp uses libacl, the Access Control Lists library; libc, the C library; libattr, the Extended Attributes library; and ld-linux, the runtime linker:

 $ ldd /bin/cp         libacl.so.1 => /lib/libacl.so.1 (0x40026000)         libc.so.6 => /lib/i686/libc.so.6 (0x42000000)         libattr.so.1 => /lib/libattr.so.1 (0x4002d000)         /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x40000000) 

Running ldd on /usr/bin/gnome-session (a program that starts a graphical GNOME session) lists 59 libraries from /usr/lib, /usr/X11R6/lib, and /lib.

The program that does the dynamic runtime linking, ld-linux.so, always looks in /usr/lib for libraries. The other directories that ld searches vary depending on how ld is set up. You can add directories for ld to look in by specifying a search path at compile (actually link) time, using the r option followed by a colon-separated list of directories (do not put a SPACE after r). Use only absolute pathnames in the search path. Although you use this option on the gcc command line, it is passed to the linker (ld). The gnome-session desktop manager was likely linked with a command such as the following:

 gcc flags  o gnome-session objects  r/lib:/usr/X11R6/lib libraries 

This command line allows ld.so (and ldd) to search /lib and /usr/X11R6/lib in addition to the standard /usr/lib for the libraries needed by the executable.

The compiler needs to see the shared libraries at link time to make sure that the needed functions and procedures are present as promised by the header (.h) files. Use the L option to tell the compile-time linker to look in the directory mylib for shared or static libraries: L mylib. Unlike the search path, L can use relative pathnames such as L ../lib handy when a program builds its own shared library. The library can be in one location at build time ( L) but in another location at runtime after it is installed ( rpath). The SPACE after L is optional and is usually omitted; r must not be followed by a SPACE. You can repeat the L and the r options multiple times on the link line.

Fixing Broken Binaries

The command line search path is a fairly new idea. The search path was traditionally created by using the LD_LIBRARY_PATH and, more recently, the LD_RUN_PATH environment variables. These variables have the same format as PATH (page 284).

The directories in LD_LIBRARY_PATH are normally searched before the usual library locations. Newer Linux releases extend the function of LD_LIBRARY_PATH to specify directories to be searched either before or after the normal locations. See the ld man page for details. The LD_RUN_PATH variable behaves similarly to LD_LIBRARY_PATH. If you use r, however, LD_LIBRARY_PATH supersedes anything in LD_RUN_PATH.

The use of LD_LIBRARY_PATH brings up several problems. Because only one environment variable exists, it must be shared among all programs. If two programs have the same name for a library or use different, incompatible versions of the same library, only the first will be found. As a result one of the programs will not run or worse will not run correctly.

security: LD_LIBRARY_PATH

Under certain circumstances a malicious user can create a Trojan horse named libc.so and place it in a directory that is searched before /usr/lib (any directory in LD_LIBRARY_PATH, which appears before /usr/lib). The fake libc will then be used instead of the real libc.

Wrappers

LD_LIBRARY_PATH still has its place in the scripts, called wrappers, that are used to fix broken binaries. Suppose that the broken binary bb uses the shared library libbb.so, which you want to put in /opt/bb/lib and not in /usr/lib, as the bb programmer requested. The command ldd bb will tell you which libraries are missing. Not a problem: Rename bb to bb.broken, and create a /bin/sh wrapper named bb.

 #!/bin/sh LD_LIBRARY_PATH=/opt/bb/lib export LD_LIBRARY_PATH exec bb.broken "$@" 

(Using $@ rather than $* preserves SPACEs in the parameters; see page 482.) A wrapper can also allow you to install programs in arbitrary locations.

Creating Shared Libraries

Building a dynamically loadable shared library is not a trivial matter: It involves using reentrant function calls, defining a library entrance routine, and performing other tasks. When you want to create a shared object library, you must at a minimum compile the source files with the fPIC (position-independent code) option to gcc and link the resulting object files into the libxx.so file using the shared x options to the linker (for example, ld shared x o libmylib.so *.o). The best resource for investigating shared library construction and usage is existing code on the Internet. For example, you can look at the source files for zlib (www.gzip.org/zlib).

C+ +

C+ + files have special needs, and libraries (shared or not) often have to be made by the compiler rather than ld or ar. Shared libraries can depend on other shared libraries and have their own search paths. If you set LD_LIBRARY_PATH, add the i flag to the link phase when compiling to ignore the current LD_LIBRARY_PATH or you may have unexpected results. Ideally, you would not have LD_LIBRARY_PATH set on a global level but would use it only in wrappers as needed.