Section 12.4. Upgrading Software Not Provided in Packages

12.4. Upgrading Software Not Provided in Packages

A lot of interesting software is offered outside the package systems, although as it becomes popular, the developers tend to provide Linux packages. In order to install or upgrade applications that don't yet exist as packages, you'll have to obtain the newest release of the software. This is usually available as a gzipped or compressed tar file. Such a package could come in several forms. The most common are binary distributions, in which the binaries and related files are archived and ready to unpack on your system, and source distributions, in which the source code (or portions of the source code) for the software is provided, and you have to issue commands to compile and install it on your system.

Shared libraries make distributing software in binary form easy; as long as you have a version of the libraries installed that is compatible with the library stubs used to build the program, you're set. However, in many cases, it is easier (and a good idea) to release a program as source . Not only does this make the source code available to you for inspection and further development, but it also allows you to build the application specifically for your system, with your own libraries. Many programs allow you to specify certain options at compile time, such as selectively including various features in the program when built. This kind of customization isn't possible if you get prebuilt binaries.

There's also a security issue at play when installing binaries without source code. Although on Unix systems viruses are nearly unheard of, it's not difficult to write a Trojan Horse, a program that appears to do something useful but, in actuality, causes damage to the system.^[*] For example, someone could write an application that includes the "feature" of deleting all files in the home directory of the user executing the program. Because the program would be running with the permissions of the user executing it, the program itself would have the ability to do this kind of damage. (Of course, the Unix security mechanism prevents damage being done to other users' files or to any important system files owned by root.)

^[*] A virus in the classic sense is a program that attaches to a "host" and runs when the host is executed. On Unix systems, this usually requires root privileges to do any harm, and if programmers could obtain such privileges, they probably wouldn't bother with a virus.

Although having source code won't necessarily prevent this from happening (do you read the source code for every program you compile on your system?), at least it gives you a way to verify what the program is really doing. Also, if source code is available, it is likely that some people will peruse it, so that using source is a bit safer; however, you can't count on that.

There are techniques for verifying binary packages as well, namely signed packages. The packager can sign a package with his PGP key, and package tools such as RPM have means of verifying such a signature. However, you will still have to rely on the packager having packaged correctly and without bad intentions. All the signature tells you is that the package really comes from who it says it comes from, and that it hasn't been tampered with on its way from the packager to your hard disk.

At any rate, dealing with source and binary distributions of software is quite simple. If the package is released as a tar file, first use the tar t option to determine how the files have been archived. In the case of binary distributions, you may be able to unpack the tar file directly on your systemsay, from / or /usr. When doing this, be sure to delete any old versions of the program and its support files (those that aren't overwritten by the new tar file). If the old executable comes before the new one on your path, you'll continue to run the old version unless you remove it.

Source distributions are a bit trickier. First, you must unpack the sources into a directory of their own. Most systems use /usr/src for just this. Because you usually don't have to be root to build a software package (although you will usually require root permissions to install the program once compiled!), it might be a good idea to make /usr/src writable by all users, with the command:

     chmod 1777 /usr/src

This allows any user to create subdirectories in /usr/src and place files there. The first 1 in the mode is the "sticky" bit, which prevents users from deleting each other's subdirectories.

You can now create a subdirectory under /usr/src and unpack the tar file there, or you can unpack the tar file directly from /usr/src if the archive contains a subdirectory of its own.

Once the sources are available, the next step is to read any README and INSTALL files or installation notes included with the sources. Nearly all packages include such documentation. The basic method used to build most programs is as follows:

Check the Makefile. This file contains instructions for make, which controls the compiler to build programs. Many applications require you to edit minor aspects of the Makefile for your own system; this should be self-explanatory. The installation notes will tell you if you have to do this. If there is no Makefile in the package, you might have to generate it first. See item 3 for how to do this.
Possibly edit other files associated with the program. Some applications require you to edit a file named config.h; again, this will be explained in the installation instructions.
Possibly run a configuration script. Such a script is used to determine what facilities are available on your system, which is necessary to build more complex applications.
Specifically, when the sources do not contain a Makefile in the top-level directory, but instead a file called Makefile.in and a file called configure, the package has been built with the Autoconf system. In this (more and more common) case, you run the configuration script like this:
```
     ./configure
```
The ./ should be used so that the local configure is run, and not another configure program that might accidentally be in your path. Some packages let you pass options to configure that often enable or disable specific features of the package. (You can find out what these options are with configure --help.) Once the configure script has run, you can proceed with the next step.
Run make. Generally, this executes the appropriate compilation commands as given in the Makefile. In many cases you'll have to give a "target" to make, as in make all or make install. These are two common targets; the former is usually not necessary but can be used to build all targets listed in a Makefile (e.g., if the package includes several programs, but only one is compiled by default); the latter is often used to install the executables and support files on the system after compilation. For this reason, make install is usually run as root.
Even after the installation, there is often one major difference between programs installed from source or from a binary package. Programs installed from source are often installed below /usr/local by default, which is rarely the case with binary packages.

You might have problems compiling or installing new software on your system, especially if the program in question hasn't been tested under Linux, or depends on other software you don't have installed. In Chapter 21, we talk about the compiler, make, and related tools in detail.

Most software packages include manual pages and other files, in addition to the source and executables. The installation script (if there is one) will place these files in the appropriate location. In the case of manual pages, you'll find files with names such as foobar.1 or foobar.man. These files are usually nroff source files, which are formatted to produce the human-readable pages displayed by the man command. If the manual page source has a numeric extension, such as .1, copy it to the directory /usr/man/man1, where 1 is the number used in the filename extension. (This corresponds to the manual "section" number; for most user programs, it is 1.) If the file has an extension such as .man, it usually suffices to copy the file to /usr/man/man1, renaming the .man extension to .1.

12.4.1. Upgrading Libraries

Most of the programs on a Linux system are compiled to use shared libraries . These libraries contain useful functions common to many programs. Instead of storing a copy of these routines in each program that calls them, the libraries are contained in files on the system that are read by all programs at runtime. That is, when a program is executed, the code from the program file itself is read, followed by any routines from the shared library files. This saves a great deal of disk spaceonly one copy of the library routines is stored on disk.

If you're lucky, using the package system means that the right versions of the libraries each program needs are installed along with the programs. The package system is supposed to be aware of dependencies on shared libraries. But because different programs may depend on different versions of libraries, or because you might install a program without using the package system, you occasionally have to understand the conventions for libraries explained in this section.

In some instances, it's necessary to compile a program to have its own copy of the library routines (usually for debugging) instead of using the routines from the shared libraries. We say that programs built in this way are statically linked, whereas programs built to use shared libraries are dynamically linked.

Therefore, dynamically linked executables depend upon the presence of the shared libraries on disk. Shared libraries are implemented in such a way that the programs compiled to use them generally don't depend on the version of the available libraries. This means that you can upgrade your shared libraries, and all programs that are built to use those libraries will automatically use the new routines. (There is an exception: if major changes are made to a library, the old programs won't work with the new library. You'll know this is the case because the major version number is different; we explain more later. In this case, you keep both the old and new libraries around. All your old executables will continue to use the old libraries, and any new programs that are compiled will use the new libraries.)

When you build a program to use shared libraries, a piece of code is added to the program that causes it to execute ld.so, the dynamic linker, when the program is started. ld.so is responsible for finding the shared libraries the program needs and loading the routines into memory. Dynamically linked programs are also linked against "stub" routines, which simply take the place of the actual shared library routines in the executable. ld.so replaces the stub routine with the code from the libraries when the program is executed.

The ldd command can be used to list the shared libraries on which a given executable depends. For example:

     rutabaga$ ldd /usr/bin/X11/xterm             linux-gate.so.1 =>  (0xffffe000)             libXft.so.2 => /usr/X11R6/lib/libXft.so.2 (0x40037000)             libfontconfig.so.1 => /usr/lib/libfontconfig.so.1 (0x4004b000)             libfreetype.so.6 => /usr/lib/libfreetype.so.6 (0x40079000)             libexpat.so.0 => /usr/lib/libexpat.so.0 (0x400e8000)             libXrender.so.1 => /usr/X11R6/lib/libXrender.so.1 (0x40107000)             libXaw.so.8 => /usr/X11R6/lib/libXaw.so.8 (0x4010f000)             libXmu.so.6 => /usr/X11R6/lib/libXmu.so.6 (0x4016b000)             libXt.so.6 => /usr/X11R6/lib/libXt.so.6 (0x40182000)             libSM.so.6 => /usr/X11R6/lib/libSM.so.6 (0x401d5000)             libICE.so.6 => /usr/X11R6/lib/libICE.so.6 (0x401dd000)             libXpm.so.4 => /usr/X11R6/lib/libXpm.so.4 (0x401f5000)             libXp.so.6 => /usr/X11R6/lib/libXp.so.6 (0x40205000)             libXext.so.6 => /usr/X11R6/lib/libXext.so.6 (0x4020d000)             libX11.so.6 => /usr/X11R6/lib/libX11.so.6 (0x4021c000)             libncurses.so.5 => /lib/libncurses.so.5 (0x40318000)             libutempter.so.0 => /usr/lib/libutempter.so.0 (0x4035d000)             libc.so.6 => /lib/tls/libc.so.6 (0x4035f000)             libdl.so.2 => /lib/libdl.so.2 (0x40478000)             /lib/ld-linux.so.2 (0x40000000)

Here, we see that the xterm program depends on a number of shared libraries, including libXaw, libXt, libX11, and libc. (The libraries starting with libX as well as libSM and libICE are all related to the X Window System; libc is the standard C library.) We also see the version numbers of the libraries for which the program was compiled (that is, the version of the stub routines used), and the name of the file that contains each shared library. This is the file that ld.so will find when the program is executed. The first file in the list, by the way, linux-gate.so.1, is not a real shared library, but rather a so-called dynamic shared object provided by the kernel, a technicality that speeds up system calls into the kernel and provides other useful low-level things.

To use a shared library, the version of the stub routines (in the executable) must be compatible with the version of the shared libraries. Basically, a library is compatible if its major version number matches that of the stub routines. The major version number is the part right after the .so. In this case, libX11 (the most basic library used by the X Window System) is used with the major Version 6. The library file libX11.so.6 (which usually resides in /usr/X11R6/lib) might very well just be a symbolic linkfor example, to libX11.so.6.2. This means that the library has the major version number 6 and the minor version number 2. Library versions with the same major version number are supposed to be interchangeable. This way, if a program was compiled with Version 6.0 of the stub routines, shared library Versions 6.1, 6.2, and so forth could be used by the executable. If a new version with the major version number 6 and the minor version number 3 were released (and thus had the filename libX11.so.6.3), all you would need to do to use this new version is change the symbolic link libX11.so.6 to point to the new version. The xterm executable would then automatically benefit from any bug fixes or similar that are included in the new version. In "More Fun with Libraries" in Chapter 21, we describe how to use shared libraries with your own programs.

The file /etc/ld.so.conf contains a list of directories that ld.so searches to find shared library files. An example of such a file is:

     /usr/lib     /usr/local/lib     /usr/X11R6/lib     /opt/kde3/lib

ld.so always looks in /lib and /usr/lib, regardless of the contents of ld.so.conf. Usually, there's no reason to modify this file, and the environment variable LD_LIBRARY_PATH can add additional directories to this search path (e.g., if you have your own private shared libraries that shouldn't be used system-wide). However, if you do add entries to /etc/ld.so.conf or upgrade or install additional libraries on your system, be sure to use the ldconfig command, which will regenerate the shared library cache in /etc/ld.so.cache from the ld.so search path. This cache is used by ld.so to find libraries quickly at runtime without actually having to search the directories on its path. For more information, check the manual pages for ld.so and ldconfig.

Now that you understand how shared libraries are used, let's move on to upgrading them. The two libraries that are most commonly updated are libc (the standard C library) and libm (the math library). Because naming is a little bit special for these, we will look at another library here, namely libncurses, which "emulates" a graphical windowing system on the text console.

For each shared library, there are two separate files:

<library>.a: This is the static version of the library. When a program is statically linked, routines are copied from this file directly into the executable, so the executable contains its own copy of the library routines.^[*]
^[*] On some distributions, the static versions of the libraries are moved into a separate package and not necessarily installed by default. If this is the case, you won't find the .a files unless you install them.
< library>.so.<version>: This is the shared library image itself. When a program is dynamically linked, the stub routines from this file are copied into the executable, allowing ld.so to locate the shared library at runtime. When the program is executed, ld.so copies routines from the shared library into memory for use by the program. If a program is dynamically linked, the <library>.a file is not used for this library.

For the libncurses library, you'll have files such as libncurses.a and libncurses.so.5.4. The .a files are generally kept in /usr/lib, and .so files are kept in /lib. When you compile a program, either the .a or the .so file is used for linking, and the compiler looks in /lib and /usr/lib (as well as a variety of other places) by default. If you have your own libraries, you can keep these files anywhere, and control where the linker looks with the -L option to the compiler. See "More Fun with Libraries" in Chapter 21 for details.

The shared library image, <library>.so.version, is kept in /lib for most system-wide libraries. Shared library images can be found in any of the directories that ld.so searches at runtime; these include /lib, /usr/lib, and the files listed in ld.so.conf. See the ld.so manual page for details.

If you look in /lib, you'll see a collection of files such as the following:

     lrwxrwxrwx    1 root     root           17 Jul 11 06:45 /lib/libncurses.so.5 \         -> libncurses.so.5.4     -rwxr-xr-x    1 root     root       319472 Jul 11 06:45 /lib/libncurses.so.5.4     lrwxrwxrwx    1 root     root           13 Jul 11 06:45 libz.so.1 -> libz.so.1.2.2     -rwxr-xr-x    1 root     root        62606 Jul 11 06:45 libz.so.1.2.2

Here, we see the shared library images for two libraries -- libncurses and libz. Note that each image has a symbolic link to it, named <library>.so.<major>, where <major> is the major version number of the library. The minor number is omitted because ld.so searches for a library only by its major version number. When ld.so sees a program that has been compiled with the stubs for Version 5.4 of libncurses, it looks for a file called libncurses.so.5 in its search path. Here, /lib/libncurses.so.5 is a symbolic link to /lib/libncurses.so.5.4, the actual version of the library we have installed.

When you upgrade a library, you must replace the .a and .so.<version> files corresponding to the library. Replacing the .a file is easy: just copy over it with the new versions. However, you must use some caution when replacing the shared library image, .so.<version>; many of the text-based programs on the system depend on shared library images, so you can't simply delete them or rename them. To put this another way, the symbolic link <library>.so.<major> must always point to a valid library image. To accomplish this, first copy the new image file to /lib, and then change the symbolic link to point to the new file in one step, using ln -sf. This is demonstrated in the following example.

Let's say you're upgrading from Version 5.4 of the libncurses library to Version 5.5. You should have the files libncurses.a and libncurses.so.5.5. First, copy the .a file to the appropriate location, overwriting the old version:

     rutabaga# cp libncurses.a /usr/lib

Now, copy the new image file to /lib (or wherever the library image should be):

     rutabaga# cp libncurses.so.5.5 /lib

Now, if you use the command ls -l /lib/libncurses, you should see something like the following:

     lrwxrwxrwx    1 root     root           17 Dec 10  1999 /lib/libncurses.so.5 ->     libncurses.so.5.4     -rwxr-xr-x    1 root     root       319472 May 11  2001 /lib/libncurses.so.5.4     -rwxr-xr-x    1 root     root       321042 May 11  2001 /lib/libncurses.so.5.5

To update the symbolic link to point to the new library, use the command:

     rutabaga# ln -sf /lib/libncurses.so.5.5 /lib/libncurses.so.5

This gives you:

     lrwxrwxrwx  1 root  root      14 Oct 23 13:25 libncurses.so.5 ->\         /lib/libncurses.so.5.4     -rwxr-xr-x  1 root  root  623620 Oct 23 13:24 libncurses.so.5.4     -rwxr-xr-x  1 root  root  720310 Nov 16 11:02 libncurses.so.5.5

Now you can safely remove the old image file, libncurses.so.5.4. You must use ln -sf to replace the symbolic link in one step, especially when updating crucial libraries, such as libc. If you were to remove the symbolic link first, and then attempt to use ln -s to add it again, more than likely ln would not be able to execute because the symbolic link is gone, and as far as ld.so is concerned, the libc library can't be found. Once the link is gone, nearly all the programs on your system will be unable to execute. Be very careful when updating shared library images. For libncurses, things are less critical because you will always have command-line programs left to clean up any mess you have made, but if you are used to using ncurses-based programs, such as Midnight Commander, this might still be an inconvenience for you.

Whenever you upgrade or add a library to the system, it's not a bad idea to run ldconfig to regenerate the library cache used by ld.so. In some cases, a new library may not be recognized by ld.so until you run ldconfig.

One question remains: where can you obtain the new versions of libraries? Several of the basic system libraries (libc, libm, and so on) can be downloaded from the directory ftp://ftp.gnu.org/pub/gnu/glibc. This contains source versions of libc and related libraries. Other libraries are maintained and archived separately. At any rate, all libraries you install should include the .so.version files and possibly the .a files, as well as a set of include files for use with the compiler.

12.4.2. Upgrading the Compiler

One other important part of the system to keep up to date is the C compiler and related utilities. These include gcc (the GNU C and C++ compiler itself), the linker, the assembler, the C preprocessor, and various include files and libraries used by the compiler itself. All are included in the Linux gcc distribution. Usually, a new version of gcc is released along with new versions of the libc library and include files, and each requires the other.

You can find the current gcc release on the various FTP archives, including ftp://ftp.gnu.org/pub/gnu/gcc. The release notes there should tell you what to do. If you don't have Internet access, you can obtain the newest compiler from CD-ROM archives of the FTP sites, as described earlier.

To find out what version of gcc you have, use the command:

     gcc -v

This should tell you something like:

     Reading specs from /usr/lib/gcc-lib/i586-suse-linux/3.3.5/specs     Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local-     prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --enable-     languages=c,c++,f77,objc,java,ada --disable-checking --libdir=/usr/lib --enable-     libgcj --with-slibdir=/lib --with-system-zlib --enable-shared --enable-_ _cxa_atexit     i586-suse-linux     Thread model: posix     gcc version 3.3.5 20050117 (prerelease) (SUSE Linux)

The last line is the interesting one, telling you the version number of gcc and when it was released. Note that gcc itself is just a frontend to the actual compiler and code-generation tools found under

     /usr/lib/gcc-lib/machine/version

gcc (usually in /usr/bin) can be used with multiple versions of the compiler proper, with the -V option. In "Programming with gcc" in Chapter 21, we describe the use of gcc in detail.

We would at this point like to warn you not to try newer compilers without knowing exactly what you are doing. Newer compilers might generate object files that are incompatible with the older ones; this can lead to all kinds of trouble. Version 3.3.x of gcc is, at the time of this writing, considered the standard compiler for Linux that everybody expects to find available, even though Versions 3.4.0 and even 4.0.0 are already available. Earlier, when one distributor (Red Hat) started to ship a newer version instead (and even that newer version was not officially released), users ran into lots of trouble. Of course, by the time you read this, another compiler version might be considered the standard. And if you feel adventurous, by all means try newer versions, just be prepared for some serious tweaking.