Section 5.3. Setting Up a Local Repository

5.3. Setting Up a Local Repository

Ideally, you won't have to go through the steps required to convert a group of RPMs into an apt repository. If you find a suitable mirror, you should be able to use the techniques described in Chapters 3 and 4 to download and synchronize your mirror to the apt repositories of your choice.

However, many repositories do not include apt databases. If you want an apt repository on your local network, you may need to download a regular mirror and then create the apt package databases from the files copied to your system. That becomes a local repository that you can use to keep your systems up to date.

Note

In this section, we assume that all you need to do is configure binary RPM packages into local repositories. If you have a need to configure source RPMs into an apt repository, the techniques are similar; generally, the Web sites mentioned in this chapter provide more information.

5.3.1. Mirroring a Remote Repository

If you're running Fedora Linux, there are several repositories that accept rsync connections. The master list of Fedora mirrors is available at fedora.redhat.com/download/mirrors.html. Read through the list, looking for the URLs that start with rsync. As an example, I can synchronize my repository with those available from mirrors.kernel.org in San Francisco. If you want to create a repository in the local /mnt/repo directory, you can use the command format described in Chapter 3.

First, make sure your chosen mirror still supports rsync connections. The following command, if successful, confirms rsync access and lists directories available for synchronization (don't forget the two colons at the end of the command):

rsync -n mirrors.kernel.org:: 

Now you can see fedora.us as part of the list. To find the appropriate subdirectory, you can do some searching. Generally, rsync directories on a mirror are structured in the same way as the HTTP or FTP server for that mirror. I've found the apt repositories for Fedora Core 3 at the following URL:

http://mirrors.kernel.org/fedora.us/fedora/fedora/3/i386/ 

Now you see the subdirectory associated with the desired apt repository:

fedora.us/fedora/fedora/3/i386/ 

Next, you can add this subdirectory to the rsync command. As you've seen in Chapter 3, the -avv switch adds appropriate levels of information. The dot (.) at the end of the following command points to all of the files in that directory. Now you can synchronize those files to a local directory of your choice; I use /mnt/repo in the following command:

rsync -avv mirrors.kernel.org::fedora.us/fedora/fedora/3/i386/. /mnt/repo 

If you don't want to use rsync, one alternative for RPM-based distributions is the ftpcopy command. Binary versions are available for Red Hat, Fedora, SUSE, and Yellowdog Linux. If you're using another RPM-based distribution, you can download and install the package from the ftpcopy home page at www.ohse.de/uwe/ftpcopy.html.

As an example, assume you want to use the ftpcopy command to synchronize a local /mnt/repo directory with the SUSE mirror at ftp://suse.osuosl.org/pub/suse/suse/i386/9.3/suse. In that case, you would run the following command to synchronize with the noted server subdirectory:

ftpcopy ftp://suse.osuosl.org/pub/suse/suse/i386/9.3/suse /mnt/repo 

As with rsync, you can use the --exclude switch to customize what you download. For example, if you don't want to download any of the SUSE 64-bit RPMs, you could modify the command accordingly:

ftpcopy --exclude '*.x86_64.rpm' ftp://suse.osuosl.org/pub/suse/suse/i386/9.3/suse /mnt/repo 

If you weren't able to find an apt repository, you might have downloaded a regular repository of files and updates. Read on to see how you can reconfigure that download into an apt repository for your network.

Note

If you want to limit what you have to keep synchronized, analyze the mirror. Look for the directories and files that are updated on a regular basis. For example, Fedora Linux limits new stable packages for a distribution to the Updates directory; in that case, all you have to synchronize is the Updates part of the Fedora repository.

5.3.2. Creating an apt Repository

Now we'll see what you can do to configure an apt repository for your network. If you're starting with a set of RPMs downloaded for your network, you'll have to configure the directories you need. While not required, it can help to use the same directory structure that you might find in an apt repository for your distribution.

When you configure a repository, you'll want to create it in a partition with sufficient space. It may be helpful to configure a separate partition for your repository, to ensure that there is sufficient space on your system, and to keep updates from crowding out the space required by other services on your server.

Organizing a Repository Like a Mirror

If you've downloaded the RPMs from a standard Fedora repository, you can organize them in that distribution's basic categories: Core, Extras, and Updates. If you're creating an apt repository for standard 32-bit PC workstations, you'll want to start with an i386/ subdirectory.

As an example, assume that you've configured a separate hard drive and mounted it on the /mnt/repo directory. To create the directory structure that you need, you'll run the following commands:

mkdir -p /mnt/repo/i386/RPMS.core mkdir /mnt/repo/i386/RPMS.updates mkdir /mnt/repo/i386/RPMS.extras 

Next, you'll copy the appropriate RPMs to the noted directories. For Fedora Linux, you don't have to download the regular Fedora installation files. If you have the DVD mounted on /media/cdrecorder, you can copy the RPMs directly with the following command:

cp /media/cdrecorder/Fedora/RPMS/* /mnt/repo/i386/RPMS.core/ 

Naturally, you'll also want to copy the RPMs that you've presumably downloaded from a Fedora mirror. But the structure of Fedora mirrors is not consistent. For example (as of this writing), if you've connected to the mirror at Portland State University at http://fedora.cat.pdx.edu/linux/core/, the Extras directory is not readily visible. You actually have to navigate up and down the server to get to the extras RPMs at http://fedora.cat.pdx.edu/linux/extras/4/i386/. On the other hand, the regular and extras RPMs are right there when you navigate to the University of Oregon mirror at ftp://limestone.uoregon.edu/fedora/.

Stable changes to Fedora are stored in the Updates directory. It's important to keep this part of any Fedora repository up to date. You can do so with an appropriate cron job that uses a command, such as rsync, to keep your local update repository synchronized with your selected public mirror.

Fedora includes other repository directories. Whether you choose to make them part of your repository depends on the needs of you and your users. However, unless your users are developers, there is no reason to download or synchronize the development/ or updates/testing/ directories. In fact, because these packages are not stable, access can be dangerous for those who link to these repositories.

Organizing a Customized Repository

It might not be easy to configure an apt repository for your distribution. It's not fun to analyze the many different directories in the SUSE apt repository.

When you create your own apt repository, you can create your own apt directories. For example, for my SUSE computers, I've created a simple two directory repository. One group consists of the operating system files, which I've copied to my /mnt/test/i386/RPMS.base directory. The other group includes the updates I've downloaded via the YaST Online Update Server described in Chapter 3. I've copied those files to my /mnt/test/i386/RPMS.updates directory.

Processing an RPM-based Repository

Now you have RPMs (and perhaps SRPMs) in appropriate directories. Unfortunately, for Red Hat, Fedora, and SUSE, there is no apt-ftparchive or dpkg-scanpackages command as described in Chapter 4. These commands allowed you to create generic compressed package databases for apt repositories on Debian-based distributions. However, if you do have the apt-ftparchive command available on your distribution, you can use this command as described in Chapter 4 to make your repository useful. But there are alternatives.

For some RPM-based distributions, you'll need the apt4rpm and apt-server packages. If you've previously configured apt on your system, and this package is available from the repositories defined in your /etc/apt/sources.list, you can download it along with associated dependencies by using the following command:

apt-get install apt4rpm apt-server 

Alternatively, you may need to download some other packages to satisfy dependencies. For guidance, see http://apt4rpm.sourceforge.net/dl-instruction.html.

When these packages are downloaded and installed, you'll have the tools you need to process the RPMs that you've copied to directories earlier in this section. The key is the genbasedir script. You can read it for yourself in the /usr/bin/genbasedir file. For the directory created earlier, you could create the apt databases with the following command:

genbasedir --progress --flat --bz2only --bloat /mnt/test/i386 

These options aren't absolutely necessary. All that is required is that you cite the directory with the RPM subdirectories. In other words, the following command is sufficient:

genbasedir /mnt/test/i386 

However, I've added the genbasedir command options shown for the following reasons:

The --progress option allows you to monitor the progress of the command.

The --flat option tells the script that the RPMs or SRPMs are in the directory defined.

The --bz2only option tells the command to generate only compressed versions of the package database. Without it, you'll find both compressed and uncompressed versions of the database in the base/ subdirectory.

The --bloat option keeps the package file list as part of the database.

Whenever you update the RPMs, you'll need to update the associated database. You don't need to update all databases in your repository. For example, if you update the contents of the /mnt/test/i386/RPMS.updates directory, you could update the associated database with the following command:

genbasedir --progress --flat --bz2only --bloat /mnt/test/i386 updates 

Naturally, if you synchronize your RPM directories with a mirror on a regular basis, you'll want to make this command a regular part of that job.