27.3. Objective 3: Creating and Configuring Filesystem OptionsIn Chapter 6 we covered mounting various types of filesystems, but up to this point we have discussed creating only ext2 filesystems. In this objective, LPI expects you to be able to create other types of filesystems, such as ISO9660 filesystems for CD-ROMs. In addition, you should be able to set up automounting for network and device filesystems. 27.3.1. Automounting FilesystemsAn automounter is a system that automatically mounts disks on demand. It can mount any kind of filesystem that mount can mount, as long as there is no interactive authentication. The handiest property of an automounter is that it is dynamic. If you make an update of the configuration, it's ready to mount at once. Linux supports an automounter called autofs that is inspired by a Solaris utility of the same name. autofs is configured by two kinds of files. The master map (usually /etc/auto.master) defines which mount points should be governed by autofs. Each mount point again has an automount map associated with it. The maps are usually stored in files called /etc/auto .mount_point. autofs can store the maps in NIS and LDAP as well as in flat files. We'll concentrate here on flat files. Consider the following /etc/auto.master file: # Format of this file: # mountpoint map options # For details of the format look at autofs(8). /misc /etc/auto.misc --timeout=15 /home /etc/auto.home The contents of /etc/auto.misc are: usbdisk -fstype=vfat,umask=0,dotsOK=yes :/dev/sda1 cdrom -fstype=iso9660 :/dev/cdrom This will mount a USB memory stick or disk that appears as /dev/sda1 and the CD-ROM. All the user has to do is insert the CD and issue cd /misc/cdrom; there is no need for a mount command. Due to the --timeout=15 option, the mount will disappear after 15 seconds of disuse. Quite convenient on end user machines. How this works is that when a user attempts to access /misc/something, or anything underneath it, something is looked up in the automount map, and if it comes up with a match, the indicated device is mounted with the indicated options. The syntax of the file is fairly trivial, looking generally like this: key [options] location The first column contains lookup keys, the directory names specified by users. The rest of each line is the matching value. Any mount options must come first and be prefixed by a hyphen (-). After looking up cdrom in our previous example, we use -fstype=iso9660 :/dev/cdrom. The -fstype option is passed to mount as the filesystem type (mount -tfstype), while the other options are mount options (mount -ooptions). All the same mount options that were discussed in Chapter 6 can go here. Any string containing a : specifies the thing to mount. If the colon is a prefix, as in the previous example, it's a local device. If something precedes the colon, the device is (by default) NFS-mounted. You can mount SMB/CIFS mounts by specifying something like this: windows -fstype=smbfs ://windows/c In a real environment, the main use of autofs is for NFS mounts. Home directories are a favorite example. The contents of /etc/auto.home could be: janl -rw,intr,hard server:/server/bighome/& killroy -rw,intr,hard server:/server/bighome/& * -rw,intr,hard server:/server/homes/& Here you see that /home/janl is mounted from server:/server/bighome/janl. You may recognize the use of the ampersand (&) from sed; it means to put the string that was matched (i.e., the whole key) in the place of the ampersand. This comes in very handy with the wildcard mount. Whenever a home directory other than janl or killroy is requested, it matches the last line and therefore mounts the corresponding directory from server:/server/homes. For instance, requesting /home/silug results in an attempt to mount server:/server/homes/silug. The options shown are pretty common for NFS mounts. autofs becomes more useful once you add a way to distribute the map files, whether by copying them from a master to all takers with FTP or by using NIS or LDAP. 27.3.2. Other FilesystemsThe Objective vaguely calls for knowledge of "other" filesystems. Earlier chapters have concentrated on ext2, Linux's vintage filesystem, while mentioning ext3 in passing. ext3 is an extension to ext2 that adds metadata journaling (or logging, if you will). If a system crashes, the log enables playback of the filesystem events (file creation, deletion, etc.) that happened up to that point from the log. This enables quick recovery after crashes, rather than having to do a full multipass scan of the filesystem to discover inconsistencies caused by the crash. Thus, the log playback guarantees consistency: some recent changes may be lost, but the files and directories are not corrupted. Among the "other" filesystems on Linux are ReiserFS, JFS, and XFS. ReiserFS originated from independent development on Linux. JFS comes from IBM and is used in both OS/2 and AIX. XFS was made by SGI for use with IRIX. All of these are journaling filesystems but otherwise support a wide array of different goals and features. IBM and SGI have been porting JFS and XFS to Linux on their own initiatives. They started before ext3 was really stable. Of these journaling filesystems, only ext3 (and to some degree, ReiserFS) see widespread use. While both ReiserFS and JFS are included in the kernel sources, many distributions do not offer to make them during installation. The most marginal of these filesystems seems to be XFS: it is not included in kernel sources and its utilities are not included in most distributions. For now we'll concentrate on making and tweaking ext3 filesystems. ext3 requires both a pretty recent kernel and updated ext2 utilities. Reiserfs, JFS, and XFS all need additional utility packages installed. Please read the following warnings carefully: each one offers some sound advice! Warning: If you use a journaling filesystem, it is quite important not to muck about with caching and flushing. A disk drive that does write caching (and most do, because their benchmarks are much nicer) also often do write reordering. But a journaling filesystem depends on the journal entries being written in the correct order. If the disk reorders them, the journal becomes corrupt. The problem becomes even worse if you use RAID, because different stripes of the same data become inconsistent. Experiments (http://www.sr5tech.com/write_back_cache_experiments.htm) have shown that there is about a 10% chance of filesystem corruption on a large filesystem after an uncontrolled shutdown. Warning: The 2.6 kernels contain "write barriers" in appropriate places to prevent such corruption. For earlier kernels, the very best cure is having a UPS and shutting down systems in an orderly manner before the power disappears. This is smart in any case, but more important if you have a lot of storage. Warning: On a risky system that runs an old kernel and has no UPS, consider disabling write back caches on your drives. If you have IDE drives, a simple hdparm -W0 /dev/hda (and similar commands for other disks) suffices. With SCSI and RAID, there may be BIOS settings to do the job, or you may need to locate a package called xscsi to manipulate the drive mode pages directly. The price for this is considerably higher latency for disk writes, and this may kill the performance of some system/software combinations. 27.3.2.1. Using ext3Since ext3 is an extension of ext2, making an ext3 filesystem is done by invoking mke2fs with the -j option. # mke2fs -j -L test /dev/sda1 mke2fs 1.34 (25-Jul-2003) Filesystem label=test OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 51200 inodes, 204800 blocks 10240 blocks (5.00%) reserved for the super user First data block=1 25 block groups 8192 blocks per group, 8192 fragments per group 2048 inodes per group Superblock backups stored on blocks: 8193, 24577, 40961, 57345, 73729 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 36 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. The -j option is all that is needed. You now have an ext3 filesystem. ext3 has the nice property that after it is fscked it's a valid ext2 filesystem as well. Thus, you may access ext3 disks on systems that support only ext2, providing that you make sure they are fscked. Conversely, a plain ext2 filesystem can very easily be upgraded to an ext3 filesystem; all you need to do is add the journal like this: # tune2fs -j /dev/sda1 tune2fs 1.34 (25-Jul-2003) Creating journal inode: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. After you upgrade a filesystem to ext3, change its /etc/fstab entry to match. 27.3.2.2. vfat filesystemsThere is one more group of filesystems that deserves mention: the Microsoft DOS/Windows FAT filesystems and their variants. Under Linux, these are accessed through the vfat filesystem type. vfat is not used much on enterprise Linux systems, but it is used a lot on desktop systems for devices such as USB memory sticks or disks, MP3 players, camera memory cards, and so on. vfat really encompasses a class of different FAT filesystems with support for differently sized devices and short or long filenames. vfat can also be used for NTFS partitions on modern derivatives of Windows NT. Linux can create, mount, read, and write reliably to all these. The support for checking them is not complete, though it should be good enough for most jobs. Simply use mount -t vfat to mount them, mkfs.vfat to create them, and fsck.vfat to check them. The last two commands come from a package called dosfstools in both Debian and Red Hat. Linux also offers an ntfs filesystem, but it's experimental and not offered by default in many distributions. Luckily, it's not needed either. vfat can be used with partitions considered to be NTFS partitions on Windows. Linux's ntfs is also less desirable because it doesn't reliably support writes, which vfat does. 27.3.3. ISO9660 FilesystemsISO9660 filesystems are quite a bit different from the filesystem types mentioned before; hence we offer a whole section about them. As a sysadmin, you must be comfortable creating ISO9660 for applications that write to CDs or DVDs. When writing to CD media, you don't simply save files to them as you would a floppy drive. Even non-Linux operating systems require some type of program to simulate that behavior transparently to the user. Instead, you have to create a filesystem to be written, or burned, to CD-writable media. This filesystem is the ISO9660 filesystem, called ISO for short. Standards with an ISO prefix, such as ISO9660, are defined by the International Organization for Standardization. While you may find references to ISO being an acronym for this organization, it is actually intended to refer to a pun on the Greek prefix "iso-" which means "same." 27.3.3.1. Creating ISO9660 imagesBefore you start burning CD-ROMs, you must first create your image, or ISO. This is a job done in two steps. First, you create a directory hierarchy containing all the files you want the CD to contain. Ensure that it's no larger than the maximum size supported by the CD-R (or CD-RW or even DVD) medium you're going to burn on. After that, use mkisofs to generate an ISO image (sometimes called a CD image). This image can in turn be burned onto a CD.
27.3.3.2. Burning CD-ROMsBurning a CD is, in a mechanical sense, quite complicated. The CD burner must be fed with data quickly enough to keep the disk rotating at just the right speed while the laser moves in a precise spiral. With older burners, if the data does not arrive soon enough, the whole burning will fail and you will have a new coaster. New burners support different kinds of BurnProof and BurnFree technologies, meaning that the CD drive has precise enough control of speed, the angular position of the disk, and the spiral position of the laser that when a data under-run occurs, it can resume the burn when data arrives. Nonetheless, CD burning is considerably more complex than writing to a disk, and special software is needed. Linux provides cdrecord and sometimes dvdrecord. We will concentrate on the former. 27.3.3.2.1. Burning with IDE CD burnersThis applies to all except a few CD burners that are SCSI units. In the beginning, all CD burners were SCSI units and cdrecord understood only SCSI CD burners. At some point, IDE CD burners appeared. To allow use of these with cdrecord, the ide-scsi kernel module was extended so that the kernel could show IDE burners as SCSI devices to cdrecord and then translate the SCSI commands to IDE commands. Before you can burn anything, this module must be installed, which means first of all that your kernel must be configured with it. We'll assume it is compiled as a module. On Red Hat, the lines shown next should be entered directly into /etc/modules. On Debian, put them in /etc/modules.d/local (for example) and run modules-update afterwards. alias scd0 sr_mod pre-install sg modprobe ide-scsi pre-install sr_mod modprobe ide-scsi pre-install ide-scsi modprobe ide-cd These lines set up the system so that if the SCSI CD device or the SCSI "generic" driver is loaded, the ide-scsi module will be loaded first. In your boot loader, add the option hdc=ide-scsi on the kernel command line. For LILO, insert append="hdc=ide-scsi" to the lilo.conf, while in GRUB simply type the option into each kernel command line. The hdc assumes that your CD-burner is the master of the second IDE bus; if it's something else, change the device name appropriately. Now reboot; you should be all set to burn CDs. 27.3.3.2.2. Using cdrecordIn most cases cdrecord needs quite a long command, but fortunately it's not hard to figure out which options you need.
27.3.3.2.3. CD miscellanyA CD, burned or obtained commercially, can be accessed as a block device with ordinary Linux commands. To verify a burned CD, simply use the cmp to check it against your original input directory, as shown below. If there is no output, you're fine. $ cmp /dev/cdrom /var/tmp/image.iso If you want to burn a copy of a data CD, you can simply copy the image from the CD to disk with dd. For example: $ dd bs=2k if=/dev/cdrom of=/var/tmp/cd-image.iso 56942+0 records in 56942+0 records out After the operation is done, you can turn around and burn new CDs with the image (if you have the necessary legal rights to copy it, of course). This is also a great way to back up CDs and to have them available for use quickly without running around finding the disk, inserting it in the right computer, and so on. ISO filesystems on CDs are easy to mount. This is discussed in Chapter 6. The complete command to mount an ISO CD is: # mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom If loop devices are supported by your kernel, you can mount the image file directly with this command: # mount -t iso9660 -o loop,ro /var/tmp/image.iso /mnt/cdrom You can also use the losetup command to easly manage loop devices.
27.3.4. ddOn the face of it, dd may seem like an odd program, but it's widely useful because it is very good at copying data from devices to files and vice versa. In the ISO9660 section earlier in this chapter, you've already seen dd used to copy a CD image from the CD to a disk file. dd can be used with almost any Linux tool to facilitate I/O with block devices. (Moving data between files and character devices can be done with just cat.) For example, if you want to move a filesystem from one device to another and the receiving device is at least as big as the originating one, just enter something such as: # dd bs=1k if=/dev/from of=/dev/too If you combine dd with rsh or ssh, a filesystem can be transferred from system to system over the networkjust not very efficiently, because the free blocks will also be transferred. Add gzip -1 -c to the pipeline before the network copy command, and all the blocks containing zeros will be transfered much faster. When operating on block devices, the block size can be very important. One point to remember is that larger blocks are faster. When writing to and from disks, bs=8k can help immensely. When operating on tar files, you should always use bs=10k, because tar uses a block size of 10 KB by default. The dd command can copy raw data and disk images to and from such devices as floppies and tape drives. A common use is to create boot floppies: dd if=kernel_image.iso of=/dev/fd0 Similarly, dd can copy the entire contents of a floppy, even one formatted with a "foreign" OS, to the hard drive as an image file. |