Section 27.3. Objective 3: Creating and Configuring Filesystem Options


27.3. Objective 3: Creating and Configuring Filesystem Options

In Chapter 6 we covered mounting various types of filesystems, but up to this point we have discussed creating only ext2 filesystems. In this objective, LPI expects you to be able to create other types of filesystems, such as ISO9660 filesystems for CD-ROMs. In addition, you should be able to set up automounting for network and device filesystems.

27.3.1. Automounting Filesystems

An automounter is a system that automatically mounts disks on demand. It can mount any kind of filesystem that mount can mount, as long as there is no interactive authentication. The handiest property of an automounter is that it is dynamic. If you make an update of the configuration, it's ready to mount at once.

Linux supports an automounter called autofs that is inspired by a Solaris utility of the same name. autofs is configured by two kinds of files.

The master map (usually /etc/auto.master) defines which mount points should be governed by autofs. Each mount point again has an automount map associated with it. The maps are usually stored in files called /etc/auto .mount_point. autofs can store the maps in NIS and LDAP as well as in flat files. We'll concentrate here on flat files. Consider the following /etc/auto.master file:

 # Format of this file: # mountpoint map options # For details of the format look at autofs(8). /misc   /etc/auto.misc  --timeout=15 /home   /etc/auto.home 

The contents of /etc/auto.misc are:

 usbdisk         -fstype=vfat,umask=0,dotsOK=yes :/dev/sda1 cdrom           -fstype=iso9660                 :/dev/cdrom 

This will mount a USB memory stick or disk that appears as /dev/sda1 and the CD-ROM. All the user has to do is insert the CD and issue cd /misc/cdrom; there is no need for a mount command. Due to the --timeout=15 option, the mount will disappear after 15 seconds of disuse. Quite convenient on end user machines.

How this works is that when a user attempts to access /misc/something, or anything underneath it, something is looked up in the automount map, and if it comes up with a match, the indicated device is mounted with the indicated options. The syntax of the file is fairly trivial, looking generally like this:

 key           [options]          location 

The first column contains lookup keys, the directory names specified by users. The rest of each line is the matching value. Any mount options must come first and be prefixed by a hyphen (-).

After looking up cdrom in our previous example, we use -fstype=iso9660 :/dev/cdrom. The -fstype option is passed to mount as the filesystem type (mount -tfstype), while the other options are mount options (mount -ooptions). All the same mount options that were discussed in Chapter 6 can go here.

Any string containing a : specifies the thing to mount. If the colon is a prefix, as in the previous example, it's a local device. If something precedes the colon, the device is (by default) NFS-mounted. You can mount SMB/CIFS mounts by specifying something like this:

 windows         -fstype=smbfs           ://windows/c 

In a real environment, the main use of autofs is for NFS mounts. Home directories are a favorite example. The contents of /etc/auto.home could be:

 janl        -rw,intr,hard           server:/server/bighome/& killroy     -rw,intr,hard           server:/server/bighome/& *           -rw,intr,hard           server:/server/homes/& 

Here you see that /home/janl is mounted from server:/server/bighome/janl. You may recognize the use of the ampersand (&) from sed; it means to put the string that was matched (i.e., the whole key) in the place of the ampersand. This comes in very handy with the wildcard mount. Whenever a home directory other than janl or killroy is requested, it matches the last line and therefore mounts the corresponding directory from server:/server/homes. For instance, requesting /home/silug results in an attempt to mount server:/server/homes/silug. The options shown are pretty common for NFS mounts.

autofs becomes more useful once you add a way to distribute the map files, whether by copying them from a master to all takers with FTP or by using NIS or LDAP.

27.3.2. Other Filesystems

The Objective vaguely calls for knowledge of "other" filesystems. Earlier chapters have concentrated on ext2, Linux's vintage filesystem, while mentioning ext3 in passing. ext3 is an extension to ext2 that adds metadata journaling (or logging, if you will). If a system crashes, the log enables playback of the filesystem events (file creation, deletion, etc.) that happened up to that point from the log. This enables quick recovery after crashes, rather than having to do a full multipass scan of the filesystem to discover inconsistencies caused by the crash. Thus, the log playback guarantees consistency: some recent changes may be lost, but the files and directories are not corrupted.

Among the "other" filesystems on Linux are ReiserFS, JFS, and XFS. ReiserFS originated from independent development on Linux. JFS comes from IBM and is used in both OS/2 and AIX. XFS was made by SGI for use with IRIX. All of these are journaling filesystems but otherwise support a wide array of different goals and features. IBM and SGI have been porting JFS and XFS to Linux on their own initiatives. They started before ext3 was really stable.

Of these journaling filesystems, only ext3 (and to some degree, ReiserFS) see widespread use. While both ReiserFS and JFS are included in the kernel sources, many distributions do not offer to make them during installation. The most marginal of these filesystems seems to be XFS: it is not included in kernel sources and its utilities are not included in most distributions.

For now we'll concentrate on making and tweaking ext3 filesystems. ext3 requires both a pretty recent kernel and updated ext2 utilities. Reiserfs, JFS, and XFS all need additional utility packages installed.

Please read the following warnings carefully: each one offers some sound advice!


Warning: If you use a journaling filesystem, it is quite important not to muck about with caching and flushing. A disk drive that does write caching (and most do, because their benchmarks are much nicer) also often do write reordering. But a journaling filesystem depends on the journal entries being written in the correct order. If the disk reorders them, the journal becomes corrupt. The problem becomes even worse if you use RAID, because different stripes of the same data become inconsistent. Experiments (http://www.sr5tech.com/write_back_cache_experiments.htm) have shown that there is about a 10% chance of filesystem corruption on a large filesystem after an uncontrolled shutdown.

Warning: The 2.6 kernels contain "write barriers" in appropriate places to prevent such corruption. For earlier kernels, the very best cure is having a UPS and shutting down systems in an orderly manner before the power disappears. This is smart in any case, but more important if you have a lot of storage.

Warning: On a risky system that runs an old kernel and has no UPS, consider disabling write back caches on your drives. If you have IDE drives, a simple hdparm -W0 /dev/hda (and similar commands for other disks) suffices. With SCSI and RAID, there may be BIOS settings to do the job, or you may need to locate a package called xscsi to manipulate the drive mode pages directly. The price for this is considerably higher latency for disk writes, and this may kill the performance of some system/software combinations.
27.3.2.1. Using ext3

Since ext3 is an extension of ext2, making an ext3 filesystem is done by invoking mke2fs with the -j option.

 # mke2fs -j -L test /dev/sda1 mke2fs 1.34 (25-Jul-2003) Filesystem label=test OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 51200 inodes, 204800 blocks 10240 blocks (5.00%) reserved for the super user First data block=1 25 block groups 8192 blocks per group, 8192 fragments per group 2048 inodes per group Superblock backups stored on blocks:         8193, 24577, 40961, 57345, 73729 Writing inode tables: done Creating journal (4096 blocks): done Writing superblocks and filesystem accounting information: done This filesystem will be automatically checked every 36 mounts or 180 days, whichever comes first.  Use tune2fs -c or -i to override. 

The -j option is all that is needed. You now have an ext3 filesystem. ext3 has the nice property that after it is fscked it's a valid ext2 filesystem as well. Thus, you may access ext3 disks on systems that support only ext2, providing that you make sure they are fscked. Conversely, a plain ext2 filesystem can very easily be upgraded to an ext3 filesystem; all you need to do is add the journal like this:

 # tune2fs -j /dev/sda1 tune2fs 1.34 (25-Jul-2003) Creating journal inode: done This filesystem will be automatically checked every 29 mounts or 180 days, whichever comes first.  Use tune2fs -c or -i to override. 

After you upgrade a filesystem to ext3, change its /etc/fstab entry to match.

27.3.2.2. vfat filesystems

There is one more group of filesystems that deserves mention: the Microsoft DOS/Windows FAT filesystems and their variants. Under Linux, these are accessed through the vfat filesystem type. vfat is not used much on enterprise Linux systems, but it is used a lot on desktop systems for devices such as USB memory sticks or disks, MP3 players, camera memory cards, and so on.

vfat really encompasses a class of different FAT filesystems with support for differently sized devices and short or long filenames. vfat can also be used for NTFS partitions on modern derivatives of Windows NT. Linux can create, mount, read, and write reliably to all these. The support for checking them is not complete, though it should be good enough for most jobs. Simply use mount -t vfat to mount them, mkfs.vfat to create them, and fsck.vfat to check them. The last two commands come from a package called dosfstools in both Debian and Red Hat.

Linux also offers an ntfs filesystem, but it's experimental and not offered by default in many distributions. Luckily, it's not needed either. vfat can be used with partitions considered to be NTFS partitions on Windows. Linux's ntfs is also less desirable because it doesn't reliably support writes, which vfat does.

27.3.3. ISO9660 Filesystems

ISO9660 filesystems are quite a bit different from the filesystem types mentioned before; hence we offer a whole section about them. As a sysadmin, you must be comfortable creating ISO9660 for applications that write to CDs or DVDs. When writing to CD media, you don't simply save files to them as you would a floppy drive. Even non-Linux operating systems require some type of program to simulate that behavior transparently to the user. Instead, you have to create a filesystem to be written, or burned, to CD-writable media. This filesystem is the ISO9660 filesystem, called ISO for short.

Standards with an ISO prefix, such as ISO9660, are defined by the International Organization for Standardization. While you may find references to ISO being an acronym for this organization, it is actually intended to refer to a pun on the Greek prefix "iso-" which means "same."

27.3.3.1. Creating ISO9660 images

Before you start burning CD-ROMs, you must first create your image, or ISO. This is a job done in two steps. First, you create a directory hierarchy containing all the files you want the CD to contain. Ensure that it's no larger than the maximum size supported by the CD-R (or CD-RW or even DVD) medium you're going to burn on. After that, use mkisofs to generate an ISO image (sometimes called a CD image). This image can in turn be burned onto a CD.


Syntax

 mkisofs options [ -o filename ] path [ path... ] 


Description

Generate an ISO9660 filesystem image for burning onto CD media. The original ISO9660 standard is quite rudimentary; for example, it supports only filenames in DOS 8.3 style. Therefore, it has many mutations, and mkisofs has options to support a lot of them. Most of them are obsolete or irrelevant to us.

Two mutations that are still interesting are: Rock Ridge and Joliet. The Rock Ridge extensions support filenames with up to 30 characters, UID/GID file permissions, and POSIX symbolic links and devices. Joliet is the Windows CD-ROM filesystem.mkisofs can generate both, and Linux understands both natively (but support for Joliet requires a kernel option).


Interesting options


-J

Generate Joliet extensions. These are understood by Windows starting with Windows 95 and NT. Filenames are Unicode encoded and can be up to 64 characters long. Joliet alone should be used for CDs intended for Windows only.


-r -ldots -l

These options allow Unix-style filenames. The -r option activates Rock Ridge extensions, mostly interesting because of the long filenames. It also defaults the UID/GID/permission settings in a way that is appropriate for a CD-ROM. The -r option is absolutely necessary if you want to create a CD that contains Linux executables and scripts. Omitting it removes the executable bits from all files so that they will not be executable any more. The -ldots allows filenames to start with a period (.), which is otherwise forbidden. The -l option allows 31-character names in the basic ISO9660 directories on the image. There is space for this in the directory entries, but the ordinary filename standard for ISO9660 is 8.3. Both -ldots and -l are violations of the standard, but CD-ROM images burned with seem to work on all known systems.


-T

This creates TRANS.TBL files in each directory of the image. The literature says that "some" systems use these to support long filenames with special characters. These files are very often seen on old Linux CD-ROM distributions.


-o filename

Write the image to the named file.


Example

Generate an ISO image named /var/tmp/image.iso from the files in /var/tmp/backup:

 $ mkisofs -T -r -ldots -l -J -o /var/tmp/image.iso /var/tmp/backup Warning: creating filesystem that does not conform to ISO-9660.   6.74% done, estimate finish Wed Jan 21 14:53:06 2004  13.46% done, estimate finish Wed Jan 21 14:53:06 2004  20.21% done, estimate finish Wed Jan 21 14:53:06 2004  26.93% done, estimate finish Wed Jan 21 14:53:06 2004  33.67% done, estimate finish Wed Jan 21 14:53:06 2004  40.39% done, estimate finish Wed Jan 21 14:53:06 2004  47.14% done, estimate finish Wed Jan 21 14:53:06 2004  53.86% done, estimate finish Wed Jan 21 14:53:06 2004  60.60% done, estimate finish Wed Jan 21 14:53:06 2004  67.32% done, estimate finish Wed Jan 21 14:53:06 2004  74.07% done, estimate finish Wed Jan 21 14:53:06 2004  80.79% done, estimate finish Wed Jan 21 14:53:06 2004  87.53% done, estimate finish Wed Jan 21 14:53:06 2004  94.25% done, estimate finish Wed Jan 21 14:53:06 2004 Total translation table size: 459 Total rockridge attributes bytes: 429 Total directory bytes: 0 Path table size(bytes): 10 Max brk space used 21ac4 74269 extents written (145 Mb) 

Despite the warning about nonconformance with ISO9660, the image should be compatible with all modern operating systems.

27.3.3.2. Burning CD-ROMs

Burning a CD is, in a mechanical sense, quite complicated. The CD burner must be fed with data quickly enough to keep the disk rotating at just the right speed while the laser moves in a precise spiral. With older burners, if the data does not arrive soon enough, the whole burning will fail and you will have a new coaster. New burners support different kinds of BurnProof and BurnFree technologies, meaning that the CD drive has precise enough control of speed, the angular position of the disk, and the spiral position of the laser that when a data under-run occurs, it can resume the burn when data arrives. Nonetheless, CD burning is considerably more complex than writing to a disk, and special software is needed. Linux provides cdrecord and sometimes dvdrecord. We will concentrate on the former.

27.3.3.2.1. Burning with IDE CD burners

This applies to all except a few CD burners that are SCSI units.

In the beginning, all CD burners were SCSI units and cdrecord understood only SCSI CD burners. At some point, IDE CD burners appeared. To allow use of these with cdrecord, the ide-scsi kernel module was extended so that the kernel could show IDE burners as SCSI devices to cdrecord and then translate the SCSI commands to IDE commands. Before you can burn anything, this module must be installed, which means first of all that your kernel must be configured with it. We'll assume it is compiled as a module. On Red Hat, the lines shown next should be entered directly into /etc/modules. On Debian, put them in /etc/modules.d/local (for example) and run modules-update afterwards.

 alias scd0 sr_mod pre-install sg  modprobe ide-scsi pre-install sr_mod modprobe ide-scsi pre-install ide-scsi modprobe ide-cd 

These lines set up the system so that if the SCSI CD device or the SCSI "generic" driver is loaded, the ide-scsi module will be loaded first.

In your boot loader, add the option hdc=ide-scsi on the kernel command line. For LILO, insert append="hdc=ide-scsi" to the lilo.conf, while in GRUB simply type the option into each kernel command line. The hdc assumes that your CD-burner is the master of the second IDE bus; if it's something else, change the device name appropriately. Now reboot; you should be all set to burn CDs.

27.3.3.2.2. Using cdrecord

In most cases cdrecord needs quite a long command, but fortunately it's not hard to figure out which options you need.


Syntax

 cdrecord -scanbus cdrecord [ general_options ] dev=device image_file 


Description

Burn an audio or data CD according to the relevant standards. We will restrict ourselves to data CDs. The device refers to a bus,target,lun tuple rather than a Unix-style device name. This tuple identifies the burner device. The appropriate tuple is found using the -scanbus form of the command. See the examples that come later in this section.


Interesting options


-v

Verbose output. Use one -v to track the burning process.


-dummy

Only pretend to burn. This will go through all the motions of burning, but the laser will be off. If your burner does not support BurnFree or some such feature, you can use this to verify that your system can feed data fast enough to avoid data underruns. Combine the option with -v to get information about the data feeding process.


driveropts=burnfree

Enables the burning mode that tolerates data underruns.


speed=speed

Set the burning speed. This is quite often autodetected. But if your drive does not tolerate underruns, you can reduce the speed so that data is fed to it fast enough to prevent them.


dev=device

Specify which device to burn to.


Examples

First locate your CD burner device.

 # cdrecord -scanbus Cdrecord-Clone 2.01a19 (i686-redhat-linux-gnu) Copyright (C) 1995-2003 Jorg Schilling Linux sg driver version: 3.1.25 Using libscg version 'schily-0.7' cdrecord: Warning: using inofficial libscg transport code version (schily - Red Hat-scsi-linux-sg.c-1.75-RH '@(#)scsi-linux-sg.c        1.75 02/10/21 Copyright 1997 J. Schilling'). scsibus0:         0,0,0     0) 'PLEXTOR ' 'DVDR   PX-708A  ' '1.02' Removable CD-ROM         0,1,0     1) *         0,2,0     2) *         0,3,0     3) *         0,4,0     4) *         0,5,0     5) *        0,6,0     6) *         0,7,0     7) * 

The output indicates that you want to use dev=0,0,0. To test burn a CD:

 # cdrecord -v -dummy driveropts=burnfree dev=0,0,0 /var/tmp/image.iso Cdrecord-Clone 2.01a19 (i686-redhat-linux-gnu) Copyright (C) 1995-2003 Jorg Schilling TOC Type: 1 = CD-ROM scsidev: '0,0,0' scsibus: 0 target: 0 lun: 0 Linux sg driver version: 3.1.25 Using libscg version 'schily-0.7' cdrecord: Warning: using inofficial libscg transport code version (schily - Red Hat-scsi-linux-sg.c-1.75-RH '@(#)scsi-linux-sg.c        1.75 02/10/21 Copyright 1997 J. Schilling'). Driveropts: 'burnfree' ... [ verbiage about drive and media capabilities deleted for brevity ] Driver flags   : MMC-3 SWABAUDIO BURNFREE VARIREC FORCESPEED SINGLESESSION HIDECDR ... Power-Rec write speed:     40x (recommended) Starting to write CD/DVD at speed 40 in dummy TAO mode for single session. Last chance to quit, starting dummy write    0 seconds. Operation starts. Waiting for reader process to fill input buffer ... input buffer ready. BURN-Free is OFF. Turning BURN-Free on Starting new track at sector: 0 Track 01:  145 of  145 MB written (fifo 100%) [buf  97%]  27.9x. Track 01: Total bytes read/written: 152102912/152102912 (74269 sectors). Writing  time:   52.236s Average write speed  19.0x. Min drive buffer fill was 97% Fixating... WARNING: Some drives don't like fixation in dummy mode. Fixating time:    0.000s Last selected write speed: 40x Max media write speed:     40x Last actual write speed:   26x BURN-Free was never needed. cdrecord: fifo had 2396 puts and 2396 gets. cdrecord: fifo was 0 times empty and 1388 times full, min fill was 93%. 

An altogether successful dummy burning. If your drive can't tolerate underruns, look closely at the last line detailing fifo filling: 93% is quite acceptable; 50% should be more worrying and perhaps a clue to reduce the writing speed.

27.3.3.2.3. CD miscellany

A CD, burned or obtained commercially, can be accessed as a block device with ordinary Linux commands. To verify a burned CD, simply use the cmp to check it against your original input directory, as shown below. If there is no output, you're fine.

 $ cmp /dev/cdrom /var/tmp/image.iso 

If you want to burn a copy of a data CD, you can simply copy the image from the CD to disk with dd. For example:

 $ dd bs=2k if=/dev/cdrom of=/var/tmp/cd-image.iso 56942+0 records in 56942+0 records out 

After the operation is done, you can turn around and burn new CDs with the image (if you have the necessary legal rights to copy it, of course). This is also a great way to back up CDs and to have them available for use quickly without running around finding the disk, inserting it in the right computer, and so on.

ISO filesystems on CDs are easy to mount. This is discussed in Chapter 6. The complete command to mount an ISO CD is:

 # mount -t iso9660 -o ro /dev/cdrom /mnt/cdrom 

If loop devices are supported by your kernel, you can mount the image file directly with this command:

 # mount -t iso9660 -o loop,ro /var/tmp/image.iso /mnt/cdrom 

You can also use the losetup command to easly manage loop devices.


Syntax

 losetup [ [-e|-E] encryption ] [-o offset] [-p pfd] loop_device file losetup [-d] loop_device 


Description

losetup is used to associate loop devices with regular files or block devices, to detach loop devices, and to query the status of a loop device. If only the loop_device argument is given, the status of the corresponding loop device is shown.


Interesting options


-e encryption

Enable data encryption with the specified name.


-d

Detach the file or device associated with the specified loop device.


Examples

You can use this example to encrypt (painfully) a file.

 # dd if=/dev/zero of=/file bs=1k count=100# losetup -e des /dev/loop0 /file Init (up to 16 hex digits): # mkfs -t ext2 /dev/loop0 100 # mount -t ext2 /dev/loop0 /mnt 

27.3.4. dd

On the face of it, dd may seem like an odd program, but it's widely useful because it is very good at copying data from devices to files and vice versa. In the ISO9660 section earlier in this chapter, you've already seen dd used to copy a CD image from the CD to a disk file. dd can be used with almost any Linux tool to facilitate I/O with block devices. (Moving data between files and character devices can be done with just cat.)

For example, if you want to move a filesystem from one device to another and the receiving device is at least as big as the originating one, just enter something such as:

 # dd bs=1k if=/dev/from of=/dev/too 

If you combine dd with rsh or ssh, a filesystem can be transferred from system to system over the networkjust not very efficiently, because the free blocks will also be transferred. Add gzip -1 -c to the pipeline before the network copy command, and all the blocks containing zeros will be transfered much faster.

When operating on block devices, the block size can be very important. One point to remember is that larger blocks are faster. When writing to and from disks, bs=8k can help immensely. When operating on tar files, you should always use bs=10k, because tar uses a block size of 10 KB by default.

The dd command can copy raw data and disk images to and from such devices as floppies and tape drives. A common use is to create boot floppies:

 dd if=kernel_image.iso of=/dev/fd0 

Similarly, dd can copy the entire contents of a floppy, even one formatted with a "foreign" OS, to the hard drive as an image file.



LPI Linux Certification in a Nutshell
LPI Linux Certification in a Nutshell (In a Nutshell (OReilly))
ISBN: 0596005288
EAN: 2147483647
Year: 2004
Pages: 257

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net