Files and More

If you use Linux, you will have no choice in working with text files. Text files make up 99.9% of the configuration files. If you have an Apache Web server running, you'll need to manipulate the text configuration file. Adding a hard disk to your system will require you to edit the fstab configuration file if you want the new disk to be checked and mounted at boot time. Text files make up most of what Linux is, and so it stands to reason that there is a vast number of tools that assist in the manipulation of text files.

Compiled here are the utilities we feel will aid you in your daily administrative tasks , programming and debugging, configuration management, file manipulation, and much more.

Vi

"Real" Unix hackers use vi exclusively. With so many Windows programmers around using GUI-based programmer editors and Integrated Development Environments (IDE), you'd swear vi was a relic. What keeps vi around? Simple answer, really: It works very well and can be found on almost every Unix and Unix-variant system. For programmers, vi offers amazing functionality. Admittedly, it is not the sexiest of editors, but that too is a noteworthy feature. vi is small and fast and works nicely over a 1200-baud modem link with a VT-100 terminal. Unlike Emacs, it won't suck up 20 “200MB of disk space. Finally, clones of vi are also available on every OS in existence, thus making vi the most widely available program on Earth and nearby planets.

Sed

The stream editor program allows you to transform the data flowing through a pipeline.

Dd

This program is more than just a copier program, it can also perform conversions and other important functions. Input can be blocked, chopped, skipped , or converted. The data can be converted from ASCII to EBCDIC and back if you need that kind of thing. Trust me, if you're in an information system (IS) shop that includes mainframes, you'll need this ability.

Diff

This tool allows you to compare two files and finds the differences between them.

Od

Octal dump. If you don't want octal, then try one of the other formats it supports, like hex. This tool is great for finding unwanted data in a text file. For instance, when your EDI trading partners insist that they sent you clean files, you can find their junk data by using od to print out a dump of the files. Then just sit back and wait for the apologies.

Ispell

Didn't think there was a spellchecker for Linux? Guess again.

Tar

tar stands for "tape archive." Don't let the tape part fool you, though. tar is a handy tool for creating a collection or archive of files. It's like the zip utility found on the old MS-DOS OS but without the compression. Of course, compression utilities, like gzip, can be used in combination with tar. Look for filenames with .tar.gz or .tgz.

G[un]zip

gzip uses Lempel-Ziv coding (LZ77) to compress files. Look for the extension .gz (in most cases) to indicate a gzipped file. Use gunzip to uncompress the file.

B[un]zip2

Yet another file compression tool. Bzip generally makes files smaller than gzip.

CVS

Concurrent Versioning System is used to keep track of changes between files. Typically, this tool is used for source code, but it can be used for other files as well. For instance, the authors of this book used cvs to manage a central repository for the chapters in this book, which were written in HTML. Then a cron-executed script was used to extract the latest version of each chapter from the archive and place them into a Web server document root. The authors then had a central place to view each other's chapters as they were being written. It also allowed our publisher and reviewers to view the progress of the book.

Cut

cut is an invaluable tool for pulling out a single field or character of data from an entire line. Many Unix utilities, like ps, like to show you only a whole line of data. Often, you only want one piece ”like a PID (process ID). Using cut, you can simply grab out the PID and discard the rest of the line. This means that tasks such as figuring out the PID of a daemon process and issuing a SIGHUP (restart) signal can be done in one mongo line, such as:

  kill 'ps waux  grep inetd  cut -c10-14'

more is a great way either for viewing the output of another command in a screen-by-screen mode or for viewing text files. Unlike less, it automatically exits when you get to the end of the file and hit the space bar. Unlike more on almost all other forms of Unix (except for AIX, which has a proper version), it allows you to scroll backwards through the input.

Perl

The pathologically eclectic rubbish lister is the single most useful scripting language you can ever learn. Perl runs on many operating systems (including VAX/VMS, MVS/ESA, and MacOS), and it is the undisputed king of text-processing tools. Using Perl, you can simulate cut or sed or awk or diff or build anything else you might need ”such as a little program that looks at three or four text files and tells you which entries do not appear in all of them. You can write CGI scripts for your Web server; you can use it to automatically parse error logs and issue pages. Heck, you can even write a Telnet daemon (or any other daemon for that matter) in Perl.

File

file will tell you what kind of data is in a file. Using the data from /etc/magic, it will let you know if that file named real_slim_shady.mp3 is actually an mp3 file or not. This is very useful when downloading files from those pesky Mac users ”who seem to think that file extensions aren't really necessary.

Strings

Use the strings command to get a list of the printable strings in a file. Try this command against a binary file such as /usr/bin/perl. Interesting things can be found using this command.