Qshell for iSeries - page 123

Summary

In this chapter, you've learned about some advanced Qshell programming features:

Use Qshell option settings with the set utility to trace and debug your shell scripts.
Use traps and signals to handle complex and unexpected error conditions. Detecting simple errors early can prevent more serious errors later.
Use the kill utility to manage running scripts or provide simple event-notification between scripts.

Chapter 16: Archives and Compression

Overview

Working with a large number of related files can sometimes be cumbersome and time-consuming . Archive files help reduce the complexity of file manipulation. By grouping related files together into a single archive file, moving and transferring that one file has the effect of moving or transferring all of the individual files. The individual files are extracted when desired.

Users familiar with iSeries save files will find similarities in the way other archive files are used to back up, move, or transfer groups of files. The iSeries save file is an archive file.

The most common types of archive files are zip files, jar files, and tar files. The archive file type refers to the utility used to create the archive file. For example, a zip utility creates zip archive files.

A compressed archive file uses less space than the sum of all of the individual files that it contains. Some archive utilities, such as jar , use data compression when creating the archive file, while others, such as tar , do not. You can, however, use compression utilities to compress data in archive files that do not use data compression themselves . For example, archive files created by the tar utility are commonly compressed using the compress utility.

Archive files are binary files that have defined data formats. When transferring archive files (perhaps using FTP), always transfer in binary mode. The Qshell archive utility converts the data as required by the utility during insertion or extraction of files from the archive files.

Example Data

The examples in this chapter use an assortment of files, links, and directories as the source of the data. For demonstration purposes, the examples all use the v option to verbosely display the tar processing that occurs. The example data is listed in Figure 16.1


ls -l


total: 36 kilobytes


-rwxr-xr-x  1 JSMITH   0      107 Jul 13 15:45 HelloC.c


-rwxr-xr-x  1 JSMITH   0      118 Jul 13 15:45 HelloCpp.C


drwxrwsrwx  2 JSMITH   0     8192 Jul 13 15:46 data


lrwxrwxrwx  1 JSMITH   0       37 Jul 13 15:44 goodoleboys.sql ->


/home/jsmith/src/data/goodoleboys.sql


lrwxrwxrwx  1 JSMITH   0       37 Jul 13 15:44 goodoleboys.txt ->


/home/jsmith/src/data/goodoleboys.txt


lrwxrwxrwx  1 JSMITH   0       31 Jul 13 15:43 qcsrc ->


/qsys.lib/jsmith.lib/qcsrc.file


ls -lL data qcsrc


data:


total: 40 kilobytes


-rw-rw----  1 JSMITH   0     3341 Jul 13 15:46 customers.txt


-rw-rw----  1 JSMITH   0    13400 Jul 13 15:46 notes.txt


qcsrc:


total: 228 kilobytes


-rwx---rwx  1 JSMITH   0    73431 Jul 13 15:42 GOODBYE.MBR


-rwx---rwx  1 JSMITH   0    19434 Jul 13 15:42 HELLO.MBR


-rwx---rwx  1 JSMITH   0    73431 Jul 13 15:43 TEST.MBR

Figure 16.1: These files and directories are used by the archive examples in this chapter.

Tar

Tar, the tape archive utility, is named for its ability to read, write, and list files on a tape drive. A tar file is a binary file; the tar file format specifies an ASCII file format. The tar utility converts files inserted into the archive to ASCII. When extracting files, the tar utility converts files extracted from the archive to EBCDIC (the default CCSID of the job). Setting the QIBM_CCSID environment variable to a value other than zero causes tar to convert the ASCII data in the archived file to that CCSID.

Like tape media, a tar file contains a sequential list of file or directory entries. The same source file may be present in the tar file more than once. If you add or update file entries in a tar file, the entries are added to the end.

When the tar utility encounters a directory, it processes the directories recursively unless the directory is a symbolic link. Symbolic-link processing is controlled by the H , L , and P options.

The syntax of tar is shown here:


tar


operation option

[




][

tar-file

][

block-size

][

files



]

Tar Options

The first option character passed to tar is the operation that tar should perform. Exactly one option for the operation is required from the list shown in Table 16.1. Specify the tar-file parameter if the f option is used; otherwise , the tar-file parameter defaults to the file archive.tar . Tar assumes that the first argument consists of one or more options. For this reason, the options do not have to be preceded by a hyphen.

Table 16.1: Operations for Tar
Operation	Description
c	Create a new archive file.
x	Extract files from an archive file.
t	List files in an archive file.
r	Append files to an existing archive file.
u	Update files in the archive file. The u option is rather misleading, because updated files are actually added to the end of the archive, similar to what you might expect the r option to do.

If no files are specified for an extract or list operation, all files in the archive are targeted . If no files are specified for a create, update, or append operation, the filenames are read from standard input.

All other options for tar are optional. They are shown in Table 16.2, and affect the way that the tar operation is carried out.

Table 16.2: Options for Tar
Option	Description
b	This option indicates that the next argument is the blocksize argument. The blocksize argument is used to define the size of the block when creating an archive file.
e	Exit immediately if an error is encountered .
f	This option indicates that the next argument is the archive file argument. Use an archive file name of “ (dash) to read or write the archive to standard input or standard output. If unspecified, tar uses a default archive filename of archive.tar.
m	Modification times of extracted files are not restored when extracting files from the archive file.
o	Extracted files are assigned the owner and group of the current user . Saved owner and group settings are not restored from the archive file.
p	Preserve the file mode, access, and modification times, as well as owner and group settings, of files when extracting files from the archive file.
v	Use verbose mode. Write additional information about files processed during any operation.
w	Wait for a confirmation from the user before taking any action.
H	Remove references to symbolic links for files specified on the command line. Archive the files referred to by the links instead of the symbolic links themselves .
L	Remove references to all symbolic links encountered. Archive the files referred to by the links instead of the symbolic links themselves.
P	Do not remove references to symbolic links. Archive the symbolic links themselves instead of the files referred to by the symbolic links. This is the default symbolic-link behavior.
X	While processing directories recursively, do not process directories that have a different device ID (for example, symbolic links to directories in a different file system).

Tar Examples

This section contains several examples that demonstrate some basic tar operations. Figure 16.2 demonstrates how to create an archive file. The default symbolic-link behavior is as if the P option were used. Therefore, the qcsrc and goodoleboys symbolic links are copied directly to the archives. The l (list) operation shows that the symbolic links are present in the archive file, not the files to which the links point.


tar vc *


HelloC.c


HelloCpp.C


archive.tar


data


data/customers.txt


data/notes.txt


goodoleboys.sql


goodoleboys.txt


qcsrc


tar: 001-2298 For archive file tar and volume 1, 9 files were


processed with 0 bytes read and 0 bytes written.


ls -l archive.tar


-rw-rw-rw-  1 JSMITH   0        30720 Jul 13 16:02 archive.tar


tar tv

-rwxr-xr-x  1 JSMITH   0        0 Jul 13 15:45 HelloC.c

-rwxr-xr-x  1 JSMITH   0        0 Jul 13 15:45 HelloCpp.C


-rw-rw-rw-  1 JSMITH   0        0 Jul 13 16:02 archive.tar


drwxrwsrwx  2 JSMITH   0        0 Jul 13 15:46 data


-rw-rw----  1 JSMITH   0        0 Jul 13 15:46 data/customers.txt


-rw-rw----  1 JSMITH   0        0 Jul 13 15:46 data/notes.txt


lrwxrwxrwx  1 JSMITH   0        0 Jul 13 15:44 goodoleboys.sql =>


/home/jsmith/src/data/goodoleboys.sql


lrwxrwxrwx  1 JSMITH   0        0 Jul 13 15:44 goodoleboys.txt =>


/home/jsmith/src/data/goodoleboys.txt


lrwxrwxrwx  1 JSMITH   0        0 Jul 13 15:43 qcsrc =>


/qsys.lib/jsmith.lib/qcsrc.file

Figure 16.2: The default archive file contains all files and directories in the current directory.

Figure 16.3 demonstrates the w (Wait for Confirmation) option that prompts the user for individual file actions. Only the HelloC.c file is added to the archive file. A subsequent tar update operation (the u option) adds HelloC.c to the archive file again, but this time the user renames the file "HelloC-version2.c." Used this way, the tar utility provides a rudimentary historical view of a changing data file.


tar cw H*


tar: Starting interactive file rename operation.


tar: The current file is HelloC.c with mode -rwxr-xr-x and a


modification time of Jul 13 17:01.


tar: Enter a new name, or a period (".") to quit, or press


Enter to skip this file:

.

tar: The file name is not changed.


tar: Starting interactive file rename operation.


tar: The current file is HelloCpp.C with mode -rwxr-xr-x and


a modification time of Jul 13 17:01.


tar: Enter a new name, or a period (".") to quit, or press


Enter to skip this file:


tar: The file is skipped.


tar tvf archive.tar


-rwxr-xr-x  1 JSMITH   0        0 Jul 13 17:01 HelloC.c


tar: 001-2298 For archive file tar and volume 1, 1 files were


processed with 0 bytes read and 102400 bytes written.


tar wuf archive.tar HelloC.c


tar: Starting interactive file rename operation.


tar: The current file is HelloC.c with mode -rwxr-xr-x and a


modification time of Jul 13 17:01.


tar: Enter a new name, or a period (".") to quit, or press


Enter to skip this file:


HelloC-version2.c


tar: The file name is changed to HelloC-version2.c.


tar tvf archive.tar


-rwxr-xr-x  1 JSMITH   0        0 Jul 13 17:01 HelloC.c


-rwxr-xr-x  1 JSMITH   0        0 Jul 13 17:01 HelloC-version2.c


tar: 001-2298 For archive file tar and volume 1, 2 files were


processed with 0 bytes read and 199168 bytes written.

Figure 16.3: Options can be used to confirm file operations.

Use the L option to create an archive file containing the current directory (the dot). With this option, the files or directories referred to by symbolic links (in Figure 16.4, qcsrc, goodoleboys.sql, and goodoleboys.txt) are copied directly into the archive instead of the links. When extracting those objects, only the files remain .


tar cLf archive-file.tar .


tar: 001-2298 For archive file tar and volume 1, 14 files were


processed with 0 bytes read and 0 bytes written.


tar tvf archive-file.tar


drwxrwsrwx  2 JSMITH   0     0 Jul 13 16:26 .


-rw-rw-rw-  1 JSMITH   0     0 Jul 12 16:39 ./goodoleboys.sql


-rw-rw-rw-  1 JSMITH   0     0 Jun 29 13:03 ./goodoleboys.txt


drwx---rwx  2 JSMITH   0     0 Jul 13 15:43 ./qcsrc


-rwx---rwx  1 JSMITH   0     0 Jul 13 15:42 ./qcsrc/GOODBYE.MBR


-rwx---rwx  1 JSMITH   0     0 Jul 13 15:42 ./qcsrc/HELLO.MBR


-rwx---rwx  1 JSMITH   0     0 Jul 13 15:43 ./qcsrc/TEST.MBR


-rwxr-xr-x  1 JSMITH   0     0 Jul 13 15:45 ./HelloCpp.C


-rwxr-xr-x  1 JSMITH   0     0 Jul 13 15:45 ./HelloC.c


drwxrwsrwx  2 JSMITH   0     0 Jul 13 15:46 ./data


-rw-rw----  1 JSMITH   0     0 Jul 13 15:46 ./data/customers.txt


-rw-rw----  1 JSMITH   0     0 Jul 13 15:46 ./data/notes.txt


-rw-rw-rw-  1 JSMITH   0     0 Jul 13 16:28 ./archive.tar


-rw-rw-rw-  1 JSMITH   0     0 Jul 13 16:26 ./test.tar


tar: 001-2298 For archive file tar and volume 1, 14 files were


processed with 0 bytes read and 266240 bytes written.

Figure 16.4: The L option follows symbolic links.

Figure 16.5 demonstrates a combination of the X and L options. While the L option causes tar to follow all symbolic links, the X option prevents tar from writing the contents of directory qcsrc, because qcsrc refers to a different file system.


tar cvXL .

.

./goodoleboys.sql


./goodoleboys.txt


./qcsrc


./HelloCpp.C


./HelloC.c


./data


./data/customers.txt


./data/notes.txt


./archive.tar


./test.tar


tar: 001-2298 For archive file tar and volume 1, 11 files were


processed with 0 bytes read and 0 bytes written.


tar tvf archive.tar


drwxrwsrwx  2 JSMITH   0     0 Jul 13 16:26 .


-rw-rw-rw-  1 JSMITH   0     0 Jul 12 16:39 ./goodoleboys.sql


-rw-rw-rw-  1 JSMITH   0     0 Jun 29 13:03 ./goodoleboys.txt


drwx---rwx  2 JSMITH   0     0 Jul 13 15:43 ./qcsrc


-rwxr-xr-x  1 JSMITH   0     0 Jul 13 15:45 ./HelloCpp.C


-rwxr-xr-x  1 JSMITH   0     0 Jul 13 15:45 ./HelloC.c


drwxrwsrwx  2 JSMITH   0     0 Jul 13 15:46 ./data


-rw-rw---- 1 JSMITH   0     0 Jul 13 15:46 ./data/customers.txt


-rw-rw----  1 JSMITH   0     0 Jul 13 15:46 ./data/notes.txt


-rw-rw-rw-  1 JSMITH   0     0 Jul 13 16:32 ./archive.tar


-rw-rw-rw-  1 JSMITH   0     0 Jul 13 16:26 ./test.tar


tar: 001-2298 For archive file tar and volume 1, 11 files were


processed with 0 bytes read and 92160 bytes written.

Figure 16.5: Use the X option to avoid files in other file systems.

If no file names are passed to tar on the command line, tar reads the file names from standard input. In Figure 16.6, the ls utility is used to enter all files in the current directory. As each filename is read by tar , that file is added to the archive.

{% if main.adsdop %}{% include 'adsenceinline.tpl' %}{% endif %}


ls -1


tar cvf test.tar


HelloC.c


HelloCpp.C


archive.tar


data


data/customers.txt


data/notes.txt


goodoleboys.sql


goodoleboys.txt


qcsrc


tar: 001-2298 For archive file tar and volume 1, 9 files were


processed with 0 bytes read and 0 bytes written.



tar tvf test.tar



-rwxr-xr-x 1 JSMITH    0        0 Jul 13 15:45 HelloC.c


-rwxr-xr-x 1 JSMITH    0        0 Jul 13 15:45 HelloCpp.C


-rw-rw-rw- 1 JSMITH    0        0 Jul 13 16:24 archive.tar


drwxrwsrwx 2 JSMITH    0        0 Jul 13 15:46 data


-rw-rw---- 1 JSMITH    0        0 Jul 13 15:46 data/customers.txt


-rw-rw---- 1 JSMITH    0        0 Jul 13 15:46 data/notes.txt


lrwxrwxrwx 1 JSMITH    0        0 Jul 13 15:44 goodoleboys.sql =>


/home/jsmith/src/data/goodoleboys.sql


lrwxrwxrwx 1 JSMITH    0        0 Jul 13 15:44 goodoleboys.txt =>


/home/jsmith/src/data/goodoleboys.txt


lrwxrwxrwx 1 JSMITH    0        0 Jul 13 15:43 qcsrc =>


/qsys.lib/jsmith.lib/qcsrc.file


tar: 001-2298 For archive file tar and volume 1, 9 files were


processed with 0 bytes read and 61440 bytes written.

Figure 16.6: By default, tar reads file names from standard input.

Figure 16.7 demonstrates restoring individual files from an archive. The archive contents are first shown using the t (List) option. The x (Extract) option extracts files that match the patterns passed on the command line. In this example, a wildcard is used. The pattern with the wildcard must be quoted on the Qshell command line to prevent Qshell from expanding it and allow the tar utility to expand the wildcard. Restore all files by specifying no filename parameters on the command line.



tar tvf archive.tar



drwxrwsrwx 2 JSMITH  0    0 Jul 13 16:45 .


lrwxrwxrwx 1 JSMITH  0    0 Jul 13 15:44 ./goodoleboys.sql =>


/home/jsmith/src/data/goodoleboys.sql


lrwxrwxrwx 1 JSMITH  0    0 Jul 13 15:44 ./goodoleboys.txt =>


/home/jsmith/src/data/goodoleboys.txt


lrwxrwxrwx 1 JSMITH  0    0 Jul 13 15:43 ./qcsrc =>


/qsys.lib/jsmith.lib/qcsrc.file


-rwxr-xr-x 1 JSMITH  0    0 Jul 13 15:45 ./HelloCpp.C


-rwxr-xr-x 1 JSMITH   0    0 Jul 13 15:45 ./HelloC.c


drwxrwsrwx 2 JSMITH   0    0 Jul 13 15:46 ./data


-rw-rw---- 1 JSMITH   0    0 Jul 13 15:46 ./data/customers.txt


-rw-rw---- 1 JSMITH   0    0 Jul 13 15:46 ./data/notes.txt


-rw-rw-rw- 1 JSMITH   0    0 Jul 13 16:45 ./archive.tar


-rw-rw-rw- 1 JSMITH   0    0 Jul 13 16:26 ./test.tar


tar: 001-2298 For archive file tar and volume 1, 11 files were


processed with 0 bytes read and 92160 bytes written.


tar xvf archive.tar './Hello*' ./data


./HelloCpp.C


./HelloC.c


./data


./data/customers.txt


./data/notes.txt


tar: 001-2298 For archive file tar and volume 1, 11 files were


processed with 0 bytes read and 92160 bytes written

.

tar xvf archive.tar

.

./goodoleboys.sql


./goodoleboys.txt


./qcsrc


./HelloCpp.C


./HelloC.c


./data


./data/customers.txt


./data/notes.txt


./archive.tar


./test.tar


tar: 001-2298 For archive file tar and volume 1, 11 files were


processed with 0 bytes read and 92160 bytes written.

Figure 16.7: The tar utility also restores files from an archive.

The u (Update) option updates files in an archive. Updated files are inserted into the archive again. The listings in Figure 16.8 ( tar tvf ) show the contents of the tar file before and after the update. After the update, note that the files HelloC.c and HelloCpp.C are present in the archive twice with different file timestamps. When the HelloC.c file is extracted, the tar utility processes the archive sequentially (like a tape drive). The HelloC.c file is encountered twice and extracted each time. The result is that the latest version of the HelloC.c file with timestamp "Jul 13 16:58" is extracted.


tar tvf archive.tar


-rwxr-xr-x  1 JSMITH     0      0 Jul 13 15:45 HelloC.c


-rwxr-xr-x  1 JSMITH     0      0 Jul 13 15:45 HelloCpp.C


drwxrwsrwx  2 JSMITH     0      0 Jul 13 15:46 data


-rw-rw----  1 JSMITH     0      0 Jul 13 15:46 data/customers.txt


-rw-rw----  1 JSMITH     0      0 Jul 13 15:46 data/notes.txt


lrwxrwxrwx  1 JSMITH     0      0 Jul 13 16:54 goodoleboys.sql =>


/home/jsmith/src/data/goodoleboys.sql


lrwxrwxrwx  1 JSMITH     0      0 Jul 13 16:54 goodoleboys.txt =>


/home/jsmith/src/data/goodoleboys.txt


lrwxrwxrwx  1 JSMITH     0      0 Jul 13 16:54 qcsrc =>


/qsys.lib/jsmith.lib/qcsrc.file


tar: 001-2298 For archive file tar and volume 1, 10 files were


processed with 0 bytes read and 215040 bytes written.


ls -l Hello*


-rwxr-xr-x 1 JSMITH     0      92160 Jul 13 16:58 HelloC.c


-rwxr-xr-x 1 JSMITH      0      118 Jul 13 16:58 HelloCpp.C


tar uvf archive.tar Hello*


tar: 001-2315 The archive is being read to position to the end


of the archive.


done.


HelloC.c


HelloCpp.C


tar: 001-2298 For archive file tar and volume 1, 10 files were


processed with 0 bytes read and 114176 bytes written.


tar tvf archive.tar


-rwxr-xr-x  1 JSMITH     0      0 Jul 13 15:45 HelloC.c


-rwxr-xr-x  1 JSMITH     0      0 Jul 13 15:45 HelloCpp.C


drwxrwsrwx  2 JSMITH     0      0 Jul 13 15:46 data


-rw-rw----  1 JSMITH     0      0 Jul 13 15:46 data/customers.txt


-rw-rw----  1 JSMITH     0      0 Jul 13 15:46 data/notes.txt


lrwxrwxrwx  1 JSMITH     0      0 Jul 13 16:54 goodoleboys.sql =>


/home/jsmith/src/data/goodoleboys.sql


lrwxrwxrwx  1 JSMITH     0      0 Jul 13 16:54 goodoleboys.txt =>


/home/jsmith/src/data/goodoleboys.txt


lrwxrwxrwx  1 JSMITH     0      0 Jul 13 16:54 qcsrc =>


/qsys.lib/jsmith.lib/qcsrc.file


-rwxr-xr-x  1 JSMITH     0      0 Jul 13 16:58 HelloC.c


-rwxr-xr-x  1 JSMITH     0      0 Jul 13 16:58 HelloCpp.C


tar: 001-2298 For archive file tar and volume 1, 10 files were


processed with 0 bytes read and 215040 bytes written.


tar xvf archive.tar HelloC.c


HelloC.c


HelloC.c


tar: 001-2298 For archive file tar and volume 1, 12 files were


processed with 0 bytes read and 307200 bytes written.