CCSIDs

CCSIDs (pronounced "see-sids") allow data to be stored in character sets that correspond to different collating sequences and national languages. Support for CCSIDs enhances the ability of the iSeries to act as a server and to exchange data with other systems.

The examples in this section illustrate some of the issues you might encounter when working with IFS files.

Displaying a File's CCSID Attribute

There are several ways to determine a file's CCSID attribute. The ls command displays CCSIDs if the -S option is specified:

ls -S [e-h]*

37 echoprocess.qsh     819 ftpmodel.txt          37 helpme


37 edity.txt           819 goodoleboys.txt

The file dump command, od , displays the CCSID if you specify the - C option. As shown, the CCSID is shown before the dumped contents:

od -C data.ascii

data.ascii CCSID = 819


0000000  066151 067145 020061 005154 064556 062440 031012

0000016

The attr command, introduced with V5R2 and shown below, can also display or set a file's CCSID:

attr data.ascii CCSID

819

Setting the CCSID of an Existing File

You can use the setccsid utility to change the CCSID attribute of a file without changing the data in the file. You might need this capability, for example, after having received a file via FTP, as the system might not tag the file with the proper CCSID. As of V5R2, you can also use the attr utility to change a file's CCSID.

The setccsid and attr utilities are illustrated in Figure 9.30 and 9.31.

ls -S data.ascii


819 data.ascii


/home/jsmith $


cat data.ascii


line 1


line 2


/home/jsmith $


setccsid 37 data.ascii


/home/jsmith $


cat data.ascii


%>   %>    /home/jsmith $

Figure 9.30: The setccsid utility changes a file's CCSID, but not its contents.

attr data.ascii CCSID


819


/home/jsmith $


cat data.ascii


line 1


line 2


/home/jsmith $


attr data.ascii CCSID=37


/home/jsmith $


cat data.ascii


%>   %>    /home/jsmith $

Figure 9.31: You can use the attr command to change the CCSID attribute. As with setccsid, the contents of the file are unaffected.

File Creation

Many Qshell utilities create files. By default, Qshell tags new files with the current job's default CCSID. This is not always desirable, however. For example, a file that is to be accessed from networked PCs might need to be created in ASCII format.

Use the touch utility, shown in Figure 9.32, to create a file with the desired CCSID. In this example, the default CCSID for this job is 37. File "one" was created with CCSID 37 by default. The C option causes touch to create file "two" with CCSID 819. The S option causes the ls command to display each file's CCSID attribute

touch one


/home/JSMITH $


touch -C 819 two


/home/JSMITH $


ls -S1 one two


37 one


819 two

Figure 9.32: Use the touch utility to ensure that a file is created with the proper CCSID.

Environment Variable QIBM_CCSID

The QIBM_CCSID environment variable was added in V5R2. It specifies the CCSID with which new files are to be tagged. It also indicates the CCSIDs to use for certain translations of data. Table 9.14 lists utilities that use the QIBM_CCSID value.

Table 9.14: Qshell Commands for the IFS

Utility	Change Working Directory	Overridden by -C Option?
nohup	CCSID of the nohup.out file	Yes
rexec	CCSID of data sent to a remote system	Yes
sed	CCSID of created file	No
pax, tar	CCSID of file extracted from archive	No
touch	CCSID of a created file	Yes
tee	CCSID of created files	No

The default value of QIBM_CCSID is zero, which tells Qshell and the utilities to use the job's default CCSID. Another special value is 65535, which means that no data translation is to be done.

Preserving the CCSID Attribute during Copy

By default, the cp utility changes the CCSID of an output file to that of the copied file. The effect of this process is demonstrated in Figure 9.32.

ls -1S data*


819 data.ascii


37 data.ebcdic


/home/jsmith $


cat data.ebcdic


line 1


line 2


/home/jsmith $


cp data.ebcdic data.ascii


/home/jsmith $


ls -1S data*


37 data.ascii


37 data.ebcdic


/home/jsmith $


cat data.ascii


line 1


line 2

Figure 9.32: By default, cp changes the CCSID of the target file.

To preserve the output file's existing CCSID, use the -t (text) option, as shown in Figure 9.33.

ls -1S data*


819 data.ascii


37 data.ebcdic


/home/jsmith $


cp -t data.ebcdic data.ascii


/home/jsmith $


ls -1S data*


819 data.ascii


37 data.ebcdic


/home/jsmith $


cat data.ascii


line 1


line 2

Figure 9.33: Include the -t option when copying if you wish to preserve the target file's CCSID attribute.

The Iconv Utility

The iconv utility, which was added to Qshell at V5R2, reads one or more input files (or standard input), converts the data from one CCSID to another CCSID, and writes the converted data to standard output. Here is the syntax of iconv :

iconv -f fromCCSID -t toCCSID [file


]

The QIBM_CCSID environment variable and the C options discussed earlier in this section will usually be enough support for you when you're dealing with data in different code sets or languages. However, there might be a time when you have a file that is corrupted. Perhaps you want to simply translate some data. By sending the file over a network connection, or because of a bug in your application, you have written data to a file that is tagged with a different CCSID. When you look at a file that shows up as garbage, you might have a data or CCSID problem. The iconv utility might be able to help.

Figure 9.34 shows how you might detect and fix such a problem. Using a Qshell utility like cat , you see what looks like data corruption in a file named unicode.txt. Looking at the CCSID of the file, you see that it is set to a U.S. English CCSID of 37. Knowing that the file is a Unicode file that came from your Web application, you use iconv with a Unicode CCSID of 13488 as the from CCSID, and the default CCSID as the to CCSID.

cat unicode.txt

+ ?               _    ?     / % %     ? ?     _  >
 ?    ? _      ?        /      ?              ? 
>   '
/home/jsmith $

ls -S

819 ascii.txt     37 unicode.txt
/home/jsmith $

iconv -f 13488 -t 0 unicode.txt

Now is the time for all good men
to come to the aid of their country

touch -C 13488 unicode2.txt

/home/jsmith $

iconv -f 13488 -t 0 unicode.txt >> unicode2.txt

/home/jsmith $

ls -S

819 ascii.txt     37 unicode.txt    13488 unicode2.txt
/home/jsmith $

cat unicode2.txt

Now is the time for all good men
  to come to the aid of their country

Figure 9.34: Use the iconv utility to do explicit data translations.

The data appears correct, so you know that you've got the CCSID correct. Use the touch utility with the C option to create a file with the correct CCSID, then use iconv again, this time redirecting the output to the newly created file. The iconv utility explicitly translates the data being read to the current CCSID of your session. Then, the system implicitly translates that redirected data to match the 13488 CCSID that you've tagged the file with. The result is that unicode2.txt contains the correctly tagged data.

File Comparisons

The file-comparison utility, cmp , does not allow for differences in CCSID unless you use the -t option. The - t option tells cmp to translate each input file to the job's CCSID before comparing. Figure 9.35 shows how you might use this option. Files "one" and "two" contain the same data, encoded under different CCSIDs. Without the - t option, cmp returns an exit status of 1 because it considers the files to be different. When the - t option is added to the command, cmp returns a status of 0, indicating that the two files contain the same text.

ls -S one two


37 one 819 two


/home/jsmith $


cat one


line 1


line 2


/home/jsmith $


cat two


line 1


line 2


/home/jsmith $


cmp one two ; print $?


one two differ: char 1, line 1


1


/home/jsmith $


cmp -t one two ; print $?

Figure 9.35: Use the t option when comparing files of different CCSIDs.