Advanced File IO: Text Files, Binary Files, and Configuration Files

Advanced File I/O: Text Files, Binary Files, and Configuration Files

You've already seen from Chapter 9, "Exploring Strings and File I/O," the most common file I/O functions LabVIEW has to offer. You've seen how to read and write dynamic data types, spreadsheet data, text files, and binary files. You will now learn how to operate on files by reference, performing a sequence of operation and introspection steps, rather than simply reading or writing the file in one fell swoop. This will allow you to achieve some more complex and high-performance use cases.

Opening and Closing File References

A file is a set of ordered bytes on your computer disksimilar to the way a string is a set of ordered characters or an array is a set of ordered elements. Because files can be very big, it is often desirable to not read the entire contents of a file into memory all at once, but rather, only read small portions of it (maybe one line at a time) and perform operations based on the data that is read.

Most of the functions for writing and reading data to and from files use a refnum (short for "reference number") as a handle to a file that has been opened for writing or reading. By handle, we mean that we are not passing around the data of the file, but rather, something that refers to the open file. You can think of a file refnum like a path to an open fileit is not the data of the file, but something that helps us know where to find the data.

The file read and write functions in LabVIEW are polymorphic. You can use them with either file path inputs OR file refnum inputs. In this section, we'll learn how they work if we are using file refnums.

To open a refnum to a file, we use the Open/Create/Replace File function (see Figure 14.6). The refnum will be passed out via the refnum out terminal. When we are done writing or reading the file, we will pass the file refnum to the Close File function (see Figure 14.7), to tell LabVIEW that we no longer need access to the file.

Figure 14.6. Open/Create/Replace File

Figure 14.7. Close File

Now that we know how to open and close a file, let's take a look at how to write and read data.

Advanced File Functions

Sometimes you'll need to do specific file tasks, such as create directories, change the permissions on a specific file, or delete a file. LabVIEW gives you many more file I/O functions in the Programming>>File I/O>>Advanced File Functions subpalette, shown in Figure 14.8.

Figure 14.8. Advanced File Functions palette

We won't cover every one of these functions in this chapter, but we'll discuss a few of them next.

File Position

The file marker is a number that tells us the byte offset where you are currently working in a file. The end of file is the offset past (just beyond) the last byte in the file (meaning the end of the file)the end of file is also the file size.

When you first open a file, the file marker is set to an initial value of 0, and as you write or read to the file, the marker is moved forward each time by the number of bytes written or read. The file marker helps you by keeping track of where you are in the filethere is no need for you to explicitly specify the position where you want to write or read data (although you can specify the position to read or write, if you wish). Generally, you will start at the beginning of a file and keep writing or reading data contiguously until you reach the end of the filethis is typically referred to as sequential access, as compared to random access (explicitly specifying the position of each read and write operation).

For example, if you open a file, the file marker is initialized to 0. Now, if you write 100 characters to the file, the write operation will start at byte 0, and after the write operation, the new value of the file marker will be 100. Then, if you write 50 more characters to the file, the write operation will start at 100 (the current file marker value) and when you are done, the new file marker value will be 150. Pretty easy, isn't it?

So, LabVIEW automatically adjusts the file marker as you write or read the file, but if you wish, you can also move the file marker explicitly to any value you like (so that read and write operations will occur in this new location), using the Set File Position function, shown in Figure 14.9. Use the offset input to define the marker position, and the from input to specify whether the offset value is relative to the start of the file, end of the file, or the current file marker position. Use the Get File Position function, shown in Figure 14.10, to get the current value of the file marker.

Figure 14.9. Set File Position

Figure 14.10. Get File Position

You can use data files on disk, in a similar way to how you might use data arrays in memorythis is a useful fact because data files can be very large compared to the amount of data you can keep in memory (RAM). For example, when you build an array, you append elementswhen you build a file, you append data. You can replace array elements by specifying an index and writing the new elementsyou can replace data in a file by specifying the offset and writing the data (which overwrites the existing data). You can read array elements by specifying their indexyou can read data from a file by specifying the offset to the data. So, if you need to operate on HUGE data sets, consider using binary files without loading the whole thing into memoryjust read and write the portions you need at a given time.

Normally you don't need to manually move the file marker around, but it's good to know you have that option if you are doing some specialized or low-level file I/O.

End of File

The end of file is the offset past the last byte in the filenamely, the file size. Use the Set File Size and Get File Size functions, shown in Figure 14.11 and Figure 14.12, to set and get the end of file (respectively).

Figure 14.11. Set File Size

Figure 14.12. Get File Size

As you write data to a file, the end of file is automatically adjusted if the write operation would move the file marker past the end of file (meaning that the file will grow in size). Increasing the file size using the Set File Size function will cause the file to grow, and be padded with null data (zeros). Conversely, if the file size is reduced by setting the end of file to a value smaller than its current value, the data between the new end of file and the original end of file will be clipped (deleted).

Moving, Copying, and Deleting Files and Folders

The following functions allow you to move, copy, and delete both files and folders. Move will move a file from the source path to the target path (see Figure 14.13).

Figure 14.13. Move

Copy copies a file from the source path to the target path (see Figure 14.14).

Figure 14.14. Copy

Delete deletes a file or directory (see Figure 14.15). WARNING!!! Be very careful using this function. It can delete files or whole directories without any prompting!!! That reminds us, when was the last time we backed up our data?

Figure 14.15. Delete

Creating Folders and Listing Folder Contents

The following functions allow you to create folders and list the files and subfolders contained in folders.

Create Folder will create a directory specified by the path (see Figure 14.16). Note that this function will create multiple parent directories, if they do not already exist. For example, if "C:\Data" does not exist and you attempt to create the directory "C:\Data\Run01," this function will first create "C:\Data" and then "C:\Data\Run01."

Figure 14.16. Create Folder

List Folder returns the contents of a directory specified by path (see Figure 14.17). You can also optionally specify a file pattern to filter the results.

Figure 14.17. List Folder

Activity 14-1: Reading a Text File

You will create a VI that reads the entire contents of a text file. It will display the file's contents in a string indicator and the file's size in a numeric indicator.

1.	Build the front panel shown in Figure 14.18. Figure 14.18. Front panel of the VI you will create during this activity The VI will read the contents of the file specified by the File Path control and display the output in the string indicator. The digital indicator will display the file's length.
2.	Build the block diagram shown in Figure 14.19. Figure 14.19. Block diagram of the VI you will create during this activity

Open/Create/Replace File Function

Open/Create/Replace File function (Programming>>File I/O palette) opens the file and returns a refnum that will be used by the other File I/O VIs.

Get File Size Function

Get File Size function (Programming>>File I/O>>Advanced File Functions palette) returns the size of the file, which will be used for specifying the number of characters to read.

Read from Text File Function

Read from Text File function (Programming>>File I/O palette) returns the number of characters from a file specified by the count input, starting at the current file position (which is zero, in our case). Note that this function is polymorphic. You can wire either a file refnum as we are doing here, OR a file path as we did in Chapter 9, "Exploring Strings and File I/O." If you wire a path and do not wire refnum out, the file reference will be automatically closed.

Close File Function

Close File function (Programming>>File I/O palette) closes the file refnum that was created by Open/Create/Replace File.

Simple Error Handler.vi

Simple Error Handler.vi (Programming>>Dialog & User Interface palette) displays a dialog if there is an error in one of the file I/O functions.

3.	Return to the front panel and enter a path to a text file into the file path control. Make sure that you don't select a file that is too large (which will cause LabVIEW to use a lot of memory), or is a binary file (which will just display funny characters).
4.	Run the VI and look at the file contents that are displayed in the string indicator, and the file size that is displayed in the numeric indicator.
5.	Save and close the VI. Name it Read Text File Advanced.vi and place it in your MYWORK directory. Great jobyou're just made a text file viewer!

Activity 14-2: Writing and Reading Binary Files

In this activity, you'll use the file I/O functions to write a binary file, and then read back that file at arbitrary points in the file.

1.	First, create a VI called Write Binary File Advanced.vi with a front panel like the one shown in Figure 14.20. Figure 14.20. Write Binary File Advanced.vi front panel
2.	Create the block diagram shown in Figure 14.21. You can use the Sine Wave.vi (found in the Signal Processing >>Signal Generation palette) to generate an array that represents a sine wave pattern. Use the File I/O VIs from the Advanced palette to handle the file dialog, opening and creating the file, and closing the file. Figure 14.21. Write Binary File Advanced.vi block diagram Notice that we write an array of DBL (double-precision floating point numbers) to the binary file. It's important that we know this, and that we know that each DBL point takes up 8 bytes on disk, when it comes time to read back this file. You can find out how many bytes each data type in LabVIEW uses from the Fundamentals>>How LabVIEW Stores Data in Memory section of the LabVIEW Help. (Open the LabVIEW Help by selecting Help>>Search the LabVIEW Help from the menu.) Run the Write Binary File Advanced.vi and verify it works. Save your binary file somewhere easily accessible.
3.	Now let's create the VI to read it. Suppose that you wanted a VI that didn't necessarily read the entire file, but gave you the option of reading from an arbitrary point to any other arbitrary point. For example, if your file has stored 1,000 points (not bytes) of data, you might want to read points 100 through 200. Create the front panel shown in Figure 14.22 and save the VI as Read Binary File at Arbitrary Point.vi. Figure 14.22. Read Binary File at Arbitrary Point.vi front panel
4.	The key to being able to read at arbitrary points in the binary file is to use the Set File Position function (refer to Figure 14.9). You can use this function to set the file mark position. Keep in mind the file mark position must be figured in bytes, not "data points." So in our case, because our binary file is an array of DBL numbers, we have 8 bytes for each data point. If you want to set the file mark to data point position 100, you need to set the file mark at 800 bytes. Set File Position
5.	To read the remaining number of data points, we need to tell Read from Binary File how many points to read. There's an input called count on this function that behaves in two different ways: If you wire the "data type" input (recommended), then the "count" input assumes you're telling it to count data points (elements), not bytes. It will automatically figure out how many bytes because it knows the data type. Read from Binary File If you do not wire the "data type" input, then the "count" input assumes you're telling it how many bytes to read, because it doesn't "know" what a data point would be in this file anyway.
6.	Can you do this activity without peeking at the block diagram shown in Figure 14.23? (Note that the TRUE case of the Case Structure is shown for illustration purposes.) Give it a try; you can always come back here to check your work. Figure 14.23. Read Binary File at Arbitrary Point.vi block diagram

If you come back to this assignment after reading about the Type Cast function later in this chapter, you may want to change this diagram to programmatically calculate the size of the data element (a DBL, in this case) instead of using a numeric constant in multiple locations. You can use the String Length function to determine the number of bytes in the flattened data (output by Type Cast) of the data element type, as shown in Figure 14.24.

Figure 14.24. Type casting a DBL numeric to a string and checking the length (number of bytes)

This might not seem to be as easy as just wiring a numeric constant to each location that requires the data length value, but if we ever change the data type of our binary file (for example, from a DBL to a EXT), then we will have to update each of those constants. We might forget about this, and so each of those constants is a potential bug, and thus, a liability.

It might seem that this trick would make the code slower (due to the extra processing steps), but LabVIEW's compiler can determine whether a deterministic calculation is being performed from a constant and replace the calculation with the derived constant (in the compiled code)in this case, an I32 constant equal to 8. This feature is referred to as "constant folding" and is common to most modern compilers.