User Considerations

User Considerations

The hard disk partition, file system, and cluster size all affect the efficiency with which an application can write digital media files. A user configuring a computer that will run such an application needs to consider these issues.

Hard Disk Partition

If you install the operating system in one partition and write files in a different partition in the same physical drive, the performance of your applications might degrade. Performance degrades because of the additional time it takes to move between one partition (for access to the file system log file, the operating system modules, and so on) and the other partition (for file I/O access).

On the other hand, separate partitions in separate drives can improve the performance of your applications. In this scenario, you install the operating system, the operating system paging file, and your applications in one drive and use the other drive for application data. This approach reduces file I/O time because the movements of the disk heads in each drive are completely independent.

You gain the most benefit when the two hard disk drives are not using a single IDE controller, configured as master and slave; such drives cannot work in parallel. But even this arrangement gives better performance than you can achieve with multiple partitions in a single physical drive.

You should configure one partition per hard disk drive and place separate disk drives on separate IDE controllers if at all possible. (Several SCSI drives can share the same controller.)

File System

When you install Microsoft Windows NT, Windows 2000, or Windows XP, you select whether to use the NTFS or the FAT32 file system. When an application is writing a file to a disk, these file systems extend the file in different ways and the difference affects performance. You should use NTFS.

If an application is written to do so, it can extend a file much more quickly on NTFS than on FAT32, which is important, for example, when you are capturing real-time digital media and writing it to the hard disk. Typically, the file must be repeatedly extended.

The two file systems differ in another way. Every time NTFS changes file metadata (such as the particular clusters that a file occupies), it also logs the change. Doing so guarantees that even in the case of power failure, the file system will be accurate.

Logging takes measurable time, which makes file I/O technically faster on FAT32 than on NTFS. The difference is negligible, however, especially with the larger data blocks that we recommend to developers. When writing 8-KB blocks, NTFS is typically 3 to 5 percent slower than FAT32. When writing 64-KB blocks, NTFS is only 1 to 3 percent slower than FAT32. When writing 256-KB or larger blocks, there is no measurable difference between the file systems because the file metadata changes so slowly.

For computers that use a digital media application that has been optimized for extending files, you should use NTFS. You should also regularly defragment the partition, which will ensure that very large files can be written and read as quickly as possible.

Do not use NTFS compression on the folder or on the entire drive where you store digital media files. First, using compression would require additional processing time for reading and writing the files, which would particularly degrade the writing of streaming media content. Second, the general compression algorithm used by the operating system does not compress digital media content very well. Third, you gain no benefit at all when you read or write files that are already compressed by the application.

Cluster Size

When you format a hard disk volume, you specify the size of the clusters that the file system uses. The size of these clusters also affects performance. It s much faster to write very large files using large clusters than using the default cluster size of 4 KB.

As an application is writing a very large file to the disk, it must frequently extend the size of the file. If the application requests, for example, an additional 64 KB of space for the file, the file system must locate 64 KB of free clusters. If each cluster is 4 KB, the file system must carry out 16 searches in its data structures to locate 16 free clusters. On the other hand, if each cluster is 64 KB, the file system must search only once in its data structures to locate one free cluster. The larger the cluster size, the less time the file system requires to allocate additional space for a very large file.

Larger clusters also reduce file fragmentation. Suppose again that the file system uses the default 4-KB clusters and the application writes data in 64-KB blocks. When the application writes a block of data, the file system tries to put the entire block in contiguous free clusters. In this example, it tries to locate 16 contiguous free clusters.

The file system is much more likely to locate two contiguous 32-KB clusters or one 64-KB cluster immediately following the current end of the file data than it is to locate 16 contiguous free clusters. The larger the cluster size, the less fragmentation of a given file.

However, one drawback to larger cluster sizes is the risk of wasted space on the disk. A cluster is the smallest data unit that the file system handles, and a given cluster can contain data from only a single file. If an application writes a 123-byte file to a file system configured with 64-KB clusters, an entire cluster must be assigned to that file even though almost all of it is empty. Similarly, if the application writes a file that is 64 KB plus one byte, that file must be assigned two entire clusters.

You must consider this tradeoff between performance and wasted space when you configure a hard disk volume. If the volume will have many small files, it should use smaller clusters. If it will have many very large files, it should use larger clusters. If at all possible, you should use different hard disk drives for files that will benefit from very different cluster sizes. For example, install the operating system and applications on a volume with the default cluster size and use a volume with a large cluster size to store large digital media files.

For a hard disk that will store and manipulate very large digital media files, you should configure the file system to use clusters of at least 8 KB, with 64 KB preferred.

Hardware

Hardware considerations do not lead to a choice between NTFS and FAT32. An application that uses carefully implemented, asynchronous, non-buffered, sequential file I/O can easily reach the transfer rate of the hardware. This is true for both file systems.

User Summary

To achieve the best file I/O performance in applications that handle digital media files, users should take these steps:

• Format each physical hard disk drive with only one partition.

• If you have a separate hard disk drive for data files, try to use a separate IDE controller.

• Format the drive where you will store multimedia files using NTFS. Do not use FAT32.

• Format the drive where you will store multimedia files with the largest cluster size that is practical. If it s a separate drive used only to store multimedia files, use clusters at least 64 KB in size.

• Regularly defragment the partition. Do not use NTFS compression on the partition.