Resource Files

Many hard disks rotate as fast as 7200rpm. This means that on average, the processor must wait an average of 4ms for a desired piece of data to be located in the right position to be read, assuming the read/write head doesn't have to seek to a new track. For a modern day processor operating at 2GHz or more, this time is interminable. It's a good thing processors aren't conscious because they'd go mad waiting for hard disks all the time. Seeking time is much slower. The head must accelerate, move, stop, and become stable enough to accurately read the magnetic media. For a CPU, that wait is an eternity.

Optical media is even worse. Their physical organization is a continuous spiral from the inside of the disc to the outside, and the read laser must traverse this spiral at a constant linear velocity. This means that not only does the laser read head have to seek to an approximate location instead of an exact location, but the rotational velocity of the disc must change to the right speed before reading can begin. If the approximate location was wrong, the head will re-seek. All this mechanical movement makes optical media exceedingly slow. This is why Microsoft put a hard drive in the Xbox. If you really want to know why play Halo. Halo uses the hard drive to cache level files from the DVD-ROM enabling huge levels without any noticeable load time.

The only thing slower than reading data from a hard drive or optical media is to have an intern actually type the data in manually from the keyboard.

Needless to say you want to treat data in your files like I treat baubles in stores like Pier One. I do everything in my power to stay away from these establishments (my wife loves them) until I have a big list of things to buy. When I can't put it off any longer I make my shopping trip a surgical strike. I go in, get my stuff, and get out as fast as I can. When your game needs to grab data from the hard drive or DVD, it should follow the same philosophy.

The best solution would completely compartmentalize game assets into a single block of data that could be read in one operation with a minimum of movement of the read/write head. Everything needed for a screen or a level would be completely covered by this single read. This is usually impractical because some common data would have to be duplicated in each block. A fine compromise factors the common data in one block leaving and the data specific for each level or screen in their own blocks. When the game loads it is likely you'll notice two seeks—one for the common data block and one for the level specific block. You should make sure the common data stays in memory even if new levels are loaded.

Best Practice

Knowing how hardware works is critical to writing any kind of software. You don't have to be a guru writing device drivers to crack the books and learn exactly how everything works and how you can take advantage of it. This same lesson applies to the operating system and how the hardware APIs work under the hood. This knowledge separates armchair game programmers from professional game programmers.

Packaging Resources into a Single File

It's a serious mistake to store every game asset such as a bitmap or sound effect in their own file. You might want to do this to keep your game architecture open so that your players can add their own player skins, but separating thousands of assets in their own files wastes valuable storage space and makes it impossible to get your load times fast.

It turns out that hard drives are logically organized into blocks or clusters that have surprisingly large sizes. Most hard drives in the gigabit range have cluster sizes of 16Kb–32Kb. File systems like FAT32 and NTFS were written to store a maximum of one file per cluster to enable optimal storage of the directory structure. This means that if you had 500 sound effect files, each 1/2 second long and recorded at 44KHz mono, you'd have 5.13Mb of wasted space on the hard disk:

 0.5 seconds * 44KHz mono = 22,000 bytes 32,768 bytes minimum cluster size - 22,000 bytes in each file = 10,768 bytes wasted per file 10,768 bytes wasted in each file * 500 files = 5.13Mb wasted space 

You can easily get around this problem by packing your game assets into a single file. If you've ever played with DOOM level editors you're familiar with WAD files; they are a perfect example of this technique. These packed file formats are file systems in miniature, although most are read only. Ultima VIII and Ultima IX had a read/write version (FLX files) that had multiuser locking abilities for development. Almost every game on the market uses some custom packing scheme for more reasons than saving hard drive space.

Other Benefits of Packaging Resources

The biggest advantage of combining your resources by far is load time optimization. Opening files is an extremely slow operation on most operating systems, and Windows is no exception. At worst you'll incur the cost of an extra hard disk seek to read the directory structure to find the physical location of the file.

Another advantage is security. You can use a proprietary logical organization of the file that will hamper arm chair hackers from getting to your art and sounds. While this security is quite light and serious hackers will usually break it before the sun sets the first day your game is on the shelves, it's better than nothing.

Another advantage is managing production. You can track where the original assets are on your internal network and whether they've been locked down against changes. You can even track who was the last person to import them. Of course, you'll want to store this information in a separate file that doesn't ship with your product.

Best Practice

During development keep your ear tuned to the sounds your hard drive makes while you play your game. At worst you should hear a "tick" every few seconds or so as new data is cached in. This would be common in a game like Ultima IX, where the player could walk anywhere on an enormous outdoor map. At best your game will have a level design that grabs all the data in one read.

A great trick is to keep indexes or file headers in memory while the file is open. These are usually placed at the beginning of a file, and on large files the index might be a considerable physical distance away from your data. Read the index once and keep it around to save yourself that extra seek.