Data Compression

Compression is a double-edged sword. Every game scrambles to store as much content on the distribution media and the hard drive as possible. Compression can achieve some impressive space ratios for storing text, graphics, and sound at the cost of increasing the load on the CPU and your RAM budget to decompress everything. The actual compression ratios you'll get from using different utilities is completely dependant on the algorithm and the data to be compressed. Use algorithms like ZLIB or LZH for general compression that can't afford lossiness. Use JPG or MPEG compression for anything that can stand lossiness such as graphics and sound.

Consider the cost of decompressing MP3 files for music, speech, or sound effects. On the upper end, each stream of 128K stereo MP3 under the Miles Sound System from RAD Game Tools, Inc. will suck about 25Mhz from your CPU budget, depending on your processor. If you design your audio system to handle 16 simultaneous streams, a 2GHz desktop will only have 1.6GHz left, losing 400Mhz to decompressing audio.

Best Practice

You can always trade some loading time and system RAM for real time decompression. Instead of decoding MP3 streams in real time, decompress them into a WAV buffer (Miles Sound will let you do this) and keep it in memory. You can use this trick for almost any compressed data stream.

Decompression limits overall performance because of all the extra RAM it uses. The compressed data stream must be read into memory and decompressed in chunks into a temporary buffer, usually allocated by the decompressor. These decompressed blocks are usually copied into the destination buffer. Copying this data around takes time. On operating systems that use virtual memory, this copying can cause page faulting in virtual memory—a real nightmare.

Whether or not you'll use compression is not the question. Almost every game compresses some data to get the most out of permanent storage. Be careful about monitoring the costs of using compression in your data files; loading files can be choppy and decoding compressed streams of data in real time can use more CPU budget than you have to give.

Gotcha

If your decompressor API uses a callback it is quite likely that the decompression will forward Windows system messages into your message pump. This can create a real nightmare since mouse clicks or hotkeys can cause new art and sounds to be recursively sent into the decompression system. Callbacks are necessary for proving user feedback like a progress bar, but they can also play havoc with your message pump. If this is happening to your application, trap the offending messages and hold them in a temporary queue until the primary decompression is finished.

Zlib: Open Source Compression

If you need a lossless compression/decompression system for your game, don't look any further than Zlib which is found at http://www.gzip.org/zlib/. It's free, open source, legally unencumbered and simple to integrate into almost any platform. Typical compression ratios with Zlib are 2:1 to 5:1, depending on the data stream.