Resource Caching

This is one of those hidden systems I told you about. A game like Grand Theft Auto: Vice City has gigabytes of art and sound, and the CPU only has a fraction of the memory, perhaps less than one hundred megabytes. Imagine the problem of getting a crowd of people out of a burning building. Left to their own devices, the crowd will panic, attempt to force themselves through every available exit, and only a small fraction of the people will escape alive.

Now imagine another scenario, where the evacuation was completely organized. The crowd would divide themselves into single file lines, each line going out the nearest exit. If the lines didn't cross, people could almost run. It would be very likely that even a large building could be completely evacuated.

This analogy works well for game resources. The burning building is your optical media, and the doors are the limited bandwidth you have for streaming this media. The bits on the disk represent the crowd. Your job is to figure out a way to get as many of the bits from the optical media into memory in the shortest possible time. That's not the entire problem, though. A resource cache is exactly what the name implies—commonly used assets like the graphic for a mouse pointer are always in memory, and rarely used assets like the cinematic endgame are only in memory while it's playing, and most likely only a piece of it at that.

The resource cache manages assets like these in a way that fools the game into thinking that they are always in memory. If everything works well the game will never have to wait for anything, since the resource cache should be smart enough to predict which assets will be used and load them before they are needed.

Every now and then the cache might miscalculate and suffer a cache miss. Depending on the resource, the game might be able to continue without it until it has been loaded, such as the graphics for an object in the far distance. In that case the graphic can fade in once it is safely in memory. In other cases, the game isn't so lucky, such as a missing audio file for a character's lines. Since they are synched to the facial animations, the game has to wait until the audio is loaded before the character can begin speaking.

So, it's not enough to write a little cache that knows whether resources exist in memory at the moment they are needed. It has to be clever, predicting the future to some extent and even providing the game with a backup in case the cache suffers a miss.