Smart game programmers realize early on that some problems are harder than others. If you thought that creating a good flight simulator was a piece of cake, I'd tell you that the part that's a bitch isn't simulating the airplane, but simulating the ground. The newbie game programmer could spend all his time creating a great flight model, and when he started the enormous task of representing undulating terrain with smooth detail levels he would fold like laundry.
Games need enormous amounts of data to suspend disbelief on the part of players. No one, not even Rockstar Games, can set their system RAM requirements to hold the entire contents of even one disk of current day optical media. It's also not enough to simply assume that a game will load resources as needed, and the game designers can do what they want. That is a tragic road traveled by many games that never shipped. Most games that suffer fatal frame rate issues ignored their cache constraints.
It's up to programmers to code the best cache they can, and figure out a way to get game level designers, artists, and sound engineers to plan the density of game areas carefully. If everyone succeeds in their task, you get a smooth game that plays well. If you succeed brilliantly you'll get a game that even predicts the future.