Under-engineering is far more common than over-engineering. We under-engineer when we produce poorly designed software. This may occur for several reasons.
We don't have time, don't make time, or aren't given time to refactor.
We aren't knowledgeable about good software design.
We're expected to quickly add new features to existing systems.
We're made to work on too many projects at once.
Over time, under-engineered software becomes an expensive, difficult-to-maintain or unmaintainable mess. Brian Foote and Joseph Yoder, who authored a pattern language called Big Ball of Mud, describe such software like this:
Data structures may be haphazardly constructed, or even next to non-existent. Everything talks to everything else. Every shred of important state data may be global. Where state information is compartmentalized, it may be passed promiscuously about though Byzantine back channels that circumvent the system's original structure.
Variable and function names might be uninformative, or even misleading. Functions themselves may make extensive use of global variables, as well as long lists of poorly defined parameters. The functions themselves are lengthy and convoluted, and perform several unrelated tasks. Code is duplicated. The flow of control is hard to understand, and difficult to follow. The programmer's intent is next to impossible to discern. The code is simply unreadable, and borders on indecipherable. The code exhibits the unmistakable signs of patch after patch at the hands of multiple maintainers, each of whom barely understood the consequences of what he or she was doing. [Foote and Yoder, 661]
While systems you've worked on may not be so gruesome, it's likely you've done some under-engineering. I know I have. There's simply an overwhelming urge to get code working quickly, and it's often coupled with powerful forces that impede our ability to improve the design of our existing code. In some cases, we consciously don't improve our code because we know (or think we know) it won't have a long shelf life. Other times, we're compelled to not improve our code because well-meaning managers explain that our organization will be more competitive and successful if we "don't fix what ain't broke."
Continuous under-engineering leads to the "fast, slow, slower" rhythm of software development, which goes something like this.
You quickly deliver release 1.0 of a system with junky code.
You deliver release 2.0 of the system, and the junky code slows you down.
As you attempt to deliver future releases, you go slower and slower as the junky code multiplies, until people lose faith in the system, the programmers, and even the process that got everyone into this position.
Somewhere during or after release 4.0, you realize you can't win. You begin exploring the option of a total rewrite.
This kind of experience is far too common in our industry. It's costly and it makes organizations far less competitive than they could be. Fortunately, there is a better way.