This chapter presents a basic, practical, and working knowledge of Markov models. Markov models fit within the general modeling paradigm which involves model construction, solution, calibration, alteration, and validation. Markov models are useful for both descriptive as well as predictive purposes. They are versatile and can model a wide range of applications. Two quite diverse applications are considered in this chapter and each of the modeling steps is demonstrated in detail. The assumptions and limitations of Markov models are summarized. The basics, as well as building blocks for more advanced topics, are presented.
In general, the primary limitation of Markov models is that they are susceptible to state space explosion. This explosion poses a danger that the computational complexity of solving the balance equations is prohibitive.
Fortunately, there are subclasses of Markov models that lend themselves to efficient, alternative solution techniques. One such subclass of models is known as separable Markov models. (The database server example in this chapter is one example of a separable Markov model.) Separable Markov models can be solved using the Mean Value Analysis (MVA) technique, which is the topic of the following chapter.
We believe that the best way to learn about and to understand the subtleties of system modeling is not a passive process, but rather an active engagement. Arguably, the primary benefit of system modeling is the development of keen insights and intuition by the system analyst concerning the interdependencies between various modeling parameters. To this end, a rich set of exercises is provided. The reader is encouraged to participate by attempting to solve these exercises. They are not trivial.