2.1 The Big Paradigm Shift

All of this changes in the world of parallel programming. In this world, multiple instructions can execute at the same instant. A single instruction might be broken down into smaller pieces with each piece being executed simultaneously . A program can be broken into multiple tasks that can each execute at the same time. Instead of one task, a program might consist of hundreds or thousands of routines executing concurrently. In the world of parallel programming, the sequence and location of things is not always predictable. Multiple tasks can start at the same time on any processor with no guarantee what task will finish first, in what order they'll finish, or on what processor they will execute. In addition to tasks executing in parallel, a single task may have concurrently executing parts or subtasks. In some configurations, it is possible for the subtasks to run on separate processors, possibly separate computers. Figure 2-1 shows three levels of parallelism that are possible within a single computer program.

Figure 2-1. The three levels of parallelism that are possible within a single computer program.

graphics/02fig01.gif

The programmer and developer's model of a program undergoes a big paradigm shift because of the three levels of parallelism shown in Figure 2-1 and how these levels of parallelism can be distributed to multiple processors. Figure 2-2 illustrates how the three levels of parallelism are combined with the basic parallel processor configurations.

Figure 2-2. The three levels of parallelism combined with the basic parallel processor configurations.

graphics/02fig02.gif

Notice in Figure 2-2 that multiple tasks can run on a single processor even when the computer has more than one processor. This situation can be created by operating system scheduling policies. Scheduling policies, process priorities, thread priorities, and input/output device performance all impact where and for how long a task, subtask, or partial instruction will execute. Figure 2-2 emphasizes the different architectures that a programmer must face when moving from the sequential programming model to a parallel programming model. The model changes from a strictly ordered sequence of tasks to only a partially ordered (possibly unordered) collection of tasks. Parallelism turns order of execution, time of execution, and location of execution into wildcards. Any combination of these wildcards is subject to change values in often unpredictable ways.



Parallel and Distributed Programming Using C++
Parallel and Distributed Programming Using C++
ISBN: 0131013769
EAN: 2147483647
Year: 2002
Pages: 133

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net