Threaded IO

Threaded I/O

Some applications can process data faster than the data can be delivered to the application. When an application cannot keep the available CPUs busy, the application is said to be I/O-bound .

SAS supports threaded I/O for SAS applications by providing the SAS Scalable Performance Data (SPD) engine. The SPD engine boosts the performance of SAS applications that are I/O bound through parallel processing of partitioned data sets. Partitioned data sets can span disk drives but still be referenced as a single data set. In this way, the SPD engine can read many millions of observations into SAS applications very rapidly by spawning a thread for each data partition and evaluating WHERE expressions in multiple CPUs. SAS 9.1 support for multiple CPUs, for example on a Symmetric Multiprocessor (SMP) machine, and support for multiple disks per CPU make SPD engine's parallel I/O possible. See SAS Scalable Performance Data Engine: Reference for full details on this engine's capabilities.

The benefits of support for multiple disks in SAS 9.1 is not limited to use by the SPD engine. Multiple disks can be on an SMP machine, but they can also be a bank of disks on a single-CPU machine. Increasing the number of disks that can be accessed simultaneously increases the amount of data that can be delivered to an application. This is because reading or writing data to and from disk takes much more time than the associated CPU processing that directs the I/O. Even a single-CPU machine can support multiple disk drives, which boosts performance. When an SMP machine is used, the performance gain can be quite significant because each CPU can support multiple disk drives. However, multiple CPUs cannot speed up I/O from a single disk drive. The minimum configuration for I/O performance gain is at least one controller per CPU and at least two disk drives for each controller. For example, a site with four CPUs should have at least four controllers and eight disk drives.