13.3 Workloads

 < Free Open Study > 



13.3 Workloads

Workloads are the most crucial part of any performance evaluation project. There are some important considerations when we select workloads for the evaluation, such as services rendered by the workloads, level of details, effective representation, and timeliness.

13.3.1 Workloads description

Workloads can be described as follows:

  1. Processes: To test the OS's ability to handle multiple processes. This includes creating, scheduling, allocating resources, and killing the process. This can be implemented by running a C or Java program that will create a large number of processes and threads. This program will run without termination-that is, infinitely. Using this method the system is bound to crash after a certain number of processes. The program will record in a file the processes and at what time they were created. In this way we will have a good feel for what the OS can handle.

  2. Computation: To test the computational ability of each OS, assuming the hardware is the same for all the configurations. The OS handling of the ALU can be tested by forcing it to execute a large number of difficult mathematical functions.

The following sections include the detailed description of the three workloads considered for the performance evaluation of the operating systems, which are as follows:

  1. A MATLAB program, which does operations on a matrix, such as adding, multiplying, calculating the determinant, and inverse matrices.

  2. An I/O intensive C program.

  3. A C program that creates multiple processes.

Workload programs used by Windows ME/Windows XP/Windows NT/LINUX operating systems

The programs that were used by the three Microsoft groups and one LINUX group are as follows. The first, workload1, is a C program, which opens a number of files and writes a number of bytes to them. It runs nine different experiments three times each. The second is similar to the first except that it opens up a number of processes with different sleep times between the processes to see how many the OS can handle. It runs nine experiments three times each. The third workload is a MATLAB program, which runs a number of matrix operations with different matrix sizes and number of matrices. It runs nine experiments three times each. The three programs have been written to automatically run the experiments and record the data in appropriate files.

Along with these three programs, there is a fourth program called performance monitor, which will record CPU utilization and memory utilization every 500 milliseconds. Then the CPU and memory utilization will be compared with the beginning and ending times of the experiments, and the response time of those experiments are recorded in Excel files.

The first program runs by choosing among 500, 750, and 1,000 bytes to write to a file and among 100, 500, and 1,000 files to write in. This gives a total of nine experiments. Each experiment is run three times. The response time of each of the 27 experiments is recorded in a file. This is done by opening the file, appending to it, and closing it every time a new experiment is run. The program writes the number of bytes, number of files, beginning time of experiment, ending time of experiment, and response time of the experiment in one line in the file, called work1.txt. This file need not be created when the program is run.

The second program is similar to the first. It chooses among 10, 100, and 1,000 processes and sleep times of 0, 100, and 1,000 milliseconds between the processes. As in workload1, it is written to the file with the same format as in work1.txt. It records the number of processes instead of number of bytes and process rate (sleep time) instead of number of files. This is written in work2.txt.

The third program is a MATLAB program. It is composed of two files. The first is matrix.m, which is a program that takes in the number of matrices and the size of the matrix and runs a number of operations on these matrices. The matrix.m file is run by the workload3.m file. This file chooses among matrices or size 10 × 10, 50 × 50, or 100 × 100 elements and uses either 10, 100, or 1,000 matrices. The matrix.m file writes to a file called work3.txt in the same format as workload2 but with matrix size instead of number of processes and matrix number instead of process rate.

The fourth program is the performance monitor program. This program reads the CPU utilization and memory utilization of the system and writes-them to a file specified by the experimenter. The time and the CPU and memory utilization are recorded every 500 milliseconds.

In order to derive information, we had to use the beginning and ending times of an experiment in any of the workloads and then find a timestamp immediately before the beginning time and immediately after the experiment ending time. This would give us a number of readings between the beginning and ending times. We would then take the average of the readings we are concerned with and that would complete the required information for that experiment.

Running these programs on Windows ME

Running these programs on ME proved to be a little difficult at times. Initially, the three workloads were identical for the three OSs with the exception that ME does not have the Process cmd.exe, which is used in workload2 to test the number of processes that the system can handle. Instead, we use the option of mem.exe. It is a similar console application. Also, in order to use the performance monitor program, we had to use our own system calls.

While running the workload2 program, ME was not able to use the mem.exe program after executing a number of processes. It would run some of them and then it would run out of memory. An error message would appear indicating that mem.exe could not be found even though it had just run with the number of processes or lower. This occured at about 100 processes with 100 milliseconds sleep time between processes.

MATLAB program workload for LINUX 7.2

The purpose of this MATLAB program is to determine the memory percentage utilization and the CPU percentage utilization that occur during selected matrix operations written in MATLAB code. These operations were multiplication of matrices, addition of matrices, and inverse and determinant operations. To conduct these experiments, we varied the matrix size from 10 × 10 to 50 × 50 to 100 × 100 and the number of matrices from 10 to 100 to 1,000. According to the project specifications, we classified these parameters as small, medium, and large. The program was run by giving a UNIX command at the command prompt. It was of the form matfinal (experiment number, matrix size, number of matrices).

To calculate the percentage of CPU utilization, we used the MATLAB commands of tic, toc, and CPU time. These commands were called before and after the matrix operations previously mentioned; their results were stored and used to calculate the percentage of CPU utilization. The tic command starts the stopwatch timer, and the toc command stops the stopwatch timer. The toc command returns the time elapsed between its calling, and the call to tic; therefore toc measures the elapsed time for the matrix operations. The CPU time command gives the CPU time that has passed since MATLAB started. This value of CPU time was divided by the toc value to give the CPU utilization.

To calculate the percentage of memory utilization, we used a C program (amitfinal.c), which was called from the MATLAB program before and after the matrix operations. This C program was called from within the MATLAB program by creating a MATLAB executable (mex) format-that is, extending MATLAB with C. The C program made use of UNIX system commands to acquire system information regarding memory-the amount of RAM, available memory, cached memory, and so on. In order to capture these memory values, we wrote the output of the C program into a text file called mem.txt. The contents of this file were read by the MATLAB program by using fid() and fopen() commands.

Taking a look at the before and after values and by using simple mathematical formulas we calculate the memory used by the system in performing the matrix operations. Knowing the total memory used by other processes in the system, we could determine what percentage of memory our MATLAB program used. We could also verify these results by using the top command at the command prompt. The top-command returned the amount of memory used by the MATLAB software (including the memory used to set up the MATLAB software during initialization and the memory taken up by the operations of the MATLAB program), while the MATLAB program code we used returned the memory required for the matrix operations only. It's up to the user to decide which one to use for making appropriate conclusions. In order to run the C program from the MATLAB program, we had to include MATLAB executable commands to compile and run the C program. Finally, the required values, such as size of matrix, number of matrices, percentage of CPU utilization, and percentage of memory utilization, were printed. Since there was no way of verifying the percentage of memory use printed by the MATLAB program, we preferred to use the system monitor.



 < Free Open Study > 



Computer Systems Performance Evaluation and Prediction
Computer Systems Performance Evaluation and Prediction
ISBN: 1555582605
EAN: 2147483647
Year: 2002
Pages: 136

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net