NetWare 6.5 provides robust multiprocessor support that was first released with NetWare 6. However, multiprocessor support of some type has been available in the NetWare operating system since the time of NetWare 4. This chapter presents an overview of the NetWare 6.5 multiprocessor environment and how some common multiprocessor issues have been resolved.
Novell first introduced multiprocessor functionality, in a limited fashion, with NetWare 4. Although NetWare 4 provided multiprocessor capabilities, all its own core operating system processes were not multiprocessor-enabled . External multiprocessor applications could leverage secondary processors, but all operating system processes, such as disk access, network I/O, and so on, had to be funneled through the primary processor, also known as processor 0 .
With the release of NetWare 5, multiprocessor functionality was rewritten and integrated with the NetWare operating system kernel to create the MPK. This effort made all but a few of the native NetWare 5 processes multiprocessor compatible. Unfortunately, two of the most important processes, namely LAN drivers and disk drivers, remained tied to processor 0.
Finally, with the release of NetWare 6, all core operating system processes are multiprocessor compliant. This means that the entire path between network wire and server storage is now multi-processor aware and can take advantage of multiple processors in the server hardware.
The core of the NetWare operating system is the integrated multiprocessing kernel. The MPK provides symmetric multiprocessing to the NetWare environment. As such, the MPK manages threads, schedules processor resources, handles interrupts and exceptions, and manages access to memory and the I/O subsystems.
As previously mentioned, all NetWare 6.5 software subsystems are now multiprocessor-enabled. NetWare 6.5 can itself take full advantage of the power of a multiprocessing system. Table 11.1 lists many of the NetWare 6.5 services that are multiprocessor-enabled.
Table 11.1. Multiprocessor-Enabled Services in NetWare 6.5
In addition to improved performance and greater scalability, NetWare 6.5 multiprocessing offers these benefits:
Running Programs on a Multiprocessor System
When you install NetWare 6.5 on multiprocessor hardware, the MPK determines how many processors are in the system. Next , the kernel scheduler determines the processor on which to run the waiting threads. This decision is based on information about the threads themselves and on the availability of processors.
Three types of programs can run on NetWare 6.5:
All processes that run on your NetWare 6.5 server will fit into one of these categories.
In order to understand multiprocessing in general, and the NetWare multiprocessing architecture in particular, you should be familiar with the multiprocessing concepts described in the following sections.
Many people assume that a multiprocessor server with two processors will be twice as powerful as the same server with a single processor. Although this may be the theoretical goal of multiprocessor hardware engineers , you won't see this linear increase in performance in our imperfect world. Generally, as the number of processors in a server increases, the processing power of the system also increases , although to a lesser degree. Practically, this means that a two-processor system gives you about 1.8 times the processing power as the same system with a single processor. A four-processor system delivers roughly 3.5 times as much processing power, and a six-processor system offers about 5.2 times the processing power.
NetWare 6.5 supports up to 32 processors in a single server, which works out to a whole lot of processing horsepower!
A thread is not some bit of code that the processor is executing. Rather, a thread represents an independent stream of control within the processing environment. Since NetWare was first released, it has been using threads to allow operating system processes to function efficiently . Here's how they work:
NetWare Loadable Modules (NLMs) often establish multiple threads, each representing a distinct path of execution. Make sure you don't equate a thread with NLM execution.
In a multiprocessing environment, there are two methods for managing the execution of individual threads:
NetWare uses Local queues in NetWare 6.5 because of the scalability advantages over a Global queue solution.A sophisticated load-balancing algorithm was built on top of the Local queue solution to help prevent processor imbalances from occurring.
When Novell engineers began considering the details of their load-balancing algorithm, they identified two primary requirements:
NetWare 6.5 addresses the issue of stability by using a threshold. The threshold determines how far out of balance the thread distribution must get before the system takes action to fix the situation. The next issue became where to set the threshold.
A low threshold would keep processors in closer balance at the risk of causing excessive thread movement due to frequent load balancing. A high threshold would greatly reduce the risk of excessive thread movement at the risk of having some processors with a significantly higher load than others, reducing the overall performance of the system. To resolve this problem, NetWare 6.5 defines its threshold as a range within which a processor load is deemed acceptable.
To determine where in the load spectrum the threshold should be placed, the scheduler calculates the system-wide load and from that the average processor load, on a regular basis. The average processor load becomes the midpoint of the threshold range, as shown in Figure 11.1.
Figure 11.1. How processor load balancing works in NetWare 6.5.
The upper and lower bounds of the threshold become high/low trigger points for the load balancing system. A processor is overloaded when its load exceeds the high trigger. A processor is underloaded when it is below the low trigger.
When this happens, the scheduler moves threads from the overloaded processor to the underloaded processor to bring the system back into balance. You can actually configure the load-balancing threshold in NoRM by completing the following steps:
The updated threshold parameters will take effect immediately.
Novell strongly recommends using the default value, unless you have carefully considered your reasons for changing it and tested the new setting in a lab environment before committing it to your production server(s). For more information on multiprocessor SET parameters, see Appendix C .
Preemptive Thread Execution
As mentioned previously, when discussing the NetWare MPK queues, NetWare 6.5 does allow certain noncritical threads to be preempted, or blocked, in order to make operation of the system as a whole more efficient.
Earlier versions of NetWare multiprocessor support did not support process preemption. Rather, they implemented a round- robin (first-in, first-out) scheduling policy where threads were scheduled to run in the order they entered the Run queue. This makes for a simple and very fast operating environment, but at the expense of overall flexibility.
To support a preemptive environment, applications must be explicitly written to identify those critical sections that cannot be blocked. These sections are identified to the scheduler so the thread cannot be preempted while in a critical section.
NetWare will preempt an application thread only under the following conditions:
Support for preemption provides
The NetWare kernel itself cannot be preempted.
Multiprocessing Memory Issues
In a uniprocessing environment, there is no problem with memory management because a single processor controls all interaction with system memory. Unfortunately, in a multiprocessing environment, things get a lot more complicated, with multiple processors competing for the use of system memory and the I/O channel. Because of this, NetWare must have control logic to manage processor interaction with other subsystems to prevent memory corruption (process crashes) and, basically, to keep the whole thing from melting down.
For example, it is possible for a single application to have multiple threads running on multiple processors simultaneously . In this situation, it is possible that multiple threads need to update the same memory location at the same time. This is known as a race condition . Without the proper thread management, you could easily end up with bad data being written to memory.
To avoid this type of dangerous condition, NetWare 6.5 requires that threads emanating from the same connection run on the same processor. By doing this, NetWare ensures that application threads are queued up and run sequentially to eliminate the possibility of memory corruption.
Beyond this, NetWare 6.5 also has to manage requests for server subsystems between all the application threads that might be executing at any given time. To do this, NetWare 6.5 uses what are known as synchronization primitives , which are processes that manage access to shared resources, so everything stays in sync. Synchronization primitives include the following:
NetWare 6.5 also uses two other synchronization primitives that are restricted to the kernel address space (ring 0): spin locks and barriers. These primitives are not accessible to applications that run in a protected address space.
Platform Support Modules
Besides NetWare, all that is necessary to enable multiprocessing on a multiprocessor computer is the Platform Support Module (PSM) for your specific hardware platform and NetWare. No other modules are required.
The PSM is a kind of device driver for the processors in your multiprocessing server. It provides an abstraction layer between the multiprocessor hardware and the NetWare 6.5 operating system that shields NetWare from the details and intricacies of the vendor-specific multiprocessing implementation. It also enables secondary processors to be brought online and taken offline without having to shut down the server.
During installation, NetWare detects multiple processors by reading the multiprocessor configuration table in the server's BIOS. From that information, it determines which of the available NetWare PSM drivers matches the particular multiprocessing hardware platform.
Once installation is complete, you can choose not to load the PSM, which results in NetWare running only on processor 0. By default, the NetWare 6.5 installation routine will add the PSM load line to the STARTUP.NCF so that it will load whenever the server is started.
Novell provides MPS14.PSM, which supports any hardware platform that complies with the Intel Multiprocessor Specification (MPS) v1.1 and v1.4. Compaq also provides a PSM for its specific multiprocessing system requirements.
Intel MPS v1.4 defines an environment in which all the processors in the system work and function together similarly. All the processors in the system share a common I/O subsystem and use the same memory pool. MPS-compatible operating systems, such as NetWare 6.5, can run on systems that comply with this specification without any special modifications.
Because NetWare 6.5 complies with Intel's specification, it will automatically take advantage of all the processors in your MPS-compliant hardware. At this time, most major computer manufacturers already offer multiprocessing systems compatible with the Intel specification.
More information on the Intel MPS v1.4 specification is available from Intel at http://developer.intel.com/design/intarch/MANUALS/242016.htm.