NetWare Multiprocessing

NetWare 6.5 provides robust multiprocessor support that was first released with NetWare 6. However, multiprocessor support of some type has been available in the NetWare operating system since the time of NetWare 4. This chapter presents an overview of the NetWare 6.5 multiprocessor environment and how some common multiprocessor issues have been resolved.

Novell first introduced multiprocessor functionality, in a limited fashion, with NetWare 4. Although NetWare 4 provided multiprocessor capabilities, all its own core operating system processes were not multiprocessor-enabled . External multiprocessor applications could leverage secondary processors, but all operating system processes, such as disk access, network I/O, and so on, had to be funneled through the primary processor, also known as processor 0 .

With the release of NetWare 5, multiprocessor functionality was rewritten and integrated with the NetWare operating system kernel to create the MPK. This effort made all but a few of the native NetWare 5 processes multiprocessor compatible. Unfortunately, two of the most important processes, namely LAN drivers and disk drivers, remained tied to processor 0.

Finally, with the release of NetWare 6, all core operating system processes are multiprocessor compliant. This means that the entire path between network wire and server storage is now multi-processor aware and can take advantage of multiple processors in the server hardware.

Multiprocessing Improvements

The core of the NetWare operating system is the integrated multiprocessing kernel. The MPK provides symmetric multiprocessing to the NetWare environment. As such, the MPK manages threads, schedules processor resources, handles interrupts and exceptions, and manages access to memory and the I/O subsystems.

As previously mentioned, all NetWare 6.5 software subsystems are now multiprocessor-enabled. NetWare 6.5 can itself take full advantage of the power of a multiprocessing system. Table 11.1 lists many of the NetWare 6.5 services that are multiprocessor-enabled.

Table 11.1. Multiprocessor-Enabled Services in NetWare 6.5

SERVICE TYPE

SERVICE COMPONENTS

Protocol stacks

NetWare Core Protocols (NCP)

Service Location Protocols (SLP)

IP stack

HTTP

Ethernet connectivity

Token ring connectivity

Web Distributed Authoring and Versioning (WebDAV)

Lightweight Directory Access Protocol (LDAP)

Storage services

Novell Storage Services (NSS)

Distributed File Service (DFS)

Protocol services

Request dispatcher

Transport service

Fiber channel disk support

Security services

Novell International Cryptographic Infrastructure (NICI)

Authentication

ConsoleOne authentication snap-ins

Miscellaneous services

Novell eDirectory

Novell Java Virtual Machine (JVM)

Web engines

Additional Web features

In addition to improved performance and greater scalability, NetWare 6.5 multiprocessing offers these benefits:

  • Complete backward compatibility with applications written for the older SMP.NLM (written for NetWare 4.11), as well as any legacy CLIB application that does not support multiprocessing (they are just funneled to processor 0).

  • An integrated Multiprocessing Kernel (MPK) that supports both uniprocessing and multiprocessing platforms.

  • Kernel-level support for preemption. NetWare 6.5 supports applications written for a preemptive multitasking environment natively.

  • A single Platform Support Module (PSM) provides full integration between the multiprocessing hardware and the NetWare MPK.

Running Programs on a Multiprocessor System

When you install NetWare 6.5 on multiprocessor hardware, the MPK determines how many processors are in the system. Next , the kernel scheduler determines the processor on which to run the waiting threads. This decision is based on information about the threads themselves and on the availability of processors.

Three types of programs can run on NetWare 6.5:

  • Multiprocessor safe: Multiprocessor Safe programs are typically NLMs that are not multiprocessing-enabled, but are safe to run in a multiprocessing environment. These programs run on processor 0, which is home to all multiprocessing safe programs. NetWare 6.5 is very accommodating to programs that were written prior to the introduction of the NetWare MPK. These non-multiprocessing-aware applications are automatically scheduled to run on processor 0 upon execution.

  • Multiprocessor compliant: Multiprocessor compliant programs are specifically written to run in a multiprocessing environment. When one of these programs loads, the NetWare 6 scheduler automatically assigns the different threads to available processors. The Intel MPS specification allows programs to indicate whether their specific threads want to run on a specific processor. In this case, the NetWare scheduler will assign that thread to run on the requested processor. Although this functionality is available in NetWare 6.5 for those multiprocessing utilities and other programs that require the ability to run on a specific processor, Novell discourages developers from writing programs this way.

    NOTE

    When a multiprocessing-compliant program is loaded, the NetWare scheduler checks for an available processor on which to run the thread (provided its threads don't request a specific processor). If the first available processor is processor 3, the thread is scheduled to run there. The next thread would go to processor 4, and so on. This assumes that the processors make themselves available in consecutive order. If the system has only one processor, all the applications' threads will be queued up to run on processor 0, which is always the first processor regardless of whether it is an MP or non-MP environment.


  • NetWare OS: Lastly, the NetWare OS is completely MP compliant, allowing its multitude of threads to run on available processors as needed.

All processes that run on your NetWare 6.5 server will fit into one of these categories.

Multiprocessing Concepts

In order to understand multiprocessing in general, and the NetWare multiprocessing architecture in particular, you should be familiar with the multiprocessing concepts described in the following sections.

NOTE

Many people assume that a multiprocessor server with two processors will be twice as powerful as the same server with a single processor. Although this may be the theoretical goal of multiprocessor hardware engineers , you won't see this linear increase in performance in our imperfect world. Generally, as the number of processors in a server increases, the processing power of the system also increases , although to a lesser degree. Practically, this means that a two-processor system gives you about 1.8 times the processing power as the same system with a single processor. A four-processor system delivers roughly 3.5 times as much processing power, and a six-processor system offers about 5.2 times the processing power.


NetWare 6.5 supports up to 32 processors in a single server, which works out to a whole lot of processing horsepower!

Threads

A thread is not some bit of code that the processor is executing. Rather, a thread represents an independent stream of control within the processing environment. Since NetWare was first released, it has been using threads to allow operating system processes to function efficiently . Here's how they work:

  • Processes and threads are not equivalent, but they are similar. The main difference between the two is that a process can typically be swapped out of memory to make room for another process (preemptive), whereas a thread is normally allowed to run to completion once it starts (non-preemptive).

  • NetWare 6.5 keeps track of all the threads that run in the server environment with a scheduler and thread queues. The scheduler is multiprocessor-enabled itself, and an integral part of the NetWare 6.5 MPK. As a result, each processor maintains its own scheduler for managing its thread execution.

  • Each processor also maintains three thread queues for organizing thread execution. The three queues are the Run queue, the Work To Do queue, and the Miscellaneous queue.

  • The threads in the Run queue have priority over threads in the other two queues. Run queue threads are non-blocking , meaning that they do not relinquish control of the processor until they are done. The Run queue is typically reserved for critical systems such as protocol stacks and many of the other NetWare kernel processes.

  • When a Run queue thread completes, the processor checks for additional threads in the Run queue. If no threads are currently in the Run queue, the processor looks in the Work To Do queue. Unlike the Run queue, Work To Do threads can relinquish control of the processor if they rely on less-important functions that can be blocked by the scheduler. The Work To Do queue is usually used by noncritical NetWare services and NLMs.

  • Finally, the Miscellaneous queue is checked after the Work To Do queue. The Miscellaneous queue holds most application threads that are running in the NetWare environment.

NOTE

NetWare Loadable Modules (NLMs) often establish multiple threads, each representing a distinct path of execution. Make sure you don't equate a thread with NLM execution.


In a multiprocessing environment, there are two methods for managing the execution of individual threads:

  • Global Run queues: This approach to distributing threads has the advantage of automatic load balancing. Waiting threads are automatically doled out to the processor that becomes available first. Unfortunately, the Global queue itself becomes the bottleneck as the number of processors and the number of executing threads increases in a system.

  • Local Run queues: This approach does not have the bottleneck problem associated with the Global queue because a Run queue is created for each processor in the system. It also makes it possible to preferentially schedule threads on the processor on which they last ran, which can increase the efficiency of the system as a whole. The downside to Local queues, however, is it becomes necessary to manually spread the load on the processors as evenly as possible. Without a load-balancing mechanism, threads might pile up at one processor while another processor remains idle.

NetWare uses Local queues in NetWare 6.5 because of the scalability advantages over a Global queue solution.A sophisticated load-balancing algorithm was built on top of the Local queue solution to help prevent processor imbalances from occurring.

Load Balancing

When Novell engineers began considering the details of their load-balancing algorithm, they identified two primary requirements:

  • Stability: Load balancing would do little good for the system if it reacted to small changes in thread balance by wasting large amounts of time moving threads back and forth between processors.

  • Quick distribution: When the scheduler identifies a situation that requires load balancing, the algorithm better be able to make the necessary changes very quickly so as not to affect the overall performance of the system.

NetWare 6.5 addresses the issue of stability by using a threshold. The threshold determines how far out of balance the thread distribution must get before the system takes action to fix the situation. The next issue became where to set the threshold.

A low threshold would keep processors in closer balance at the risk of causing excessive thread movement due to frequent load balancing. A high threshold would greatly reduce the risk of excessive thread movement at the risk of having some processors with a significantly higher load than others, reducing the overall performance of the system. To resolve this problem, NetWare 6.5 defines its threshold as a range within which a processor load is deemed acceptable.

To determine where in the load spectrum the threshold should be placed, the scheduler calculates the system-wide load and from that the average processor load, on a regular basis. The average processor load becomes the midpoint of the threshold range, as shown in Figure 11.1.

Figure 11.1. How processor load balancing works in NetWare 6.5.

graphics/11fig01.gif

The upper and lower bounds of the threshold become high/low trigger points for the load balancing system. A processor is overloaded when its load exceeds the high trigger. A processor is underloaded when it is below the low trigger.

When this happens, the scheduler moves threads from the overloaded processor to the underloaded processor to bring the system back into balance. You can actually configure the load-balancing threshold in NoRM by completing the following steps:

  1. From NoRM, select the Set Parameters page and click Multiprocessor.

  2. Click the value currently listed for the System Threshold parameter.

  3. Specify a new value and click OK.

The updated threshold parameters will take effect immediately.

WARNING

Novell strongly recommends using the default value, unless you have carefully considered your reasons for changing it and tested the new setting in a lab environment before committing it to your production server(s). For more information on multiprocessor SET parameters, see Appendix C .


Preemptive Thread Execution

As mentioned previously, when discussing the NetWare MPK queues, NetWare 6.5 does allow certain noncritical threads to be preempted, or blocked, in order to make operation of the system as a whole more efficient.

Earlier versions of NetWare multiprocessor support did not support process preemption. Rather, they implemented a round- robin (first-in, first-out) scheduling policy where threads were scheduled to run in the order they entered the Run queue. This makes for a simple and very fast operating environment, but at the expense of overall flexibility.

To support a preemptive environment, applications must be explicitly written to identify those critical sections that cannot be blocked. These sections are identified to the scheduler so the thread cannot be preempted while in a critical section.

NetWare will preempt an application thread only under the following conditions:

  • The code where the thread is running must be preemptable. This is indicated by a flag set in the module's NLM file format. When the code is loaded into memory, the memory pages are flagged as preemptable.

  • The thread cannot be in a critical section of the code.

  • The thread has run long enough to qualify for preemption. The scheduler checks the elapsed time with every tick.

Support for preemption provides

  • An execution environment that allows simplified application development. Developers can rely on the Scheduler to handle preemption.

  • A way to forcibly manage modules that are monopolizing the processor.

NOTE

The NetWare kernel itself cannot be preempted.


Multiprocessing Memory Issues

In a uniprocessing environment, there is no problem with memory management because a single processor controls all interaction with system memory. Unfortunately, in a multiprocessing environment, things get a lot more complicated, with multiple processors competing for the use of system memory and the I/O channel. Because of this, NetWare must have control logic to manage processor interaction with other subsystems to prevent memory corruption (process crashes) and, basically, to keep the whole thing from melting down.

For example, it is possible for a single application to have multiple threads running on multiple processors simultaneously . In this situation, it is possible that multiple threads need to update the same memory location at the same time. This is known as a race condition . Without the proper thread management, you could easily end up with bad data being written to memory.

To avoid this type of dangerous condition, NetWare 6.5 requires that threads emanating from the same connection run on the same processor. By doing this, NetWare ensures that application threads are queued up and run sequentially to eliminate the possibility of memory corruption.

Beyond this, NetWare 6.5 also has to manage requests for server subsystems between all the application threads that might be executing at any given time. To do this, NetWare 6.5 uses what are known as synchronization primitives , which are processes that manage access to shared resources, so everything stays in sync. Synchronization primitives include the following:

  • Mutually exclusive lock (mutex): This mechanism ensures that only one thread can access a shared resource at a time, such as system memory or the I/O channel.

  • Semaphores: These are somewhat similar to mutexes , but semaphores use counters to control access to RAM memory or other protected resources.

  • Read-write locks: Similar to mutexes, read-write locks work with mutexes to ensure that only one thread at a time has access to a protected resource.

  • Condition variables : These are based on an external station. In so doing, they can be used to synchronize threads. Because they are external to the thread synchronization code, they can be used to ensure that only one thread accesses a protected resource at a time.

NetWare 6.5 also uses two other synchronization primitives that are restricted to the kernel address space (ring 0): spin locks and barriers. These primitives are not accessible to applications that run in a protected address space.

Platform Support Modules

Besides NetWare, all that is necessary to enable multiprocessing on a multiprocessor computer is the Platform Support Module (PSM) for your specific hardware platform and NetWare. No other modules are required.

The PSM is a kind of device driver for the processors in your multiprocessing server. It provides an abstraction layer between the multiprocessor hardware and the NetWare 6.5 operating system that shields NetWare from the details and intricacies of the vendor-specific multiprocessing implementation. It also enables secondary processors to be brought online and taken offline without having to shut down the server.

During installation, NetWare detects multiple processors by reading the multiprocessor configuration table in the server's BIOS. From that information, it determines which of the available NetWare PSM drivers matches the particular multiprocessing hardware platform.

Once installation is complete, you can choose not to load the PSM, which results in NetWare running only on processor 0. By default, the NetWare 6.5 installation routine will add the PSM load line to the STARTUP.NCF so that it will load whenever the server is started.

Novell provides MPS14.PSM, which supports any hardware platform that complies with the Intel Multiprocessor Specification (MPS) v1.1 and v1.4. Compaq also provides a PSM for its specific multiprocessing system requirements.

Intel MPS v1.4 defines an environment in which all the processors in the system work and function together similarly. All the processors in the system share a common I/O subsystem and use the same memory pool. MPS-compatible operating systems, such as NetWare 6.5, can run on systems that comply with this specification without any special modifications.

Because NetWare 6.5 complies with Intel's specification, it will automatically take advantage of all the processors in your MPS-compliant hardware. At this time, most major computer manufacturers already offer multiprocessing systems compatible with the Intel specification.

TIP

More information on the Intel MPS v1.4 specification is available from Intel at http://developer.intel.com/design/intarch/MANUALS/242016.htm.




Novell NetWare 6. 5 Administrator's Handbook
Novell NetWare 6.5 Administrators Handbook
ISBN: 0789729849
EAN: 2147483647
Year: 2002
Pages: 172

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net