Resource Management Services

   

Grid resource management involves the coordination of a number of components, including resource registries, factories, discovery, monitoring, allocation, and data access. The Globus toolkit includes a set of components to help users have a standard set of interfaces for the coordination of the above activities.

The main functionalities of these components (called GRAM) are job submission and resource management. Figure 14.5 shows the architectural components of the GRAM. Even though this architecture appears complex, in practice it is an aggregation and collective functionality of a set of well-defined components. Another advantage of this architecture is that the client is free from these architecture complexities.

Figure 14.5. The architecture of GRAM.

The Major Components of the GRAM Architecture

Master Host Environment and User Host Environment

This concept enables the separation of functionalities and provides an improved abstraction on the functions supported by each environment. The master host is responsible for providing information on aggregated resource state/status, and to manage its user hosts start and launch services. This is the direct point of contact for the client. The user host environment (UHE) provides specific abstraction capabilities and securities to execute a job. All the jobs are executed in the UHE.

Master Managed Job Factory Service

This factory is always available and responsible for receiving the client request on aggregated resource queries and subscriptions. This factory manages the aggregated service data through aggregation providers for resource status, and notifications from resources.

Virtual Host Redirect Handler

This is the core component responsible for redirecting all of the calls to the UHE. These calls include creation of the job and invoking job operations on the created job service.

Starter UHE /Launch UHE

On a client request to execute a job, the virtual hosting engine directs the calls to the starter. This Java class is responsible for security mapping, user validation, and for ensuring that the job is executing so that the virtual host can redirect the calls to the executing job service. If the user host is not up and running, then it uses the help of the Launch UHE Java class to start that host under the user's credentials.

Managed Job Factory Service/Managed Job Service

The MJF service exposes a CreateService method, which accepts an RSL-specified job. The MJF then creates a managed job instance for the user. It acts as a local scheduler, monitoring its status and sending notifications. In addition to this, the MJS will start two file streaming factory services; one for the job's stdout , and the other for the job's stderr.

File Stream Factory Service/File Stream Service

These are helpful services to manage the data needed for the job execution. The factory service creates two file stream services: stdout and stderr. Each of these services has two service data results: the URL for the stream destination, and a flag to indicate the activity.

Grid Resource Identity Mapper ( GRIM )

The GRIM service is executing in the UHE to create a user host certificate. The user host certificate is utilized for mutual authentication between the MJS service and the client.

Resource Information Provider Service ( RIPS )

A specialized notification service providing data about a scheduling system, file system, host, etc.

Two Aspects to the GRAM Architecture

1. Job submission. A user starts the job scheduling with the creation of a managed job service. This process involves multiple steps.

   • The client knows about the master host environment and the master managed factory service. It sends a request to create a job to the master factory service in the MHE.

   • The Virtual Hosting Redirect handler manages this request with the help of the starter/launch Java classes; it then creates and/or connects to the user hosting system. One must be aware that these processes are created under user credentials. Once the UHS is available, the Redirect handler transfers the Create Job request to the managed job factory service in UHS.

   • The Managed Job Factory creates a managed job service instance for the user with the necessary job description parameter encoded in the Resource Specification Language. Also, it creates the necessary file streaming classes to handle the data for the job.

   • From this stage forward, the client is able to talk to the managed job created in the user hosting environment. All the calls are then redirected to the correct job instance.

2. Resource management. A client knows about the master host environment and the master managed factory service. This process involves multiple steps.

   • The master factory provides an aggregated view of the state of the job and other resource information. The master factory gathers this information using the RIP service.

   • The client can now find and register for notification on the aggregated service state from the master factory.

   • The client can now locate the state information from the job service instance.

Resource Specification Language

The power of GRAM is associated with its capabilities to describe a job. The Resource Specification Language (RSL) is an XML standard for describing a job. The RSL includes XML standard elements for specifying executables, directory, arguments, and data files.

The RSL is created by the client, and passed to the managed job service (MJS) where it is then converted to the local scheduler-specific language. The current standard language is RSL2. Listing 14.10 describes this RSL example.

Listing 14.10. An RSL example.

 <rsl:rsl <!---insert GRAM RSL Namespace --->   <gram:job>       <gram:executable>          <rsl:pathElement path="/bin/ls"/>       </gram:executable>       <gram:directory>          <rsl:pathElement path="/user/jjp"/>       </gram:directory>       <gram:arguments>          <gram:argument>-l</gram:argument>          <gram:argument>-a</gram:argument>       </gram:arguments>   </gram:job> </rsl:rsl> 

Summary

GRAM provides a set of standard interfaces and components to collectively manage a job task, and to provide resource information including job status and resource configuration. This information can then be utilized for resource allocation of a specific Job.

We can build high-level meta-job schedulers , and resource brokers /managers on the top of this layer, while using the resource information provided by this layer. The other aspect of using this standardized middle layer is that we can also build lower-layer local jobs and resource management functions, without affecting the high-level schedulers and monitors . This layer provides an abstraction from high-level manageability to lower-layer manageability functionalities. One notable aspect of the GRAM is the absence of accounting and billing features.