| One of the most important high-level services delivered with the GT3 information services is the index services capability. This service is responsible for managing static and dynamic state data for grid services. Figure 14.3 shows the index service high-level overview. As shown is the illustration, the important components in the index service core are: -
Service data providers -
Data aggregators -
Grid service registries Figure 14.3. An overview of the GT3 Index Service.  The GT3-provided index service has the capability of constructing an interface for connecting external service data provider programs to grid service instances. These providers enable a standard mechanism for dynamic service data generation from external programs. The GT3 core provides a set of sample providers via the SimpleSystemInformation and HostScriptProvider objects. These providers are: -
SimpleSystemInformation. These are native system probes. This enumerates the following data: CPU count, memory statistics, OS type, and logical disk volumes . -
HostScriptProvider. These are Linux-specific shell scripts that monitor system-specific host data. One can always extend these providers or add on new custom providers, depending on the requirements. It is important to note that these providers are configured using the "index-service-config.xml" file. These providers are simple Java applications that are rendering their results of execution to the Java output stream. In the above two cases, the output stream is mapped to Java System.out. In our previous discussions, we reviewed a sample implementation of a provider. One can always add their own provider and register them in a configuration file. A generic framework for aggregating data from other services involves service data originating from various providers (and other services) that can be aggregated and indexed to provide efficient query processing. This index service utilizes the standard OGSI notification mechanisms for subscription, notification, and updating of the service data. A Registry of grid services is a set of available grid services that is maintained in the registry. The registry allows a soft state registration mechanism for services to periodically update the overall services availability. A dynamic data generating and indexing node is another valuable feature of the index service. The index service combines ServiceDataProviderExecution components with ServiceDataAggregator and ServiceGroup components to create a dynamic data generating and indexing node. This capability can be utilized to create hierarchical index services. Index Service Information Model The base information data model used by the index service is XML. The service data is an XML construct and both client and service know how to deal with this data model. The index service information model provides the exposure of the service data to the clients and aggregation of service data from heterogeneous grid services. As we know, every grid service has service data associated with it and the purpose of index services is to provide an interface to access, aggregate, generate, and query this data. The OGSI-defined GridService interface exposed by all grid services provide a unique way to access and update the service data. These operations are "findServiceData" and "setServiceData." The constructs that are of importance for index service are: -
Factory, the creator of a grid service. -
GSH, the unique instance identifier. These GSHs should be converted to GSR before use. -
GSR, the reference to a grid service with information that includes portTypes exposed by a service, and how the client can communicate with a service. A service data query mechanism enables a service to be queried for the service data information. We have already explained in earlier chapters that this is one of the strong areas in the grid service specification and GT3. These queries can be of different types based on the service and the service container capabilities. The types of queries supported by GT3 include query by service data name and query by XPath. Registries are used as a common repository for grid services. These registries allow soft state registration of grid services. The most important aspect of index service is the ability to get notified on service data changes anywhere in the grid system. The OGSI-defined notification mechanism provides this powerful feature. This provides a dynamic resource state and the availability of the updates. Functional Aspects of Index Service A primary function of index service is to provide an interface to a query on an aggregated view of service data collected from other services. The findServiceData operation of the GridService interface is utilized to exploit this query. The index service engine becomes registered as notification sink to other grid services. These notification-enabled grid services establish the subscriptions for notifications on topics of interest that are related to service data. A notification is sent to the sink on state changes. This is illustrated in Figure 14.3. These asynchronously sent notification messages (i.e., XML messages) are used to monitor the status of the resource, and the current service state information. This notification process can be considered as a push mechanism. The other facility provided by GT3 is the provider concept, where the service data gets pulled from the resource utilizing the service data providers. Globus GT3 has a plug-and-play provider mechanism that enables index services to dynamically pull the service data from other services. We can find some integrated clients with GT3 index services, which are service data browsers and a command-line tool called "ogsi-find-service-data." These tools are used to get the service data from a service and display the data for the user . The input for the query can be simple, based on a service data name query, or may be a complex XPath expression. Index Service Configuration Model There can be any number of index services in a service container. The configuration of each of them is available in the server-congif.wsdd file. Listing 14.7. The ServiceData aggregation. <service name="base/index/IndexService" provider="Handler" style="wrapped"> <parameter name="name" value="Index Service"/> <parameter name="schemaPath" value="schema/base/index/index_service.wsdl"/> <parameter name="className" value="org.globus.ogsa.base.index.IndexService"/> <parameter name="baseClassName" value="org.globus.ogsa.impl.base.index.IndexServiceImpl"/> ...................................... <parameter name="serviceConfig" value="index-service-config.xml"/> </service> The Index service deployment shown in Listing 14.7 is similar to other grid services with one notable exception: there is a configuration parameter indicating the index service configuration file. This index-service-config.xml file warrants some further discussion. The functions of the index service configuration file are as follows : -
Specifies the service data provider to be enabled for each of the services referencing this configuration file. -
Specifies which of the enabled providers are to be executed at startup and/or when the configuration file is read. It contains the necessary parameters relevant to the provider's execution. -
Specifies notification and subscription of service data to other service instances, which allows for aggregation of service data from multiple services. A sample configuration is described in Listing 14.8. Listing 14.8. The service data aggregation. <serviceConfiguration ...> <installedProviders> <providerEntry class="org.globus.ogsa.impl.base.providers.servicedata .impl.SimpleSystemInformationProvider"/> <providerEntry class="org.globus.ogsa.impl.base.providers.servicedata .impl.HostScriptProvider" handler="com.test.myCallBackObject"/> </installedProviders> <executedProviders> <provider-exec:ServiceDataProviderExecution> <provider-exec:serviceDataProviderName>SystemInformation </provider-exec:serviceDataProviderName> <provider-exec:serviceDataProviderImpl> org.globus.ogsa.impl.base.providers.servicedata.impl .SimpleSystemInformation </provider-exec:serviceDataProviderImpl> <provider-exec:serviceDataProviderArgs></provider- exec:serviceDataProviderArgs> <provider-exec:serviceDataName xmlns:mds="http://glue.base.ogsa.globus.org/ce/1.1"> mds:Host </provider-exec:serviceDataName> <provider-exec:refreshFrequency> 60 </provider-exec:refreshFrequency> <provider-exec:async> true </provider-exec:async> </provider-exec:ServiceDataProviderExecution> </executedProviders> <aggregatedSubscriptions> <aggregator:AggregatorSubscription> <aggregator:serviceDataName xmlns:ce="http://glue.base.ogsa.globus .org/ce/1.1"> ce:Cluster </aggregator:serviceDataName> <ogsi:source> http://127.0.0.1:8080/ogsa/services/base/gram/ MasterForkManagedJobFactoryService </ogsi:source> <aggregator:lifetime> 1200 </aggregator:lifetime> </aggregator:AggregatorSubscription> </aggregatedSubscriptions> </serviceConfiguration> Listing 14.8 illustrates some of the important configuration information for an index service provider. There are three sections in this configuration file. -
Installed providers. These are the available providers in the container. These may be GT3-provided or custom Java providers. These providers must implement either of the DOMDataProvider, SimpleDataProvider, or AsyncDataProvider interface. In Listing 14.8, we have two providers installed, SimpleSystemInformationProvider and HostScriptProvider. There is only one required parameter for each provider entry, a "class" attribute that informs the implementation class name of the provider. There can be an optional "handler" attribute indicating a user-provided custom callback object for post-processing of the data. This handler must implement the ServiceDataProviderDocumentCallback interface. -
Executed providers. These providers are executed, producing one or more service data results. This configuration allows one to pass parameters to this execution behavior. The service data name parameter indicates the name of the new service data to create. If this parameter is not present, the service data name will be created from the tag name of the root element of the XML document produced by the execution. The "refresh" property indicates how often the provider should execute. -
Aggregated subscriptions. These subscriptions specify the grid services to be indexed by the index service. Here, we are specifying the GSH of the grid service from which we are receiving notifications, and the service data name for which we are subscribing. These subscriptions are controlled by the lifetime attributes. Monitoring and Discovery The MDS in GT3 is merged with the core GT3 (i.e., service data and notification model), and the higher-level index service (i.e., the collective layers functionalities). One must be aware of this fundamental change. This way, there will not be a specific MDS component in GT3. However, at the same time GT3 provides all the facilities available with MDS (version 2) through the previously described core and index services. Summary The above index service architecture provides a data aggregation (push and pull mechanisms) from external services and data sources, service collection management, while providing an information model based on service data aggregation XML schema. These are valuable sources of information for applications built upon Globus GT3 high-level services. The sources can exploit these indexing behaviors of the data collected from various services/resources for purposes of creating meaningful models, such as resource utilization, availability, performance, and cost matrices. | 