Rationale, Background, and Design Constraints

The following records those contextual and requirements aspects of ECS that were the primary motivators behind the major architectural decisions. There were, of course, a large number of specific requirements that had to be satisfied, but these are the major ones that had profound architectural influence.

• Business context: ECS is expected to have a long operational life (1999 to around 2012), meaning that it needs to be easy to change in a variety of ways. It will be installed and distributed across four data centers in South Dakota, Maryland, Virginia, and Colorado. It is designed to serve a broad community of stakeholders: the data centers' staffs, instrument-specific science teams, interdisciplinary science users, other agencies and data user groups, and even the general public. Decentralized authority is the rule; data centers maintain control of their own facilities and interface with data users, and instrument teams maintain control over their science software and data products. ECS represents an unprecedented problem size for NASA and must support about 1,300 product types. It must archive large amounts of data, adding about 1 terabyte per day (today), about 3 picobytes by 12/03, and about 9 picobytes by 2011. It must manage large geospatial databases adding about 140 megabytes per day, or about 293 gigabytes by 12/03. And it must distribute large amounts of data in several formats: almost 2 terabytes per day. ECS must execute complex science algorithms provided by the science community. It must interface with about 35 external systems. ECS is a large system with heavy dependence on COTS. The architecture includes almost 50 COTS products, many integrated with custom code.
• Key data management features: In order to support the widely varying data needs, ECS relies on a common metadata model. Core attributes are standardized for all data types, to permit queries across data types. Each data type may define additional data typespecific attributes. Access to both core and data typespecific attributes are provided through system interfaces; for example, the same interface can be used for searching and/or access. New data types can be dynamically added to the system. Data types can have unique services, such as subsetting and transformation. Data type versioning supports management of evolutionary changes to data. Data type updates allow new attributes and attribute values to be added on-the-fly. Access control is based on data quality.
• Key data ingest features: ECS needs to supports concurrent ingest from multiple data sources, including external processing systems. It must accommodate diverse interface requirements and allow tailoring of data checking, conversion, and preprocessing. ECS must perform data source authentication, data transmission checking, and metadata validation.
• Key data processing features: ECS platforms and infrastructure support the integration and execution of standards-based software algorithms developed by Earth science teams. Automated scheduling of algorithm executions are based on either data arrival or user requests. Multiple algorithms may be chained. Production rules determine which data is used for input. Example rules include spatial/temporal selection, optional inputs, and alternative inputs. Product lineage information is collected, archived, and available to users. Production by external systems is also supported. Standard interfaces are provided, for example, for distributing input data to external systems or other ECS sites.
• Key user interface features: The primary user interface is a Web-based client, EOS Data Gateway (EDG). This permits searches against common and data typespecific metadata attributes; provides integrated, online access to browse data prior to ordering a full product; provides interfaces to support external search and order systems; and supports data navigationbased interfaces, that is, "select and get," as well as search and order interfaces.

[etc.]

ECS Software Architecture Documentation