7.5 Information Storage Infrastructure

Purpose

An information storage infrastructure lets us store information persistently-that is, it's still there when we switch the machine off-and retrieve it later. Information infrastructures vary significantly in their characteristics, and for this reason, a system might use more than one of them, and sometimes more than one of the same type (such as a heavyweight server database plus a lightweight database on client PCs).

The most prevalent kind of storage mechanism for commercial systems is a database. Other kinds are a file system for reading and writing flat files (though there's usually-but not always-little point in saying anything about this as an infrastructure, because it's built in to every operating system and programming language, so it's there anyway); a transaction monitor (if we need to co-ordinate changes made to more than one database); and a document management system (if our system accesses the contents of "documents"). You can also regard information retrieval mechanisms as information infrastructures that make use of information stored elsewhere; these include a directory service (such as one adhering to LDAP) and a search engine (to provide powerful means of finding things). This is a very diverse range of types of products, and we can expect requirements for them to vary accordingly.

Invocation Requirements

What will a calling system expect to be able to ask this information infrastructure to do for it? What kinds of requests will it make? If we're asking for a database, a system will invoke it using standard database operations (of which the most important are the so-called CRUD ones: create, read, update and delete). It's reasonable to assume that any database product will supply these operations, so there's little to be gained by spelling them out. Similarly, we can expect any file system to let us read and write the raw content of any flat file and to read the file system to see what files are present.

We might want the ability to easily replace one product with another. In this case, it might be helpful to go into a bit more detail about how we talk to it-perhaps by mentioning a standard (such as SQL or ODBC, for a database), which will, in theory, let us use any product that adheres to that standard.

Implementation Requirements

One way to look at implementation requirements is to see them as specifying the criteria by which to choose the most suitable information storage (or retrieval) product.

Some behind-the-scenes capabilities that we might want an information storage infrastructure to have are the ability to create back-ups of data and to recover data in the event of a failure (such as losing a disk drive). We might also want to ask for all changes to be atomic (all or nothing), to maintain data in a consistent state, and for each change to be isolated from other changes being made at the same time. (These are three of the ACID properties: atomic, consistent, isolated, and durable-the last of which is implicit in an information storage infrastructure.) Other considerations that might be important include capacity (especially if you need to be able to store huge quantities of data) and security features (such as being able to encrypt data, and access control).

Requirements for any kind of storage infrastructure other than databases must take into account its special characteristics.