From Blocks to Files to Objects


Today, when data bits are stored electronically within a computerized storage device, the data bits are collected into "blocks" (typically 512 bytes). When an operating system issues a command to transfer data to or from a disk, it typically reads or writes the data in a series of blocks. Each block carries with it a block number that identifies it. Higher software functions contained within a file system (on the host or attached computer) logically string many blocks together (that are likely scattered across a disk) into a unit typically known as a file. However, today's disk drives only have knowledge of the lower-level blocks and their associated unique identifiers. They have no knowledge of filesa knowledge reserved for file systems that run somewhere else, usually in the application host processor.

Many guiding lights within the storage industry now believe that the more knowledge a storage device has about the data contained within it, the more optimally it can operate. Today, a disk drive has no knowledge that block 482 is related to block 1,988 and that both blocks contain data used by a critical application (a banking application that manages your checking account, for example). It therefore cannot know whether those blocks need a greater level of data protection than a block that has some temporary data in it that will simply be discarded in a short period of time. The software that understands that blocks are sequenced to form files, and that some files contain data that is more critical to the ongoing function of a business (for example), is higher up the processing chain, and is likely residing on the host computer. This was acceptable when disk drives were typically "inside" of a computer.

Today's medium- to large-scale corporate data centers use networked storage architecturesstorage devices, such as disk arrays, that are physically outside of host application processors and are networked with other processors so that their resources and data can be shared. Various business requirements drove IT administrators in this direction during the late 1990s, primarily because large disk "arrays" were relatively expensive to own and manage when they were held captive by single application servers (because of sophisticated RAID technology and electrical power and cooling requirements). Networking them among other application servers allowed their costs to be amortized across several connected computers. The storage area network (SAN) industry was born.

SANs allow many computers to share storage resources via a high-performance fiber-optic fabric (Fibre Channel). Storage elements can be more easily added to and removed from the fabric as needed, and administrators in charge of the host systems benefit by leveraging the infrastructure costs across many systems. However, in spite of this advance in resource sharing and storage efficiency, the standard block access methods did not change. The servers still use "block" concepts when talking to storage arrays over a SAN even though the storage arrays can now be hundreds of feet or even miles away.

Think back to Chapter 8, "Real-Time Manufacturing," in which you learned that massive increases in business efficiency and values come when barriers are torn down and interface points become more abstract. (For example, do not ask what the inventory level of thumb tacks is; instead, ask whether the ability to complete bulletin board packages is in jeopardy due to supply-chain fulfillment issues.) Most vendors of storage-related products are still not delivering solutions that enable the administrators of storage "data centers" to radically optimize their operations. Not having learned how to "see" (per the Muda concept), these vendors focus myopically on those 10 percent improvement targets. They deliver faster SANs, better SAN switches, and more SAN software to manage and configure and monitor. All Muda. What storage administrators really want is better application program value from storage.

Although SANs were (and are) useful for amortizing the cost of storage physical operations, because they did not fundamentally change the "unit of access," they did not introduce any major process benefit. If SAN elements (storage, switches, etc.) knew of the higher-level business object in transit, the files, more optimization and management automation could take place. If you are a traffic officer in a busy intersection downtown, you can myopically manage the flow of vehicles (disk blocks); if you are the city mayor, however, you can synchronize sporting events with bus schedules and office hours against public-transport availability. Keeping data access at the block level does not allow the infrastructure to capture enough of the value chain and falls short in providing higher optimizations. It also introduces more rigidity, and the SAN manufacturers only response is to focus on the pipes and blocks and attempt to squeeze value by increasing speeds or connections instead of working up to higher values.

Some storage vendors currently offer local area network (LAN)-based file access to network-attached storage (NAS) devices. NAS devices typically do not expose blocks to the host application, but rather expose a file-level interface. These vendors have observed (correctly to users of NAS devices) that ultimately the business object of interest is a file. Using NAS, storage administrators can more easily manage access to files that have common requirements for quality of service, performance, and protection. For them, NAS is clearly the right direction. The NAS device can behind the scenes operate on "blocks that comprise a file" as a set (perhaps to replicate, to index, alter the location for performance, etc.). Some NAS manufacturers have been extremely successful with a model that is still resisted by others because of performance concerns. Why? The typical interface technology for NAS devices is TCP/IP on Etherneta somewhat slower and higher-overhead protocol than the Fibre Channel protocol used for raw block access. However, the real reason in their minds is that NAS essentially requires a storage transaction to be processed twiceonce to pass through the "computer" in the NAS itself and once more when the actual data moves through the host computer for the application. Corporate data-center storage administrators who grew up in the mainframe world spent decades through the 1970s, 1980s, and 1990s looking at removing all possible I/O bottlenecksstorage being uppermost on the list. They instinctively reject any access method that is not the fastest one possible. However, for many, NAS performance is good enough, and the cost of NAS can be one-half to one-third the cost of SAN.

Hybrid Approaches

Some interesting hybrid solutions exist to outboard data storage. SANergy is one such product that allows computers to view storage as if it is NAS, but the actual payload contents of files is moved directly as blocks via a faster alternative path like that of a fibre-channel SAN. Other products imitating this approach are IBM's SAN file system and ADIC's StorNext file system. But the performance improvements gained by these solutions are only stop-gap measures, and the modest deployment of such techniques by data-center managers suggests that the need for faster "network file systems" is less acute than currently perceived by storage administrators; that is, speed is not the real issue. Network-attached storage changed the data management paradigm in a way that many administrators were not yet willing to embrace; for years, their hosts accessed raw blocks, and changing to a file mentality was as difficult as it was for some to switch from being phone-centric to e-mail-centric.


Business processes and operations on data need to be abstracted better, and then they can be optimized in the storage infrastructure. The reason that NAS and SAN are only incremental improvements in the ways needed to manage large accumulations of data, such as the ones generated by Inescapable Data devices, can be seen when one examines the problem storage administrators are faced with when responding to new regulations governing data-retention periods. The rules for retention (HIPPA, Sarbanes-Oxley, SEC 17-a4, etc.) can be somewhat more complicated than just "retain for seven years." It might be this: Retain for seven years once the account is closed, or after broker X leaves, or retain indefinitely while the audit is underway. Different types of material carry different retention periods and rules. (An e-mail might be required to be retained for one period, and a proposal might have a different requirement.) To manage data more effectively, data-center storage managers need to assign more management criteria to individual filesadditional information about files (metadata) beyond name and date and owner that are now needed to properly protect and manage files.

Inescapable Data generates big datadata in the petabyte range. To manage and process it in such a way that yields meaningful information in real or near-real time, a new level of abstraction has to be introducedhigher than file level. This new abstraction layer could take the form of a "super file" or some sort of macro file that is built out of XML components (but clearly something beyond the current approach that keeps a parallel database for the extra metadata that is very limited in its scope to begin with). We think that a fundamental problem worth solving is essentially one of scalability. Data blocks are too small to be stored and accessed individually, so they are aggregated into files and file systems for the sake of processing efficiency as more and more data block accumulate. Similarly, files have become too small of a denominator as well. A super-file concept will allow many components in storage fabrics and storage arrays to self-optimize and manage the data better. Essentially, this drives data management intelligence deeper into the storage domain and away from the application host environment. In the Inescapable Data world, optimizations occur not just by connectivity alone but by virtue of information that allows for intelligent optimizations.

Although the super-file concept is not yet evidenced in today's market offerings, something called Object Store is close and getting some market traction. The Storage Network Industry Association (SNIA) has a technical working group focused on object-based storage, http://www.snia.org/app/org/workgroup/osd/description.php. The base concept rejects the notion of accessing physical storage "block by block" whose meaning is only understood by a host file system. Instead, Object Store uses an object number to reference a file (object), and then uses commands to read/write pieces of that object. This process is roughly similar to the way a network computer accesses a network file system; the difference is that processing of the new Object Store protocol is performed directly on the disks (or storage arrays) themselves, hopefully yielding a quantum leap in the performance and scalability of storage systems in service to the demands of big data. The goal of Object Store is to truly replace "block numbers" with "object ID object data."

A number of commercial activities are taking place aimed at popularizing OSDs (object-based storage devices). Many companies are now participating in establishing a standard version of this model so that, once implemented, all vendors conform to basically the same way of doing OSD. At the forefront are IBM, EMC, Veritas, Seagate, and HP, among others. In addition, there is an open-source file system, Lustre, developed for the National Laboratories (Livermore, Los Alamos, Sandia) by Cluster File Systems Inc., and marketed by HP as the StorageWorks scalable file system. Lustre uses an implementation of an object store called ObjectStore Target (OST). By driving an object-oriented access method directly into storage devices, hosts systems can more readily "share" access to data at storage-wire speeds (without funneling data through an intermediary controlling file server as is the case with NAS) and the storage element can more intelligent lay out files and optimize access. Although not as powerful as a true super file, this is evidence of the storage industry at least embracing the notion that more powerful forms of metadata and metadata processing will be needed to make high-value use of the masses of Inescapable Data that will beckon those wanting to extract that value.



    Inescapable Data. Harnessing the Power of Convergence
    Inescapable Data: Harnessing the Power of Convergence (paperback)
    ISBN: 0137026730
    EAN: 2147483647
    Year: 2005
    Pages: 159

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net