Data Classification

Mark Bregman, CTO of Veritas, observes that Inescapable Data devices are causing a rethinking of data classification. "We're going to need to categorize data in a new way. In the past, we tended to think of data as 'valuable' or 'not valuable.' With pervasive devices, transient data needs to be managed accordingly and managed in real time, but for shorter life cycles. Quite possibly, this leads to new types of data management software (for transient data) in this seemingly green field," explains Bregman.

It is simpler in the traditional data management world. There are only two types of data: files (considered "unstructured data") and records in a database (considered "structured data"). Inescapable Data devices, such as RFID readers and digital cameras, generate massive volumes of transient datadata that has value for only a short period of time. Much of this transient data (or some distillation) will need to be stored for a period of time. The question is, for how long?

Companies that use, store, and manage data are now on the horns of a dilemma. To save or not to savethat is the question. For example, in a food-processing setting, it will be useful to have a record of a particular pallet crossing a particular loading dock at a certain time, perhaps because later the pallet is identified as one that inadvertently contains food contaminants. However, does the processing plant need to save every RFID blip for that pallet? Or would something more condensed suffice?

Storing digital video poses similar challenges. Would an automobile manufacturer need to save every image taken for a month of the robotic chassis welder? Or would it be more meaningful to store only the images of rejected chassis for exception analysis? On the other hand, if a year later when a particular vehicle is involved in a fatal accident and the investigators suspect a poor chassis weld made the car unstable at high speed, would the manufacturer be able to use the video images of a properly welded chassis to defend itself?

Inescapable Data and its vision for extracting greater value bring with it an added requirement to classify data properly. Data that has become less valuable over time for one business might be very valuable to anotherperhaps another member of a particular value chain. With proper data classification over time and collaborative data-sharing agreements, significant value could be realized.

The data storage industry is entering an era when more classifications of data may be required than the traditional (and simple) file and database. These new classifications will carry with them their own unique data protection, availability, and management requirements. As Bergman points out, "The enabler for In escapable Data and computing is not the endpoint devices creating the data but rather the new critical infrastructure supporting the flow and retention of that data."