Store Everything?

Are we are heading toward a world in which we might want to store absolutely everything for an indefinite period of time? As discussed in Chapter 6, "Connecting Medicine," digitized x-rays and other medical images are already being stored indefinitely. Why? There is tremendous research value in mining that data and correlating it with other forms of data. The value of this data will only be exploited if it is saved. The really difficult question of "for how long?" is only exacerbated by the fact that, in research endeavors, researchers cannot really know whether data that seems to have zero value now might be of tremendous value to another researcher in the future. Two decades worth of digitized x-ray images could prove invaluable to a cancer researcher studying the progress of a particular type of tumor, particularly when that data can be correlated with genomic data, diagnosis histories, and treatment regimens.

Your son's sixth-grade essay, created 10 years ago, on the demise of the U.S. cotton industry may seem valueless to you now. But, at some point in the future, some researcher's tool could analyze his essay against essays from various socio-economic backgrounds from that period and compare it to each year since. There could be an obscure correlation to the sales of video games or to certain uses of language evidenced in the captured history of those essays. At age 35, his career counselor could use his entire electronic history to understand and expose useful patterns, true skills, and hidden aspirations. If the cost to retain this document is insignificant, why not keep it indefinitely? (Are you not right now just a little bit curious what it would be like to be able to revisit your own sixth-grade essays?)

As an investor in a particular company, wouldn't you like to have access to absolutely all electronic records for that company, no matter how many years have passed? The federal government requires companies to retain physical records for a set number of years (and more so now than ever before), but the time limitation is primarily rooted in the practicality of doing so and the attendant costs. Before the advent of digital storage, if businesses had to keep every record forever "we'd have to hollow out Mars just store everything!" exclaims John McDonald (the "Mc" in McData and former CEO of that company). Storing physical paper documents requires warehouses and staffing and heating and cooling for what might be minor benefit to the government. So government rules for retention duration are based on a recognition that the likely utility of an aged document has to be balanced with an affordable cost for physically keeping it available.

Digital storage now replaces much physical storage. With that physical storage cost burden lifted, does that mean that companies will be required to store everything digitally forever? Although it is now conceivable that one could store nearly everything forever, it still becomes difficult, if not impossible, as a practical matter. True, disk storage densities nearly double every year or so, meaning that one can store nearly twice as much data on roughly the same-sized disk with each new generation. At the same time, the cost of each new generation of disk remains roughly the same, meaning that one could take all the data stored on the previous disk, deposit it on the new disk (that costs the same as the old one), and still have room for lots morespace you may feel you get for free. As you will understand, however, there is always a price for storing dataand the longer we save it, the greater the cost. A storage media that lasts longer than paper has yet to be invented. Therefore, to preserve digital data, you have to keep moving the data bits from old media to new every 5 to 10 years for magnetic and 15 to 20 years for optical. Otherwise, it is lost. The cost for those successive migrations adds up over time. In fact, if you store files of family pictures for long periods of time, for example, it is still cheaper (and probably safer at this point if you want to pass them on to successive generations) just to have them printed and put them in a picture album.

MRAM

Magnetoresistive random access memory (MRAM) is a new type of data storage medium that may change the data storage world. Typical computer memory, dynamic random access memory (DRAM), stores data by using electricity. Electricity must be continuously fed to the RAM to keep it charged correctly, and this, of course, is why your laptop battery does not last 100 hours. MRAM uses magnetic charges instead and, as a result, needs virtually no power. Thus, like data on your hard drive, it will persist even when the power is turned off.

The U.S. Defense Advanced Research Projects Agency (DARPA) has been funding initiatives in MRAM since 1995.^[1] A group of three companies (IBM, Motorola, and Honeywell) led the effort initially, and other companies have more recently joined in the fun.

Just another new storage media? Maybe not. How would you like to continuously record everything that happens to you throughout your life (in high fidelity detail no less) just by wearing a small digital recording device? Personal life recorders (PLRs), popularized by Don Norman in his 1992 book, Turn Signals Are the Facial Expression of Automobiles, could use MRAM storage to make the PLR possible given its density, low power, and cost advantages. Although manufacturers fabricating MRAM prototype devices are working out bugs, the expectation is that within 5 to 7 years, MRAMs will hold 400 times more data than today's tiniest hard drives within the same physical space and using farless power. Estimates of the projected revenue for this new type of storage device exceed $4 billion annually.

^[1] http://whatis.techtarget.com/definition/0,289893,sid9gci539346,00.html.

We have almost come to expect that certain types of personal data will be stored and accessible for long periods of time. We might expect every banking deposit or withdrawal to be online and available to us going back to account creation. Certainly, we would like to have childhood orthodontic records when faced with oral surgery later in life. If you have a child getting braces now, it could be quite useful for you to compare your records to your child's, given the genetic relationship. In the future, we may find it distressing if we cannot make such comparisons. Accurate family history of anything medical is now extremely important. Even if your childhood orthodontist is no longer practicing, he probably had no reason or physical space to keep your records created 40 years ago, yet there is utility in those records to you.

Perhaps you would you find it useful to have electronic records of your grammar school grades and course work as your own children make their way through the school system. Perhaps you have forgotten that you, too, had troubles with abstract concepts up through the fifth grade, but then some change occurred and you turned out perfectly well. Does this compare to what a child of yours is going through? Your own history contains valuable databoth to you and to others. Increasingly, our personal digital histories are being recorded by companies and organizations with which we come into contact. Who owns that data? Would you like to own your data? Such personal records now exist and could be collected in a more organized manner.

The Inescapable Data world researchers of all kinds thrive on massive volumes of accumulated data. Through collection and analysis of this data, extremely useful knowledge and information will result. It might be patterns of espionage, or indicators of fraud, or confirmation of assets, or medical or personal social histories.

We have always used data to help us project forward or discover the past. The difference in the Inescapable Data world is that we will use staggering amounts of historical data and correlate it with various streams (sometimes unexpected or oven odd streams) to learn of hidden values. As Jack McDonnell puts it, "The ultimate goal is not reacting, but anticipating." The first step may be to store nearly everything nearly indefinitely.