Planning and Managing Data Obsolescence

Organizations that own or use large databases and data warehouses face a major expense in the next three to 10 years as historical transactional data grow stale and require retirement away from production systems. Given the speed of creation for new information ” despite remarkable new storage technologies that seems to create endless amounts of capacity ” at some point in time it makes sense to remove information that is five, 10 or 20 years old, as its value is low.

From a security perspective, information that is deleted or removed from a networked system reduces the possibility of unauthorized access from that one system, but it creates new opportunities for theft, loss and mischief in other ways. Creating a formal process with multiple points of confirmation of security including an audit trail are critical elements of ensuring information assurance.

For example, information held by a bank on accounts closed 10 years ago probably can be completely removed from a production system and stored off-site in a secure location. Should there ever be a need for it ” perhaps in a tax investigation or lawsuit ” it could probably be retrieved and restored. But perhaps not.

If the master index for the file is lost or corrupted, the bank may not know where to look for the file. Or, assuming the index is usable, the tape, CDROM, or disk containing the file may have been removed from the storage facility. Even worse , the file could have been copied by unauthorized users and then returned to the storage facility without notice. The risk of loss has only shifted from a production system to the archive system.

Also, what about different physical or file formats that occur as technological advances quickly occur? In 1980, a high-density tape stored 6250 bits per inch. Today s tape backup devices store 100 times that density, and in a few years will exceed 1,000 times. Can the data stored at 6250 be restored to a new format? Yes, but at significant expense to the organization requesting the conversion. Obsolete hardware is often exchanged for new systems that are incompatible with very old formats and media types. Having an outsource company convert a few tapes or files may be possible, but what if you had 50,000 tapes or CDROMs? What if you did not know which one had the exact data you were looking for? For a very large company with multiple terabytes of retired information, the cost could exceed many hundreds of thousands of dollars to continually update storage indexes, upgrade tapes, CDROMs, and USB memory devices to ensure compatibility with modern equipment and formats.

Here are several concepts to keep in mind concerning the security of obsolete information:

1. Establish information usage lifetimes and periodically remove information not needed today

2. Use a logical storage index system for all files so that the location of a file or group of files can be easily determined, even if the master index is lost or corrected

3. Ensure that different staff members perform various parts of the duplication, indexing, storage and verification so that each is checking the other s work

4. Perform verification audits on the obsolescence process by periodically checking the ability to restore retired data onto new equipment and formats

5. Ensure the security of the off-site security locations ” use at least two to ensure information survival probability ” through third-party audits and confidence tests

6. When upgrading computer hardware (tape and disk storage), try to obtain the maximum backward compatibility guarantees possible to reduce the cost and effort required to upgrade retired information assets