Like Installation, Only Worse

Chapter 11, I discussed some of the fears users may experience when installing your software. Upgrading a software system takes these fears and adds new ones to them.

Upgrade Fears

Here are some of the upgrade fears I've had to deal with during my career.

Pain of Rework

Many times an upgrade requires the user , the system administrator, or IT personnel to rework one or more aspects of the system and its underlying implementation. This can take many forms, but usually centers on how your system is integrated with other systems. For example, I have created systems that relied on data produced by various government agencies. In an astonishing case of poor customer management, these agencies every now and then would simply change the format of the data being distributed to customers such as myself , breaking the programs we had written to reprocess it. When this happened we had to scramble, rapidly rewriting a variety of software to maintain business continuity. Clearly the agency knew of these changes before they were instituted. The pain and cost to my company, and to other companies that relied on these data, could have been minimized or avoided had the agency simply considered our needs beforehand. At the very least, they could have warned us of the changes!

Ripple Upgrades

A ripple upgrade is one that forces you to change otherwise stable system components . The impact can range from changing hardware (e.g., more memory, more or faster processors, more disk space or different peripherals) to changing software (e.g., new operating system, different version of a key DLL). They are caused when you upgrade your software and new features mandate these changes or when a key vendor forces you to upgrade a key in-licensed technology (an inverse ripple).

Ripple upgrades are a painful part of technology reality. If you're going to in-license technologyand you willthen you're basing part of your future offerings on one or more in-license components. In many circumstances there is simply no alternative to a ripple upgrade.

What you can do is make the ripple upgrade as painless as possible. Clearly identify all dependencies associated with an upgrade. Use ripple upgrades to simplify your matrix of pain by discontinuing support for one or more configurations.

Data Migration

Data created in version n of the system often requires some kind of transformation to be fully usable in version n ₊₁ . New features typically require new schemas. The upgrade process must be constructed in such a way that the user can move data relatively easily from the old schema to the new one. Remember that data migration may go in the other direction. Users of version n ₊₁ may have to create data that can be used by users of version n. The tools for doing this vary considerably based on the application and your choice for a persistent storage. In shrink-wrapped software for personal computers, data is primarily stored in files. In this case you will have to provide facilities for converting files between formats. You should also clearly define the features lost when moving from version n ₊₁ of the system to version n.

In enterprise class software, the bulk of the data is stored in relational databases, which usually means you will have to provide special tools for migrating data between versions. In an upgrade, this data is usually converted in one operation. In a conversion, this data may be converted in one operation or on demand as various features in the system are exercised.

My friend Ron Lunde points out that careful schema design can dramatically reduce the effort of migrating data between releases. The goal is to separate those aspects of the schema that rarely change, or that shouldn't change at all once created, from those that may change frequently. For example, in many transaction-based applications it is rare to upgrade the transaction data, so carefully separating transaction data from nontransaction data in the schema design can substantially reduce data migration efforts.

Data Retention

Old data is rarely deleted. It is retained according to some corporate policy. Reasons for such policies include the fulfillment of specific legal requirements, such as tax laws. Your customers may require you to produce valid archive copies of the data, and you may need to verify that you can access these copies for anywhere from three to seven years after the upgrade.

Certification

Upgrades, especially of enterprise-class software systems, must pass through stringent internally defined customer certifications before they can be put into production. This process usually takes at least one to two months and often considerably longer, which is why it is so rare to see enterprise-class software upgraded more than once or twice a year.

New APIs

New APIs are a special form of rework that must be carefully managed to customer needs. Changing them introduces a variety of pain, usually resulting in the upgrade being delayed. For everyone concerned , it is best to avoid this pain if at all possible. Refer to Chapter 8 for a discussion of API management.

Once Integrated, Never Upgraded

It was a tough decision, but we knew that the changes we made to our underlying server were going to invalidate earlier APIs. In the long run, these changes, and the new APIs, would provide our customers with much more flexibility and functionality. And, since the APIs were rarely used, most customers were not going to be affected. Unfortunately, our analysis indicated that the proposed changes would negatively affect a small number of our largest and most important customers, who had done a lot of custom integration work with the old APIs. The development team found a way to create a partial compatibility layer between the previous API version and the new one, easing, but not erasing, the burden of migrating their systems. Ultimately, for customers upgrading to this new version using these APIs substantial changes would be unavoidable.

Most of our customers realized the benefit of the new server and the new APIs, and decided to upgrade. Some, however, held out for a surprisingly long time. One in particular had spent almost $40K in consulting fees to integrate our system with another one, and they didn't want to spend this money again. I knew that they were serious when they spent $20K to fix a bug in the older, unsupported version. Eventually, they did adopt the new system, not by upgrading but by converting to managed services.

New Features

While customers may be excited to learn that your latest release has several new features, they may not be so excited to learn how to use them. Learning takes time and effort. It requires changing behaviors. Depending on the size of your customer and the magnitude of the upgrade, the cost of learning new features can motivate customers to delay or even cancel the upgrade.

Inaccessible System

Unless an application is truly mission critical, the upgrade process is likely to make it unavailable to users. This can range from a minor inconvenience, as when I'm upgrading a desktop application, to a major inconvenience, such as when I'm upgrading my operating system, to a completely unacceptable risk, such as when my business is upgrading its customer relationship management (CRM) or employee benefits system.

Reversion

Change management protocols for important or mission-critical applications always define how to revert to the previous version of the system should anything go wrong in an upgrade. Your architecture can help make reverting relatively easy or very dangerous.

Often, many complicated steps must be followed exactly or the overall process will fail. The order in which these steps are performed, and their management, represents a special kind of customer pain. Understanding the upgrade process by mapping out each step gives you a chance to simplify it.

My Pain Is Different

The problem with the list of upgrade fears presented is that it is relatively abstract, covering situations that more than likely don't correlate with your specific circumstances. I don't like this because alleviating upgrade pain depends on managing the specific kinds of pain associated with upgrading your system.

Therefore, if this chapter is to have any real value, you must create your own list of upgrade pain. Each of the companies I've worked for, and each of my consulting clients , has one. (Not surprisingly, each of them also had a unique architecture!) Until you've created this list, you can't critically examine what you're doing and how you can improve it to make certain you're meeting customer needs.