Preparing Supporting Environments

Lead Advocacy Group: Release/Operations

With design details being mapped out and plans and schedules being drafted, a team needs to finalize the necessary supporting environments (e.g., development, test, training, staging, training, research). As part of this, they need to identify what is needed, submit requests to acquire the necessary resources (e.g., servers, sample data, data center rack space, networks, power, and air conditioning), and build and administer the environments before the start of a Build Track.

As part of preparing these environments, configuration management and change control need to be established. Often, many different teams use and contribute to these environments, so tight controls are essential. In addition, these environments need to be synchronized (e.g., a network device change needs to be applied to all environments). Other preparation activities include these:

• Develop solution configurations

• Build deployment scripts and processes

• Employ automated deployment tools and deployment checklists

• Develop troubleshooting and problem-resolution guidelines

• Develop backup and recovery or fallback procedures, as specified in planning documents

These environments need to be as similar to production as necessary. This typically means that the environment most removed from production (e.g., development) is the least similar, whereas preproduction environments (e.g., staging) need to be the most like production.

Lesson Learned

After a seemingly successful test session, a team deployed a solution to production, where it summarily failed. After triaging the issues, the team found that test users had administrator rights so they could perform various activities beyond their defined role, whereas in the production system, those users had restricted rights, resulting in their inability to use functionality in production.

Lesson Learned

A team thought it planned sufficiently for getting lab hardware in place. They planned for the usual ordering of the hardware and having it "racked and stacked." What they failed to plan for properly was that the facilities electrician needed much more advanced notice than planned. The team mistakenly placed a "regular" work order instead of a "special" 4-week lead-time work order to run power to a rack. The team still does not understand the difference, but this misstep cost the project a few extra weeks.

The following briefly discusses the potential environments needed to deliver a solution.

Development Environment

There needs to be an environment in which builders of a solution are able to construct a solution incrementally. This includes all aspects of a solution. For example, it is where developers push their code to compile, where the trainers assemble their training collateral and potentially their online help material, where deployment engineers build their deployment solutions, where the infrastructure builders build and configure their servers, and so forth. Team members should first validate that their components work in their private environment and push to development once they have incrementally completed their components for integration to the other solution components.

This environment often has a representative set of servers and services found in production. However, those services are often combined onto the same server to save costs. For example, it is good practice to separate Web gateway and authentication servers. Because a development environment is not usually exposed to the outside, combining these services onto one server, each on its own virtual server image, is commonly done.

With this environment being the initial environment to integrate components and validate features, its stability is inherently unpredictable. As such, this environment is inhospitable for user demos and feature reviews. It also needs to be administered so that should the environment be "polluted," it is easy to rebuild and restore.

Test Environment

A test environment is a more controlled and more production-like environment. It is used to test and evaluate completed aspects of a solution as the solution is promoted from a development environment.

Like a development environment, a test environment needs an easy way to be restored. It will have test data, test scripts, and test user accounts that need to be restored. Successfully restoring the test in a timely fashion involves having a good Configuration Management Plan.

Ideally, the promotion from the development environment to the test environment should be accomplished using deployment automation tools. Although it might seem like overkill for a simple deployment, the benefits are that it gives a deployment team means to refine their deliverables (i.e., deployment process and scripts), and it helps to minimize human error in configuring a solution.

Staging Environment

A staging environment is a limited mirror of the production environment. It is limited in that it replicates a representative set of those services offered in production and is configured and managed like production (e.g., hardware, configurations, user accounts).

Another useful benefit of having a staging environment is that it can be used as a configuration management test bed (e.g., test and evaluate patches). Because staging should be a stable environment, it should be used for demos and pilot evaluations. Sometimes, organizations use staging as a disaster recovery site.

Training Environment

Given that a staging environment is part of stabilizing a solution, it likely needs to be refreshed frequently during the later part of a delivery life cycle. Therefore, dual use of a staging environment for training is not conducive. As such, if online training is part of an operational readiness strategy, it might be necessary to have a training environment.

This environment also needs an easy means to refresh servers and client machines to reset them to defined training scenarios, user accounts, and training data.

Lesson Learned

When sizing a training environment, keep in mind that the number of concurrent users might be more than the number in production. For instance, a team builds a Web-based solution sized to support 50 concurrent user sessions. If training requirements necessitate training classes of 100 users, the solution likely is not able to handle the increased load because 100 users are hitting the same part of the solution at the same time.

Research Environment

Organizations that frequently test and evaluate new technology often find it necessary to have a research environment. Because this is a test bed, it should be isolated from production. Often, research environments are used to develop and demonstrate portions of next-generation solutions.