TRADE-OFF STUDIES | Six Sigma and Beyond: Design for Six Sigma, Volume VI

Trade-off studies are designed for balancing both business and technical issues and optimizing the product for the customer, whether internal or external to the organization. Trade-off studies:

Are a structured, analytical method for objectively identifying, defining, and evaluating alternatives
Are designed for analytically presenting, evaluating, and weighting decision information based upon program targets, objectives, goals, and technical requirements
Ensure that the selected alternative is the best at meeting the program objectives, goals and technical requirements

We conduct them to:

Promote an objective evaluation and minimize subjective selection
Force requirements to drive the evaluation of the alternative
Ensure that sufficient information for making a decision is provided
Demonstrate that the alternatives satisfy the requirements (as they are understood at the time of the evaluation)
Document the evaluation

Of course, the question quite often is, "How do I know if I need to conduct a trade-off study?" The answer depends on the following questions:

Does the decision require input and concurrence from several organizations?
Does the decision require balancing inputs that may conflict and/or are inversely related ?
Is there a choice between several viable /acceptable alternatives?
Is a quick, comprehensive, and defensible decision needed?

If the answer is yes to one or more of these questions, a trade-off study may be the best approach for selecting the optimum solution. To conduct the trade-off study appropriately, there are some preliminary requirements. They are:

Prior to declaring or encountering a critical design freeze
When balancing major systems or their components ' functional performance
When considering several design alternatives at any level (e.g., systems, subsystems, and components and so on).
When conflict exists among targets, objectives, and requirements (e.g., maximizing one has negative effects on the others)
When establishing dominant attributes or prioritizing customer requirements

As with any other methodology and tool, with a trade-off study we expect to have some kind of deliverables at the end of the analysis. Typical deliverables are:

An alternative selected largely on the basis of fact, which is acceptable to and defensible by all the stakeholders
Complete documentation (Evidence Book) outlining how and why the decision was made
A risk list identifying areas of concern for all the alternatives investigated
Sensitivity analysis showing the stability of the selected alternative

HOW TO CONDUCT A TRADE-OFF STUDY: THE PROCESS

Several steps are required when conducting a complete trade-off study. Each step aids in ensuring that the end decision best meets the stated customer requirements. Each step is discussed in detail below. A checklist is also provided to assist you in conducting your trade-off study. The steps are:

1. Construct the Preliminary Matrix

The trade-off study matrix consists of two major components ” the alternative list and the category list:

The alternative list: This list is simply a listing of each alternative being considered . The alternatives are listed across the top of the trade-off study matrix, with one alternative per column.
The category list: The category list consists of musts and wants arranged by assessment items. Each assessment item is broken down into various measurable discriminators.

An example:

Attribute category: safety
Assessment item: frontal impact
Discriminator: 5 mph bumpers; bumper material; crumple space

The first step in the trade-off study process is creating a preliminary matrix. You must identify both the alternatives being examined and the list of assessment items and discriminators. Draw assessment items and discriminators from program requirements, corporate data, QFD studies, CAE analysis, etc. The preliminary matrix acts as a discussion catalyst at the first team meeting. After assembling the assessment list, sort it into those that are imperatives (or musts) and those that are desirables (or wants).

2. Select and Assemble the Cross-Functional Team

The goal of this step is to ensure that affected parties are adequately represented. It is better for a group to decline participation than to be overlooked in the team assembly process. The team is composed of representatives from each group impacted by the decision being made (it must be cross-functional and multidisciplinary ). Team size varies depending on the subject and scope of the project (initial meetings should include no more than nine to twelve people). Team membership is based on contribution potential not approval needs. Approval takes place during the presentation of results at the end of the process.

3. Assign Team Members ' Roles and Responsibilities

Although all team members will play a role in the trade-off study process, three key positions must be filled to ensure process success, as follows :

Team champion

Is usually a program manager, project manager, or someone empowered by those individuals to carry out the selection of an alternative
Is the individual who must design, build, or approve the selected alternative
Supports and participates in the process, and accepts (and backs) the team's consensus decision
Provides the resources to accomplish the task at hand

Lead facilitator

Guards against duplicating efforts, provides information to individuals between meetings, generally coordinates the entire process
Is an empowered member of the team whose role is to coordinate the work of the ranking process
Resolves any overlaps in evaluation that occur during the ranking process
Coordinates the open issues identified by the team
Compiles the ranking process data/results into the evidence book

Process coordinator

Ensures that all affected parties are represented and that each team member understands the process
Is responsible for aiding the team leader in assembling the team
Schedules and runs all core team meetings
Provides and explains the trade-off study methodology and supporting tools

4. Assign Ranking Teams to Evaluate the Alternatives

Ranking teams are designed to evaluate each alternative within a particular category or assessment item. Individuals are assigned to these teams based on their particular specialty. For example, a transmission design engineer would be assigned to the task of assessing (or ranking) an alternative's ability to handle a particular level of input torque; an individual from marketing would determine an alternative's potential volumes ; a financial expert may be assigned to develop marginal costs, and so on.

After the core team modifies and approves the preliminary matrix, determine the personnel necessary for conducting the ranking within each category. Assign team members to ranking teams based on their specialty or to the category that affects their area/product. Due to the critical aspect of the ranking teams, team members require specific direction on:

Ranking/evaluation methodologies
Documentation format and content
Reporting their findings, conclusions, and issues

Identification of Ranking Methods

The ranking team's primary function is ranking each alternative's ability to achieve the discriminator (for example, the alternative providing the best crash test gives the highest rank). Measurable discriminators are a must, and the ranking team must devise a method for determining each alternative's ability to meet that discriminator (for example, is mileage, bumper material, or crumple space the discriminator with the best measurability and effectiveness for our expected result). This ranking is accomplished as follows:

As each ranking team delves into the ranking process, the team members may expand or contract the discriminator list to better evaluate the alternative's performance for a given assessment item.
Each team selects the ranking method it feels is appropriate and is defensible to the core team.
Whenever possible, the rankings should be based on actual test results, CAE analysis, or numerical analysis (i.e., facts or directly observable data).
Expert opinion and subjective rankings should be used as a last resort when time, cost, knowledge, or value limit the reliance on tests, rigorous analysis, or computer simulation.
An alternative that fails in achieving any assessment item defined as a project must is eliminated from the evaluation. The ranking team informs the lead facilitator who stops further evaluation of that alternative by all other ranking teams (this is done to limit resource expenditure).
When ranking the alternatives, the one (or more) alternatives best satisfying a particular discriminator get the highest numerical rank for that discriminator. This is usually done by counting the number of alternatives and using that value as the highest rank (i.e. with four alternatives, the best would receive a rank of four, the next a three, and so on). Alternatives do not have to be forced ranked, nor does one have to receive the top score. If all or some of the alternatives have equal ability to satisfy the discriminator, they would receive equal rank. If that ability is high, they would all receive a top rank; if that ability is poor, they all may receive a low rank

Development of Standardized Documentation

A secondary result of a trade-off study is the evidence book documenting the entire decision-making process. Although secondary, this activity needs to be taken very seriously in order to defend the core team's decision to those both inside and outside the group. Documentation being produced by the ranking teams must be consistent in format and content to ensure ease in assembling the evidence book. To ensure this:

Provide each ranking team a standard format for reporting their findings.
Make sure that the format includes, at a minimum:
- The ranking results
- The ranking method
- Advantages and disadvantages of each alternative (as defined by the ranking teams)
- Any risks associated with the selection of each alternative
- Any issues identified during the ranking process
Include all supporting documentation generated during the ranking process in an appendix, attachment, or separately defined section.

Timing for Report Out of Selection Process

Once the members of a ranking team complete the ranking process, they forward their completed documentation to the lead facilitator to place in the evidence book. Each team is then responsible for preparing a presentation of its findings to the entire core team. The presentation includes a summary of the rankings, methods, issues, and risks associated with the selected alternative. This presentation is then made to the entire core team at the final core meeting.

5. Weight the Various Categories

While the ranking teams proceed with the ranking process, the process coordinator and team leader pull together the necessary or key personnel to assign weightings to the various assessment items.

The weighting rule: Assign weightings according to the assessment item's importance or impact on satisfying the customer and company needs/requirements and ensuring the optimum decision (for this point in the program).

With key personnel developing the weightings in parallel to the ranking process, we are able to:

Assign the weightings at a higher corporate level assuring better alignment with corporate vision.
Work more quickly and efficiently toward balancing the weightings.

6. Compile the Evidence Book

Now that the weightings have been assigned, the next step is for the lead facilitator to compile the evidence book. There are several steps to this process:

Organize the ranking teams' documentation in category sequence as it appears on the trade-off study matrix.
Calculate each alternative's score within the assessment item by determining an assessment item average. Simply sum an alternative's rank for the various discriminators and divide by the number of discriminators.
Continue calculating alternatives' scores as additional ranking teams report out.

Once each ranking team has reported out, the lead facilitator develops a summary of the evidence book for distribution during the final presentation. This summary contains:

The completed trade-off study matrix, including tallied alternative scores A section outlining the identified advantages and disadvantages of each alternative Identified risks associated with each alternative

7. Present the Results

When each ranking team has reported its results to the lead facilitator, the process coordinator reassembles the core team for a presentation of the results. Copies of the summary document are distributed to each core team member three to five days prior to the meeting. Each ranking team then presents its findings, methods, and issues to the entire team.

Sensitivity Analysis: The purpose of the sensitivity analysis is to determine the robustness of the selected alternative. The process allows the group to ask various "what if" questions regarding a particular ranking or weighting and receive an immediate answer, such as:

"What if" the ranking of that assessment category were inverted, would the alternative still be chosen ?
"What if" the weighing of that assessment item were lowered , would it change the selection?

It is recommended that a laptop computer, loaded with the trade-off study matrix, be brought to the presentation by the lead facilitator. Modifications can be made to the rankings within an assessment item or weightings on a category to see how that would affect the overall decision. This will identify how sensitive the decision is to certain changes and give the group an immediate feel for the selected alternative's robustness.

Here is a typical trade-off study process checklist:

Constructing the preliminary matrix
- Consideration has been given to all attribute categories.
- All discriminators are measurable, now .
- Assessment items considered "musts" are truly "musts."
- Assessment items considered "wants" are only "wants."
Selection and assembly of the cross-functional team
- All affected activities have been invited to participate.
- All participants are empowered by their management.
Assigning team members' roles and responsibilitie.
- Team champion will design, build, or approve the selection.
- Lead facilitator is in a position to coordinate the ranking teams.
- Process coordinator is willing to schedule and run meetings.
Assigning team members to ranking teams
- Ranking teams have the right specialists to accomplish their task.
- Standardized report format has been established and agreed upon.
- Acceptable ranking methodologies have been agreed upon.
- Freedom in expanding/contracting the discriminator list has been conveyed.
- Timing and report out procedures are understood by each team.
Weightings of the various categories
- Identification of key personnel is complete.
- Weightings are being conducted in parallel to ranking teams' evaluation.
- Assigned weightings align with customer and corporate wants.
Compilation of evidence book
- Ranking team documentation organized to trade-off study matrix
- Alternative scores calculated (weight — rank = score)
- Trade-off study summary completed, for final presentation
Presentation of results
- Entire core team reassembled
- Laptop with trade-off study matrix available for sensitivity analysis
- Consensus decision reached as to which alternative to pursue

GLOSSARY OF TERMS

Alternative rank

Defines how well an alternative compares to other alternatives in achieving a particular assessment item or discriminator.

Assessment item

A particular attribute of a given category.

Categories

Major classes of assessment items

Discriminator

A specific portion of an attribute.

Evidence book

The book containing the complete set of documentation created by the core team, ranking teams, and weighting teams during their evaluation of the alternatives.

Musts and wants

Each assessment item is grouped according to whether it is a must or a want.

Musts are defined as those items that an alternative has to meet in order for it to garner further consideration. When an alternative does not meet a must, it is dropped from the study, unless it can be brought in line with the must or the must is modified.
Wants are those items that are needed to reach maximum customer and corporate satisfaction, but an alternative would not be discarded for failing to meet them. These wants are weighted and determine which alternative gets selected from those that meet all of the musts.

Ranking team

The sub-core group/team that is assembled to evaluate (and rank) the various alternatives within a particular assessment item. This team also identifies risks, advantages, disadvantages, and issues encountered during the alternative evaluation.

Trade-off study matrix

A tabular chart used to list alternatives being evaluated (across the top) and assessment items being used to differentiate the alternatives (down the left-hand side). Alternative rankings, assessment item weightings and alternatives score are also tallied on the matrix. (A matrix can easily be created using any spreadsheet program.)

Weightings

A value given to an assessment item, categorized as a want, to show its relative importance to the other assessment items. Techniques used to develop weightings include pair-wise comparison, 100% weightings (sum of weightings add to 100), and many others.