Part OneThe Challenge


This case study outlines the advantages of a best-practice approach to deciding what to include in an IT portfolio. A well-thought-out process based on fundamental principles developed over the past 100 years can result in the following benefits to traditional seat-of-the-pants or rack-and-stack methods of portfolio composition.

  • Strategic focus An IT portfolio must reflect the organization's mission and objectives. You can do this by aligning projects and resources with the organization's goals and objectives. A "good" portfolio for one organization might be ill conceived for another. An organization's direction, as set by senior management, must set the course for what goes into the portfolio.

  • Better, faster, more competitive decisions Put in another way, this means "win or lose by how you choose." You must address risk and uncertainty. Prioritization is a key element of the "how you choose" portion of this equation, and there are good ways and poor ways (often evident in practice) to prioritize. Priorities based on color schemes (red/yellow/green), adding ranks, and other mathematically meaningless combinations of numbers are sometimes worse than no priorities at all.

  • Improved communication Decision makers at all levels of the organization are involved, applying their knowledge and experience where it is most effective.

  • Convince others you are right It isn't enough to just conceive an IT portfolio that is well aligned with an organization's objectives; you must communicate it in a way that convinces others that this is indeed the case. A best-practice process reduces battles and breaks deadlocks.

  • Build organizational buy-in Although it is rarely possible to please everyone, it is almost always possible to have everyone feel they were part of the decision and can commit their energies to successful implementation.

The Five Phases of an Iterative Process

The best practice that we describe is based on an iterative process consisting of five distinct but related phases: design, structuring complexity, measurement, synthesis, and optimization. We will not discuss the details of each phase here, but rather, will focus on the essential nature of each phase and their relationships.

  1. Design Organizations receive project proposals from numerous sources. There are numerous approaches to designing and articulating projects, including the identification of objective benefits, subjective benefits, risks, and costs. An analysis of what alternatives were considered in the design of each project is often helpful in understanding the proposed benefits.

  2. Structuring complexity Evaluating benefits, costs, scenarios, and risks is an elusive business that you must structure carefully. There are almost always numerous competing factors to consider and just assigning weights in a spreadsheet is a futile exercise. Instead, a best practice entails structuring complexity by organizing these factors into a hierarchy of homogeneous clusters of scenarios, objectives, and subobjectives so that those involved in the decision process do not get lost in a maze of complexity and become hampered by miscommunication.

  3. Measurement In the past, many organizations made decisions based primarily on financial projections. Portfolio decisions should involve both quantitative and qualitative factors. No matter how good the "data" is, there is always a need for keen human judgment, both to interpret the data and to establish priorities. Einstein observed that "Not everything that counts can be counted, and not everything that can be counted, counts." Peter Drucker observed that "We have to measure, not count." Data, no matter how complete and accurate, is not adequate. Analysis (breaking things down into parts and studying the parts) is not enough. Judgment, as well as data and analysis, is required and judgment must be applied in meaningful ways. Portfolio decisions must involve decision makers at all levels of the organization. Judgments cannot come solely from the "top" or "bottom" of the organization. Judgments must be made, aligned, and synthesized from the top of the organization down, or from the technical levels of the organization up. An organization's portfolio should be shaped by the judgments that top management makes. These judgments reflect the direction the organization should take in order to be faithful to its core ideology. The portfolio should also be shaped by judgments of middle management that reflect the relative importance of objectives that will achieve the elements of its core ideology as well as meet competitive pressures, and judgments of middle and technical management about the activities or projects that can be included in the portfolio in order to achieve these objectives. The result is a consolidation of corporate information and judgments that produces a portfolio that is aligned, from top to bottom, with the organization's goals and objectives. Engaging all levels of an organization's knowledge base is easier said than done. Meetings can be endless and can involve considerable bickeringunless judgments are properly applied to "derive" measures that lead to a coherent, optimal portfolio. But you must be careful to ensure that the measures are based on human judgment as well as on data, and that the measures accurately reflect such judgment, are justifiable, and are mathematically sound. The typical weights-and-scores approach, which many organizations use today, is far from adequate. A sound IT portfolio alignment requires the following:

    • Measures that factor in judgment about data (data alone is rarely adequate)

    • Measures that accurately reflect human judgments

    • Measures that synthesize judgments from personnel throughout an organization, from executive judgments about strategic objectives to technical judgments about the contributions of specific projects toward specific objectives

    • Measures that can be justified because they are derived, rather than arbitrarily assigned

    • Measures that can be justified because they are based on a sound mathematical process such as the Analytic Hierarchy Process (AHP)

    • Measures that are proportional, or possess "ratio scale" properties that are necessary in order to use powerful optimization techniques in deciding what the IT portfolio should and should not include

    A best practice methodology must also be capable of accommodating subtleties that can arise. Unless you incorporate subtleties such as rank reversals, ideal alternatives, and structural adjustment into the methodology, the veracity of the measures can be challenged and the credibility of the process diminished.

  4. Synthesis The development of a best-practice IT portfolio requires a meaningful synthesis or "fusion" of data, information, analyses, and judgment. This can be a formidable task, especially when some of the information is subjective and some is objective, when there are conflicting objectives, and when expertise and perspectives are distributed throughout an organization. Beware that there are numerous ways to synthesize poorly and to produce a synthesis that does not meaningfully reflect its constituent parts. Examples include adding or averaging ranks, using simple 1 to 5 scales, "normalizing" data so that it appears to be comparable, and color coding. However, if you are careful with your measurements (as noted in step 3), you can avoid these mistakes and achieve a meaningful synthesis that produces ratio scale priorities reflecting proportional measures for the alignment of projects with objectives. Such a synthesis is necessary in order to determine an "optimal portfolio." You can perform sensitivity analysis to ascertain that the results are not only mathematically meaningful, but also logical and consistent. The need for iteration to accommodate additional information, creativity, redesign, or changes in judgment becomes apparent.

  5. Optimization Optimization is often misunderstood. Optimization is not a search for a preordained "optimal" portfolio, such as searching for a needle in a haystack. Instead, an optimal portfolio is the identification of a subset of proposed projects that, in combination, aligns with an organization's objectives better than any other possible subset of projects. The measure of "better than" is based on the structuring, measurement, and synthesis described earlier. The "optimization" involves determining which subset of proposed projects is "best" and entails an efficient search for the "best" subset from a very large number of possible combinations of proposed projects that adhere to myriad monetary, physical, organizational, and political constraints.

The optimal portfolio is guaranteed to be

  • Better than a portfolio negotiated using what is sometimes referred to as the BOGSAT (Bunch of Old Guys/Gals Sitting Around Talking) approach, in which benefits are not even measured, or are assessed with whatever can be quantified, omitting important qualitative considerations.

  • Better than a portfolio in which benefits are derived based on a structuring of objectives and ratio level measurement, but where the projects are sorted by benefit and funds allocated until the budget is expended.

  • Better than a portfolio in which benefits are derived based on a structuring of objectives and ratio level measurement, but where the projects are sorted by benefit/cost ratios and funds allocated until the budget is expended.

  • An efficient optimization algorithm is required to identify an optimal portfolio because typically, the number of possible combinations is so large that an exhaustive search of all possible combinations would take hundreds or thousands of years for even the fastest known computers. Furthermore, equally important to the efficiency of the optimization algorithm is the ability for managers to understand and control the optimization process so that the resulting "optimal portfolio" is not only mathematically best, but also intuitively appealingadhering to the organization's monetary, physical, organizational, and political constraints.

Objectivity, Subjectivity, and Quality

The word objective has different meanings in different contexts and people are often confused about the difference between an objective, an objective decision, and an objective decision process. An objective, as a noun, is something someone seeks. As an adjective, an objective experiment is one in which the outcome is not subjectivethat is, it doesn't depend on any of the subjects. A scientific experiment that is objective will have the same results (plus or minus some measurement error) regardless of where it is performed and who performs it. Also as a noun, an objective process is one that commonly means systematic and fair.

In making complex decisions (such as deciding what projects should be included in an IT portfolio), we try to align the portfolio projects with the objectives that we seek to achieve. Because there are numerous objectives, we must prioritize them. There will never be any "data" or magic formulas to do this. We must do it on the basis of human (subjects') judgment. Consequently, the decision or portfolio will always be "subjective." Subjectivity in this sense does not say anything about the quality of the decision. The quality of such decisions can range from excellent to poordepending on the quality (knowledge and experience) of the subjects, the way the decision is structured, and the measurement that is used to synthesize knowledge, data, and judgments.

The quality of an optimal IT portfolio involves much more than the accuracy of any data that is used in the process. In particular, an optimization is meaningless (and very possibly misleading) unless the measurements are in "proportion" to what we expect will happen. This "proportion" requires ratio level measurement. Simple 1 to 10 scales, such as those used in the Olympics to evaluate gymnasts, seldom produce ratio level measurements. An Olympic judge that rates a gymnast's performance a perfect 10.0 would consider that performance to be 2, 5, 10, or more times better than a 9.5 performance, not simply the ratio of 10/9.5 = 1.05 times better. Ratio level measurements mean that the measures of benefits are in proportion to the benefits that are expected to occur. It does not mean that the measures are accurate to so many decimal places. The accuracy of the measurements obviously depends on the accuracy of the available data and the quality of the judgments that are applied to the data as well as to the relative importance of the organization's objectives. Although an organization should continually strive to improve this "accuracy," it must, at any given time, proceed with what is then available. It must not fall into the trap of thinking that just because the measurement is not "precise" simplistic schemes producing ordinal or interval measurements are adequate for identifying an optimal portfolio. They are not.




Design for Trustworthy Software. Tools, Techniques, and Methodology of Developing Robust Software
Design for Trustworthy Software: Tools, Techniques, and Methodology of Developing Robust Software
ISBN: 0131872508
EAN: 2147483647
Year: 2006
Pages: 394

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net