| 
Roles 
Practices 
Core Practices | Evo applies to systems engineering in general not only software development although software projects are a common domain of application. |
As with the other IID methods covered, Evo promotes evolutionary requirements analysis. Yet, when requirements and design ideas are written, Evo requires analysis with respect to a measurable evaluation of the value and impact of requirements and designs. Evo is infused with the practice and value of measurable, measuring, and adaptive response to the results. Practice | Description |
|---|
Find stakeholders | Both internal and external, friendly and foe, and across the lifecycle of the system. | Define "top 10" key reqs | Evo, as with other IID methods, encourages an early definition (in Planguage) of "critical top ten" high-level requirements. They need not all be decomposed into fine details, although those facing early implementation may be. Each iteration, they are reviewed and refined. | Define function specs | Evo functions describe what the system does. Evo does not promote major up-front detailed functional requirements analysis, but it does require at least clear definitions for the next iteration, optionally described in the Function Requirement Specification, using Planguage. example p. 232 | Define performance specs | Evo promotes describing system performance how well the system works, its benefits, and how it affects the environment. Written and refined incrementally. Performance attributes are attached to functions. Specifically, Evo performance attributes fall into three categories: 1) quality how well it performs (usability, reliability, …), 2) workload capacity, and 3) resource savings. The Performance Requirement Specification captures this information using Planguage. example p. 233 | Define clear, and (where possible) measurable, specs | When specifications are written, do so in a manner and language which exposes and minimizes misunderstanding or ambiguity. The Evo Requirement Specification examples illustrate this. Evo promotes a balance between too little and too much detail in requirements. It wants clarity and detail for the key specifications you have chosen to implement in the next short iteration. Other more speculative or unassigned requirements can wait. Evo's performance specifications should have measurable impact, which should be identified. examples p. 231 | Use Planguage for specs | Planguage is Evo's structured language for specifications in both requirements and design. It is optional, but encouraged. Evo includes Planguage templates for its requirements and design specifications. notation p. 231, examples p. 231 |
| Project Management Practices |
Practice | Description |
|---|
Evolutionary project management | Key ideas include: - evolutionary delivery to stakeholders for real use and feedback - small steps (ideally bi-weekly, or between 2 5% of total project financial cost and time) - steps with highest quality-to-cost ratios given highest priority for delivery - the existing system is preferred as the initial system base - feedback modifies future plans and requirements; adaptive planning and evolving specifications - total systems approach; do anything that helps - early results-orientation
discussion p. 227 | Evolutionary delivery | Evolutionary delivery emphasizes delivering a partial solution into production early, in order to obtain early business value, and feedback to guide and evolve future deliverables. A common delivery frequency in Evo is weekly, or more specifically, every 2 5% of duration and budget. In Evo, the solution chosen for delivery in the next iteration is based on highest value-to-cost ratio and early risk reduction. Each iteration's solution can be of a different type. For example, within a project to replace an older mainframe payroll application, early deliverables could be quick-fix operational changes in the existing system, or adding a Web-based front end to the old system, while work on the new system is underway in the backroom. discussion p. 227 | Measure impact of delivered solutions | Evo embraces Shewhart and Deming's core principle of improvement: PDSA. Plus, Drucker's maxim that you can't manage what you can't measure. The study step requires measuring, each iteration, the effect of the solution on the objectives. This data is used to help drive evolutionary project management (act in response to study), iteration by iteration. Evo plan table p. 229 |
Practice | Description |
|---|
Define design specs | Design ideas are also recorded in Planguage, in the Design Specification, and are incrementally evolved, as with requirements specs. example p. 234 | Impact estimation | A method to numerically analyze and compare the effectiveness of design ideas to meet cost and performance requirements the qualities, workload capacity, and resource savings. The results are expressed in an ch10lev3sec7. example p. 235 | Describe how design ideas meet reqs | The design specifications in Evo should explain why and to what degree they fulfill the requirements. This information is used in impact estimation, and discourages "resume-driven design" in which over-engineered or unfocused designs arise that are not really pertinent to business goals and value. example p. 234 |
| Test and Verification Practices |
Practice | Description |
|---|
Specify tests and measures in the reqs | The study step in Plan-Do-Study-Act step requires measurements or meters, in Evo terms. Although new meters can always be adopted, Evo recommends that during performance analysis, the meters for that performance attribute be defined, within the Performance Requirement Specification. example p. 233 | Specification quality control through early inspection | When goals or specifications are written, research shows that defects and misunderstanding are likely. Research also shows that early inspection is a powerful, cheap tool to reduce those defects. Note that specification defects have a precise meaning in Evo: failure to observe a formal, written, required specification rule. Gilb is an expert in the effective use of inspection which is not the same as informal review. The Evo quality control practice includes sampling, and application of the Defect Detection Process, and Defect Prevention Process. details p. 230 |
| Configuration & Change Management |
Practice | Description |
|---|
Specification relationships | The Planguage specification templates contain relationship sections to support requirements traceability. example p. 234 |
Evolutionary Project Management As with Scrum, XP and UP, Evo's project management philosophy is adaptive rather than predictive planning. And, as with the other methods, there is still attention to the long-term vision, objectives, and a robust architecture. Some controlling principles: Financial Control An iteration should be between 2 5% of the total initial financial budget before delivering some measurable results. This excludes larger capital costs that must be incurred in an iteration, such as buying a server, as these are "backroom" expenses. Deadline Control A delivery (or frontroom) iteration should be between 2 5% of total project time, with a lower-bound of one or two weeks. This leads to the official Evo rule of thumb of one-week iterations for a one-year project. The Evo consultant Niels Malotaux has found two-week delivery iterations are more sustainable than one week. Value Control Choose design ideas for the next iteration that deliver the best stakeholder value for costs.
With these control guidelines, the next iteration is chosen in response to the latest measurements and evolving understanding of the requirements. A misstep that doesn't deliver expected value consumes no more than (say) 2% of resources. Future iterations may be tentatively assigned to specific design ideas, and ordered with respect to dependencies, but Evo encourages only very light investment in this kind of predictive planning, as it is central to Evo to adapt the plan at each step. Unless there is a specific stakeholder request for the next iteration, Evo recommends the use of impact estimation table analysis to choose design ideas for the next step. For tracking and adapting, Evo also recommends the use of an impact table to record the results of delivered solutions, and to indicate the steps of the Evo plan. See Table 10.1 for a simplified example after the first iteration. Table 10.1. simplified Evo plan and results table the capitalization in Evo implies these are terms defined in Planguage elsewhereTarget Requirements | Iteration 1 (plan, actual) | Iteration 2 | Cumulative to date |
|---|
Responsive Browsing | 5%, 2%[a] | 10%, __ | 2% | System Reliability | 10%, 5% | 20%, __ | 5% | Capital Costs | 0%, 0% | 5%, __ | 0% | Development Costs | 2%, 2% | 2%, __ | 2% |
[a] the percentage of the final target
Regarding evolutionary delivery: A common Evo project management question is, "If I'm making a new plane (for example), how can I deliver it for use by stakeholders in weekly increments?" Although evolutionary delivery of software is often possible such as bi-weekly refinement to a Web site, or new updates which can be downloaded this of course will not apply to new products with long development lead times. In this case, Evo's approach is to work on their development in the backroom. It could be months before something from the backroom is available for delivery. Meanwhile, Evo still requires that something of measurable value be delivered to stakeholders each frontroom iteration (e.g., every two weeks). For example, early documentation samples, improvements to the existing system or operational environment, and so forth. The last point underlines Evo's total systems approach: Do anything that helps. It is not limited to new software or hardware constuction. Gilb believes there is an expensive and risky tendency to avoid looking at the existing system (when there is one) for the desired improvements sometimes due to technologists' delight in new technologies and thus he promotes in Evo a preference for considering the existing system as the base for improvement. Specification Quality Control (SQC) Through Early Inspection When specifications are created (iteratively), Evo recommends the use of classic systems engineering process control through sampling and inspection. Evo promotes defect removal in specs, done with agility, through its Defect Detection Process and Defect Prevention Process. Evo's SQC draws from IBM's research and practice [e.g., MJHS90], and Gilb's experience; he is co-author of Software Inspection (which emphasizes specification inspection). A key idea in SQC is that specifications are not informally inspected for any kind of fault; rather, there is only a search for defects meaning a violation of a written rule from a rule set or checklist that the "checker" is working against. Here's a simplified defect rule set[4] : [4] Adapted from [Gilb03].
Other key practices in SQC include: Two to five checkers for an inspection. Specification pages are sampled for inspection; the entire document is not checked. If the sampled defect level is above a threshold, the specification is not released for use. The checkers do not volunteer solution or correction advice to the author. They only note issues. It is up to the author to determine solutions or take the initiative to ask the checkers for suggestions.
Defect prevention in Evo is a process improvement activity that comes from collecting inspection data, reflecting on the results, and experimenting with changes in source workproduct creation. Planguage Planguage is Evo's compact specification language. Figure 10.4 shows common notation for one partial specification. Figure 10.4. Planguage
| examples: See "Workproducts" |
Workproducts Full description of Evo's workproducts and how they can be expressed in Planguage is beyond the scope of this introduction. Nevertheless, the following examples provide a sample of Evo's flavor. More detailed examples are given than for the Scrum, XP, and UP chapters, as Evo examples are less well-known and less widely available. Planguage specifications are incrementally developed over the iterations, and only to the extent that doing so adds value. Function Requirement Specification Individual functions are recorded in an Evo Function Specification, using Planguage. These could be a high-level top-ten list of functions, or detailed and decomposed functions. The following example illustrates standard parameters (e.g., "Gist") from the Evo Planguage template. Some statements are purposefully undefined, both for brevity and to emphasize the normal process of partial and evolving specifications in Evo. All capitalized tag elements (e.g., Call Center) refer to other specifications previously defined, probably hyperlinked and clickable. Observe that opinions or "facts" in a specification are sourced to a party; Evo expects claims to have some substantiation, or at least explicit acknowledgment that they are wild guesses. Tag: FLF: Type: Function Specification ======= Basic Information ================= "version, status, owner, stakeholders are elided" Gist: Find lowest fare for air travel. Description: <input: dates, airports, carriers. output: flights sorted by cost> ============= Relationships =============== Relationships: Evo supports requirements traceability in this section. |
Supra-functions: Res.Search Sub-functions: none Is Impacted By: { Call Center, Web Front End } Linked To: Supports: Res.Booking ============= Measurement =============== | Measurement: Goals in Evo should be testable and measurable. |
Test: T1: <correctness test 1> ============= Priority and Risk Management == Rationale: <Our competitors have it> <- Marketing Director Assumptions: A1 [before end of next year]: Competitor X doesn't upgrade A2: < ?? > Dependencies: Res.DB Risks: R2, R6 Priority: Must be in first public release <- Marketing Director ============= Specific Budgets ============= Financial Budget: < ?? > Performance Requirement Specification Individual performance requirements (quality, workload capacity, resource saving) are recorded in the Planguage form shown in this example. Tag: Responsive Browsing: Type: Workload Capacity Requirement: Response: Budget: < ?? > ============= Basic Information ======== "version, status, owner, stakeholders are elided" Ambition: <Many> Res.Users with <acceptable> response time. ============= Measurement =========== | Measurement: Illustrating the quantifiable emphasis in Evo. |
Scale: Average HTTP response time in seconds Meter: Automated HTTP server monitor ============= Targets =============== Goal [First Release]: response under 3 seconds for up to 1,000 requests per second <- Marketing, [Second Release]: response under 2 seconds for up to 1,000 requests per second <- Marketing ============= Constraints =============== Fail [First Release]: response over 6 seconds <- Marketing ============= Benchmarks =============== Past [Old System, last year]: response under 5 seconds for up to 1,000 requests per second <- ABC Research Report Record [CompetitorY, this year]: response under 1 second for up to 3,000 requests per second <- ABC Research Report ============= Relationships =============== | Relationships: Illustrates the performance requirement relates to other performance requirements (or perhaps, directly to functions). |
Is Impacted By: Res.DB.Response <- DBA Impacts: Usability ============= Priority and Risk Management == Value <this level will retain 95% of first-time users> <-Marketing "assumptions, dependencies, etc." Design Specification Design ideas are expressed in the Planguage template form demonstrated in this next example. Tag: Server Cluster: Type: Design Idea ============ Basic Information ============== "version, status, owner, stakeholders are elided" Gist: Cluster of 10 application servers with an IP sprayer. Description: < ?? > ============= Design Relationships =========== Design Constraints: { Use Moon Spark 5000s, Use Java Technologies, Use Open Source } Sub-Designs: < replication, fail over > ============== Impacts Relationships ========== | Impacts: design ideas must be connected to functions and/or performance requirements |
Impacts [Functions]: { Res.Search, Res.Transaction, Res.Browse } Impacts [Intended]: { [Good] Responsive Browsing, [Good] System Reliability, <more> } Impacts [Cost]: { Operations Budget, [if not open source] Development Budget } Impacts [Other Designs]: { Deployment Model, Data Model } Value: < meeting responsiveness and reliability goals will maintain customer retention at 95% <- Marketing Director > == Impact Estimation of Design on Selected Requirements == Impact Estimation: a design idea should contribute to performance objectives. Its impact on each is analyzed. |
Tag: Responsive Browsing Type: Performance Requirement Cross Reference Scale: Average HTTP response time in seconds Scale Impact: under 3 seconds for up to 1,000 requests per second Scale Uncertainty: ± 1 second <- Jill Jones | Note that claims are sourced, and uncertainty and credibility estimated. |
Percentage Impact: [if Use Moon Spark 5000s ] 100%[5] [5] From some baseline (such as "Past") in the requirement.
Percentage Uncertainty: ± 33% Evidence: CompetitorX has this configuration and response Source: Jill Jones (Chief Architect) Credibility: 0.5 as Jill worked for CompetitorX on similar project Tag: System Reliability "repeat analysis using the above set of parameters" ============== Priority and Risk Management ==== "assumptions, dependencies, risks, priority, issues are elided" Impact Estimation Table This tool is used in Evo to analyze the impact of alternative (or complementary) design ideas on performance requirements. Barring "obvious" priorities for the next iteration as indicated by stakeholders, this table is used to rationally choose the set of design ideas to implement next, based on the best benefit-to-cost ratio. Note that the horizontal and vertical summing of impact percentages do not always accurately predict a result; they may or may not provide a sense of aggregate impact. For example, can one sum the Responsive Browsing impact of both a server cluster and high-performance hardware? Perhaps… Table 10.2. simplified impact estimation tableDesign Ideas -> Requirements | Server Cluster | High-performance hardware | Sum of Impact[a] | Responsive Browsing Baseline: 5 sec. Goal: 3 sec. | Scale and % impact[b] | 3 ± 1 sec. 100% ± 50 | 4 ± 1 sec. 50% ± 50 | 150% ± 100 | Evidence and Credibility | CompetitorX has this configuration and response <- Jill Jones 0.2 | Moon Microsystems has customers achieving this <- Moon Sys Eng. 0.1 | | System Reliability Baseline: 3000 hours MTBF. Goal: 3500 | Scale and % impact | 3200 ± 200. 40% ± 40 | 3100 ± 200. 20% ± 40 | 60% ± 80 | Evidence and Credibility | CompetitorX has this config and "suspected" reliability <- Jill Jones 0.2 | Moon Microsystems has customers achieving this <- Moon Sys Eng. 0.1 | | Sum of Impact[c] | 140% | 70% | | Capital/Dev Cost Baseline: $0 USD. Budget: $200K | Amount and % | $20K ± 10K. 10% ± 5 | $100K ± 10K. 50% ± 5 | 60% ± 10 | Evidence and Credibility | Bob's friend guesses this cost on another project <- Bob Bones 0.1 | Moon firm quote <- Moon Sales Rep. 1.0 | | Benefit-to-Cost Ratio[c] | 14 (140% / 10%) | 1.4 (70% / 50%) | | Impact Credibility Adjust Cost Credibility Adjust | 0.84 (14 * 3 *.2)[d] 0.08 (0.84 * .1) | 0.01 (1.4 * .1* .1) 0.01 (0.01 * 1.0) | |
[a] Sum of impacts on a requirement may or may not be cumulative.
[b] The % impacts are with respect to the baseline.
[c] The sum of impacts of one design idea may or may not be cumulative. The total may or may not work as an estimate for comparison.
[d] Multiplying probabilities is a heuristic to reduce total to a reasonable magnitude.0.08 (0.84 * .1)
There is a lighter alternative (for prioritization) to these tables that Evo also offers: the use of simple benefit-cost estimates: Each design idea is given a 0-9 ranking for both benefit and cost. Ideally, this is in a group "delphi" ranking session. The best benefit-to-cost ratio ideas are implemented next. Other Practices and Values Evo has many detailed practices, tips, and guidelines. A sample of points: Open-ended architecture To support evolving or changing designs, and evolutionary delivery, Evo encourages open-ended architectures that encourage easier extension. That is, at predictable variation points, some kind of protection is introduced, such as an interface, data-driven declarations, and so forth. Safety factor The estimated impact of a design should deliver an estimated impact with a defined safety factor, default factor 2 (200% over the target level from the baseline). Client-driven planning If you are uncertain which step to do next, ask your dominant stakeholder. Whatever adds value Rather than a "we are building it" paradigm, focus on "what can I do for my stakeholders next week?" The techniques (such as Planguage and Impact Estimation) are only support to keep this focus, and should not get in the way.
| 