Certified ScrumMasters Take on Return on Investment (ROI)

I hold certification sessions to train people who are familiar with and have used Scrum to be more effective as ScrumMasters. We look at how these people can better fulfill the role of ScrumMaster in their organizations so that their organizations can maximize the benefits they derive from Scrum. At the end of each certification session, everyone receives Scrum software and methodology to support their work as ScrumMaster. They are also inducted into the ScrumAlliance ( www.scrumalliance.org ), an alliance of Certified ScrumMasters.

The certification session employs team exercises based on a case study to drive home the meaning of Scrum ‚ s practices. The case study involves the hypothetical launch of a Major League Baseball (MLB) Web site. The case study is presented in the following sections. One of the exercises tests the Team members to see whether they are able to engage in meaningful dialog with a really tough customer: George Steinbrenner. The exercise asks the Team members to talk with Steinbrenner about some tough choices. Typical Team performances are presented at the end of the case study.

MLBTix

Overall attendance at baseball games has increased over the last 10 years . In some cities, such as Boston, almost all games are sold out, and obtaining tickets through normal channels is nearly impossible . MLB rules prohibit the resale of tickets at a profit. Scalping is illegal and has been cracked down on recently. The primary distribution channel for buying tickets is an online auction site, xAuction. Although all auctions for tickets on xAuction are supposed to be capped at the retail price plus expenses, MLB has learned that, through a variety of workarounds, these tickets are being scalped for prices of up to 1000 percent of the retail price.

Project Plan

The MLB commissioner ‚ s office hired an external consulting organization, Denture, to plan a project to manage the resale of baseball tickets. Denture delivered the final plan on November 15, and it was subsequently approved. Excerpts of the plan are provided here.

Project Background ‚   ‚   New legislation mandates that as of the 2004 baseball season all ticket resales must take place through MLB-authorized facilities. MLB has decided to develop such a facility on the Web; the site will be known as MLBTix. Through functionality similar to the online auction site, xAuction, but specific to MLB, the public will be able to buy and sell MLB tickets online. Sellers will auction the tickets to the highest bidder, setting an initial bidding price of their own choice without floor or ceiling conditions established by MLBTix. The seller can also limit the duration of the auction by setting a start and an end date and time. If the ticket(s) are successfully sold, the buyer pays the seller through the MLBTix credit card facilities, and the seller mails the tickets to the buyer. Sellers will automatically be notified when buyers receive their tickets, at which point MLBTix will mail a check for the proceeds (less the 25 percent MLB fee) to the seller.

The commissioner will be announcing MLBTix at a news conference on January 15. He hopes to have some functionality available by opening day, March 30, 2004, and for the site to be fully functional by the All Star break, which begins July 18, 2004. Therefore, March 30, 2004 is the anticipated release date. On this date, the MLBTix site will be up, and buyers and sellers will be able to register. Sellers will be able to make tickets available at a fixed price, which buyers will be able to pay in full via credit card. MLBTix is a go-between, but all tickets are transferred directly from sellers to buyers. The release schedule mandates that on June 30, 2004, auction capability be added to site. Finally, on August 30, 2004, buyers will be able to buy groups of collocated tickets, view the locations of seats being sold, and check on inventory.

Funds for the project are ample and should not be considered an unreasonable constraint. The deliverables are the date and functionality. Facilities or packaged software to support MLBTix can be either bought or developed ‚ whichever helps meet the date. The commissioner needs a heads-up on the likelihood that the MLBTix will be available by the above dates prior to his press conference.

Product Backlog ‚   ‚   These are the functional requirements:

• Customers can register as potential sellers of tickets and be assigned a user ID and password.

• Customers can register as potential buyers of tickets and be assigned a user ID and password.

• Customers can maintain a profile under the user ID, including e-mail address, street address, preferences, and credit card information.

• Customers can place tickets up for auction, declaring a floor price, start of auction time/date, and end of auction time/date. Sufficient information should be provided so that buyers can ascertain that the tickets meet their requirements (for the right days, right teams , right number of seats located next to each other, and the seat locations in the ball park).

• Customers can cause an auction to be conducted for the tickets to registered buyers.

• Customer can have MLBTix successfully conclude the auction by awarding the tickets to the highest bidder by the end date and, at the same time, debiting the buyer ‚ s credit card and placing the funds in an MLBTix account.

• MLBTix will notify the seller of the successful sale of the tickets and provide the delivery information for the buyer.

• MLBTix will provide the buyer with a mechanism for indicating that the tickets were not successfully received by the selling date plus a specified period of time (for example, one week).

• MLBTix will transfer the funds for the ticket sale less 25 percent to the seller at the end of the specified delivery time, unless the buyer has indicated otherwise .

• MLBTix will transfer the 25 percent plus any interest to a corporate MLB account from the MLBTix account automatically.

• MLBTix will provide customers with inventory and inventory search capabilities for teams, tickets, dates, and seats.

• MLBTix will provide for promotions on MLBTix.

• MLBTix will be able to identify and ban abusers of MLBTix.

These are nonfunctional requirements. MLBTix must be able to

• Handle 250,000 simultaneous users with sub-second response time.

• Be secure at the anticipated level of financial activity (2,000 tickets per day at an average selling price of $50).

• Be scalable to 1,000,000 simultaneous users if necessary.

• Be 99 percent available, 24 hours a day, 7 days a week.

This is the development context for bidders: The system will be created in a development environment for building Open Source products, using Intel technology and software running on an OpenSQL database server. The development Team members will all live within easy commuting distance of the development site. There are currently cubicles at the development site. The development environment is wireless and has all power and networking capabilities already operating. The development environment uses Open Source development tools such as Eclipse. The development Team is required to use a source code library, check in code every time it ‚ s changed, build the software at least daily, and unit and acceptance test the software every time that it is built. Scrum will be used as the development practice. Use of any other aspects of Extreme Programming or any other engineering practices, such as coding standards, is up to the Team. All of the developers on the Team must have excellent engineering skills and at least be familiar with Scrum and Extreme Programming. The Team must consist of development engineers with excellent design and coding skills. These engineers are responsible for all testing and user documentation, although they can hire contractors to assist with this. The engineers on the Team must average 10 years of progressive experience on software projects using complex technology and Open Source software products. All Team members must be baseball aficionados.

The Project

Imagine that after a quick request for proposal (RFP) process, the commissioner of MLB selected your organization to develop MLBTix. In your response to the RFP, you assured the commissioner that you can meet the release schedule. You were present with the commissioner at a press conference on January 15 when he announced MLBTix, and at this press conference, you demonstrated the functionality completed during the first Sprint. This Sprint began on December 7, 2003, and the Sprint review meeting was held on January 7, 2004.

Your Team has just completed its third Sprint, which ended on March 7, 2004. You have demonstrated the functionality developed during this Sprint to the commissioner. All the functionality necessary for the first release is in place. You intend to pull everything together into the production environment for the planned initiation of MLBTix on March 30, 2004, the start of the MLB 2004 season.

Uh-Oh!

At the Sprint planning meeting for the fourth Sprint, you and the Team become concerned about the capability of MLBTix to handle the kind of volumes that might be encountered . MLB has hired a public relations firm to market MLBTix, and it ‚ s done almost too good of a job: MLBTix has been the rage of every sports page and sports magazine. Everyone who knows about baseball knows about MLBTix and knows that it will be available as of 12:00 p.m. Eastern time on March 30, 2004. There are over 40 million baseball fans, and you know that almost no system could handle 40 million simultaneous hits.

You provide the commissioner with the following background information: The Team contacted several e-commerce retailers and determined that there would be on average 100 visits for every sale. The Team is unable to estimate the exact number of hits that will occur when the Web site first goes up but is worried that it will be more than it can handle. The MLB commissioner ‚ s research indicates that the site will likely sell 2,000 tickets per day in April 2004 and 5,000 per day thereafter for the rest of the season. The average price that will be charged by a seller above retail is $30, of which 25 percent will go to MLBTix. You have previously alerted the commissioner that the database technology Denture recommended the Team use is an iffy proposition at best, and scaling tests have shown the application to be database intensive . Even with all the tuning efforts from the consultants that have been brought in, and even running OpenSQL on the fastest RAID devices possible, the maximum number of simultaneous transactions that can be served with sub-three-second response time is 100 per second. Loads are expected to reach significant peaks at lunchtime and after dinner. The Team is concerned that peak volumes during normal production might overwhelm the server and that the knee of the performance curve will be very close to the 110-transactions-per-second rate. You have determined that the Miracle database will readily support the scaling requirements predicted by the commissioner, but it will take one more Sprint to trade out OpenSQL and implement Miracle database. The upshot? The application won ‚ t be ready until a month after the season opener .

What Advice Should You Provide?

You tell the commissioner all of this. You notice that the commissioner gets increasingly agitated during your presentation, tapping his feet, spitting at the floor, and uttering muffled expletives. He appears to be very unhappy . The commissioner tells you to knock off all of this technology mumbo-jumbo and tell him what he should do. He wants to know whether he should call in his public relations people and tell them to announce that he can ‚ t get MLBTix up. What should you advise the commissioner based on the above risk/reward model and your best instincts ?

How the Teams Respond to This Exercise

I have used this exercise at more than 10 Certified ScrumMaster training courses, where people with some knowledge of Scrum and a lot of knowledge of systems development are trained to be ScrumMasters. I have asked over 200 of these people, grouped into 40 teams, to now provide advice to the commissioner. Here are some of their responses.

Team 1 ‚ s Advice

Team 1 advises the commissioner that he has a scalability problem. Because of his aggressive MLBTix campaigning, the infrastructure will now have to handle more transactions than he had initially predicted. He had told them to use OpenSQL, which just doesn ‚ t scale to this sort of volume. As a result, Team 1 proposes to replace the OpenSQL database with the Miracle database, and this will take at least one additional month. Work will get started on it right away, and the commissioner will be informed as soon as possible whether it will take one or two months. In the meantime, Team 1 advises the commissioner to delay MLBTix indefinitely ‚ at least until this problem is solved .

The commissioner responds, ‚“I didn ‚ t understand a word you said until you told me that I ‚ m going to be publicly humiliated. I told everyone that this site would be up on March 30, and now you ‚ re not only telling me that it won ‚ t be up then, but that you can ‚ t tell me for sure when it will be up. If Derek Jeter ‚ s agent tried this on me, I ‚ d have him run up the flagpole . ‚½

Team 2 ‚ s Advice

Team 2 advises the commissioner that he has nothing to worry about. They are really pleased that MLBTix has been such a success, and they are sure that the underlying technology recommended by Denture will work just fine. Otherwise, why would Denture have recommended it?

The commissioner ‚ s response: ‚“I heard you reassure me, but I also heard you get ready to jump ship by blaming Denture if this doesn ‚ t work. I need advice, not weasel wording. ‚½

Team 3 ‚ s Advice

Team 3 advises the commissioner that it will take steps to handle whatever number of fans come to MLBTix. Just as at Yankee Stadium, if there aren ‚ t enough ticket windows open, lines of fans will form. Team 3 will put in a facility that will tell the excess fans, ‚“Due to overwhelming desire to buy tickets, you will be put on hold until the next agent is available. ‚½ Then, every 30 seconds, the on-hold fan will be told, ‚“ Please stay in line. Your business is very important to us, and we want to serve you. ‚½ Team 3 advises the commissioner that with this approach, MLBTix can service any number of customers without any additional cost.

The commissioner ‚ s response: ‚“I really appreciate your not spending any additional money, but your frugality has really put me in a corner. I hate those recorded messages. I hate lines, but at least while I ‚ m in a real line at Yankee Stadium I can see what ‚ s happening. This is acceptable, but I ‚ m not very pleased. ‚½

Team 4 ‚ s Advice

Team 4 advises the commissioner that the success of the public relations campaign has made some additional work for the development team. It is risky to put up the site the way that it has been built so far ‚ it might work, but its first few days might be atrocious. Team 4 would like to present him with some options. The first option is to let people start accessing MLBTix to register and see what will be at the site starting next week. Customers won ‚ t be able to buy and sell tickets until March 30, but the early availability might reduce the impact of the initial hit enough to avoid the problem. There is still a risk of trouble that you can ‚ t quantify, but this option won ‚ t carry any additional cost. The second option is to beef up the current capabilities by replicating the facilities for the teams with the highest anticipated volume: Yankees, Red Sox, Mariners, Braves, and Giants. This is similar to opening additional ticket windows. The cost of this option is $3.4 million dollars and will allow MLBTix to proceed on schedule with minimal risk. The third option is to delay the introduction of MLBTix by a month while upgrading all of the facilities to handle the greater than expected customer volume. The cost of this option is $1.1 million in additional cost and $425,000 in lost revenues to MLB from the 25 percent commission.

The Commissioner ‚ s response: ‚“I understand. You have clearly laid out my alternatives so that it ‚ s easy for me to weigh my options. You spoke hardly a word of gobbledygook. Hmmm. Take an unknown risk of losing the whole thing, spend $3.4 million dollars and minimize my risk so that I can proceed as is, or pay $1.525 million and suffer the embarrassment of being late but manage to fix the problem for sure. We haven ‚ t evaluated how many customers we ‚ ll lose by being a month late, but that shouldn ‚ t be too much of an issue since we ‚ re a monopoly. Where else can the customers go? So I need to think about whether my pride is worth about $1.9 million dollars. Although I ‚ m proud, I ‚ m not dumb. Go with the third option! ‚½

Lessons Learned

It was very hard for the teams to talk to the Product Owner ‚ in this case, the commissioner ‚ in a language that he could understand. Scrum is based on collaboration, but collaboration requires understanding, which in turn requires good communication. If the Product Owner speaks only in business terms and the Team speaks only in technical terms, there will be no communication, and thus no collaboration.

Think about the MegaBank cash project. When a manager suspected that the project would take longer than he had told his supervisors, he instructed the team to do whatever was necessary to make the date. If he ‚ d had adequate data available, he could have instead weighed the cost of increasing the quality of the system during the additional two months vs. the cost of maintaining a weakly constructed and almost incomprehensible system over the course of its life. He ended up choosing to spend more on future maintenance, even though he didn ‚ t know what his choice would actually cost. As a team member commented to me, ‚“Even though we ‚ re using Scrum, it ‚ s back to business as usual. ‚½

When Denture planned MLBTix, it provided enough information to enable the commissioner to make business tradeoffs. MLBTix was a complex project and was bound to hit snags. With the financial data in hand, however, various alternatives could be posited and a rational decision made. The commissioner priced his pride and chose to keep the money in his pocketbook. Plans that contain sufficient information make it easier to make rational decisions.

Of the four teams described here, only one used the financial data from the plan to offer options that the commissioner understood . The other teams either tried to discuss the problem in a language that wasn ‚ t meaningful to the commissioner or assumed the full risk of the situation by saying that everything was fine. Unfortunately, in most of the classes that I ‚ ve held, many of the teams accept the risk and don ‚ t present options to the commissioner. This is a fairly typical response in the industry, where we in systems development hold the cards tightly to our chest until the end of the project and then let our customers find out for themselves how bad things really are. This isn ‚ t done purposefully; rather, it is the natural result of an environment in which developers don ‚ t really know where the project stands any better than management does.