Section 11.3. Collaborate with the Vendor

11.3. Collaborate with the Vendor

If you are the project manager on an outsourced project, your day-to-day work will be similar to what you would do on a project for software being developed in-house. There are, however, some important changes you need to make, in order to work with the vendor. Because it's so easy for the vendor to get lost in the details and lose context, tools and techniques in every phase of the software project must be modified, in order to keep the vendor in the loop and communicate your high-level goals for the product.

11.3.1. Plan and Manage the Project Scope

In an in-house project, you start with the project scope and the set of resources already known to the organization, and use those to estimate the schedule, budget, and due date (using the tools and techniques in Chapters 2 through 4). An outsourced project, on the other hand, is exactly the opposite: you start with a scope and a budget, and the vendor provides an estimate on the number of resources and the time expected to complete the project.

This is one of the main advantages to outsourcing: you have much more flexibility in allocation of resources. You can specify the scope and the expected budget, and ask for an estimate on both the number of resources and the expected time to complete the project. Alternately, you can specify the scope and the deadline, and ask the outsourcing vendor to estimate the number of resources and the project cost. However, in all cases, you will still need to know the scope of the project. Luckily, there is very little difference in defining the scope of an outsourced project and defining one within your organization. The description of the vision and scope document in Chapter 2 can be used in an outsourced context as well.

However, there is one aspect of the project that is known during the planning stage but that is not covered in a typical vision and scope document for an in-house project: knowledge transfer. The outsourced vendor will not have prior knowledge of your organization, its products, or its users. It is your responsibility to teach your team about your organization and its goals. This can be done through meetings, documentation, working off-site at the vendor, bringing consultants from the vendor on-site to work with your organization's management and experts, or some combination of these things.

The main advantage to having someone from the vendor come to work at your organization is that it is less disruptive to you. It can be a successful way of transferring information; the person who works with you can act as your advocate within the project team, and she can use her understanding of your needs to keep the project team on track. The disadvantage is that you are depending on that person to collect the information, rather than teaching it yourself. This means that any misunderstanding might not be easily recognized early in the project. Also, each piece of information must first be learned by the vendor's representative and then retaught to all of the other members of the team; that takes time, and it may be less reliable than teaching it yourself.

The most effective way to avoid these problems is by working directly with the vendor's team. (Many vendors are located in the same city as the organizations they work for, but if your team is in another city or country, this could mean spending weeks, or even months, living out of a suitcase!) By going directly to the team, you can be sure that you have communicated your needs effectively. You can verify the team's knowledge in conversation and, if necessary, through a test or quiz. When a project manager works directly with the vendor's team, the knowledge transfer takes less time and there is less chance for misunderstanding.

Your method for knowledge transfer should be decided and covered in the vision and scope document. One effective way to do this is to include the vendor as a project stakeholder. This should make sense: the vendor has clear needs (including knowledge transfer) that must be fulfilled on the project. By adding the vendor as a stakeholder, you ensure that those needs will be considered when planning the project tasks.

Some project managers limit their interaction with the team to a set of milestone reviews during the project. For those managers, deadlines are the only monitoring tools. This is a mistakethe project manager cannot be disconnected from the project like that. It takes time and effort to work with the individual team members to verify that the work is being done properly, and that it meets the standards that your organization needs; if you don't take the time to do this, you can lose control of your project and not even know it. Unfortunately, by default, most outsourcing projects are set up so that the project manager at the client has a very small role in the project, and the contractor is responsible for keeping the team on track.

Success for the project manager and success for the vendor are often two different things. The project manager wants to deliver the software that meets the needs of the stakeholders in the project. The subcontracted team is trying to meet its contractual obligations. If the contract is not written specifically enough, or if the project manager is not able to collaborate with the team and revise its goals when necessary, it is likely that the goals of the project manager and the goals of the subcontracted team will diverge and possibly even be in conflict with one another.

11.3.2. Do Your Own Estimation

The estimation process in Chapter 2 was based on the assumption that the work being estimated would be done by a known project team. That isn't always the case when you work with an outsourced vendor. Once your team is in place, you can use the process outlined in the chapter in much the same way. But when you are negotiating your initial contract, the people who are estimating the effort are most likely not the people who will eventually do the work.

Learn about your resources: ask them about their backgrounds and about what they are capable of, in order to understand who should be assigned to various tasks. Find out who is more senior in your team; adjust expectations, if necessary, based on the new estimates they give you. Remember, even if they don't work for your organizations, these are not just faceless resourcesthey're people with different capabilities and skill sets. The more effort you put into understanding them, the more likely they will understand your project and its goals, and the more accurate your team's estimates will be.

Once your team has been created, you should work with them; gather the team members together and hold a Wideband Delphi estimation session. (Alternately, use another method to gather estimates from the teamthe important thing is that the team uses a repeatable method for estimating the software, and that they agree that the estimates are realistic.) When the team makes the new estimate, use the same materials that were used to create the initial contract. This is a your first chance to see if your original expectations were realistic.

It's important to keep in mind that the original estimate done by the vendor may already have been written into a contract. If the numbers that your team comes up with are different than those written into the contract, you may have a problem with your budget or with your own legal people. You may need to renegotiate the contract, or you may need to add (and pay for) more resources, in order to meet your deadlines. But it's much better to know this at the beginning of the project rather than find out later on, when the work is underway.

One common and very unfortunate pitfall that many people fall into is assuming that all of the estimates from a vendor are padded (just like an in-house project!). Even worse, many project managers assume that the vendors' estimates are chronically underestimated ("They said they only need two weeks for this task, but they'll really take five"). This is a mistake. If you consistently mistrust the estimates coming from the group, the people making those estimates will very quickly catch on and begin to meet your expectations. If your team really does have a problem estimating, that's a problem that should be dealt with and corrected through tracking the schedule variance and other metrics in the same way you would track any employee on your team who has such problems (see Chapter 4).

11.3.3. Maintain Your Own Project Schedule

Giving up control of the schedule is a common mistake. It allows project managers, who are responsible for the ultimate success or failure of their project, to maintain almost no knowledge of how it is progressing or of who is doing what. It is your job to know why things are slipping, and whether or not commitments will be metand you can't expect to adequately understand the complexities of your project with just a couple of status meetings. You must be an active participant in gathering knowledge about your project. In order to find out how the work is progressing and understand the problems your project is facingwhich are necessary in order to make informed suggestionsyou need to maintain your own project schedule.

Many project managers take a hands-off approach toward managing their outsourced projects , when they would never take such a risk with projects developed in-house. And it really is a huge risk; it means giving up a lot of control. Organizations of all types stretch the truth to keep their clients happy, and outsourced vendors are no exception. Keeping control of your project means verifying the status and the quality of the work product in the exact same way you would on your in-house project team. Review the work through both formal inspections and informal peer reviews (see the following section) to maintain an active understanding of your project tasks and their progress. But above all, know who is doing what and how far they have progressed.

If your project was done in-house, you would never let anyone keep the schedule for you. The project schedule should be kept and updated by you just as you would keep it in your organization. You should be notified immediately of any slippage, and hold regular schedule reviews as well as event-driven reviews with your outsourced project team. Everything in Chapter 4 applies to your outsourced project team, and it is your responsibility to stay on top of it. If you don't, the team will keep moving forward on their schedule, perhaps toward their own goals, and will continue to bill you for their time, whether or not it meets your needs.

11.3.4. Hold Reviews and Inspections

A review is one of the most important tools a project manager has for knowledge transfer, and it is difficult to overstate the importance of reviews and inspections in an outsourced project.

The more feedback you give, the more the team will understand what you want. One of the most common causes for outsourced project failure is that the project manager does not check the team's work until major milestones are delivered. If the team misunderstands a major work product and it is not inspected until the end of a project phase, the effort for that entire phase could be wasted. Many of the most serious problems that plague outsourced projects can be caught early with inspection and constant collaboration.

Unfortunately, reviews in outsourced projects can be highly time consuming; much more so, in fact, than in an in-house project. In an in-house project, the team is already familiar with that particular organization's standards, and there are usually plenty of examples to work from. The project manager doesn't need to spend nearly as much time making sure that the team understands the work being accomplished. What's more, an in-house team normally understands the mission of the organization and the needs of its users. Many project managers take this for granted, and don't think to communicate these things to the vendor. It requires constant effort and vigilance on the part of the project manager to make sure that the needs are properly understood when moving work outside the organization.

In addition to knowledge transfer, reviews are also important tools for collaboration. It is important to encourage collaboration between the project team members at the vendor and the team members within the organization. When an inspection team is made up of people from both organizations, the only way for them to reach consensus on a work product, in order to approve it, is to collaborate on identifying and fixing the defects in that work product. After the inspection, everyone has a better understanding of the work to be done, as well as of how everyone else thinks about that work.

At the outset of the project, you must figure out which work products need to be inspected. You should add these inspections to the schedule as strategic milestones, to ensure that the vendor is in the loop. However, it will often be too time consuming to inspect every work product that you would in an in-house project, because inspections that span multiple organizations are much more effort-intensive than inspections that only involve people from a single organization. It will take some practice and experimentation before you find the "sweet spot" where you are catching enough defects, encouraging sufficient collaboration, transferring enough knowledge, and, most importantly, giving the proper guidance to your vendor team.

The most direct route for identifying and fixing defects is to have everyone on the inspection team meet in person. However, there are times (such as when an outsourced vendor team is in another country) when this is simply unfeasible. Luckily, there is a highly successful precedent in the software industry of collaboration without face-to-face meetings. Some of the most successful open source projects (such as Linux, Apache, Mozilla, Perl, PostgreSQL, and Subversion) are excellent examples of distributed teams who review each others' work without having face-to-face meetings. Project teams for most large, successful open source projects accomplish this through discussion groups, mailing lists, and other collaboration tools. They also browse log messages from the version control system, and use an automated project monitoring system (see Chapter 7) to keep the team up-to-date on the health of the code. Using these tools, project team members can collaborate on resolving the defects without having to meet face-to-face, but also without requiring enormous effort from the moderator. People collaborating in this way have produced some of the most reliable, defect-free software available at the time of this writing. You can take advantage of this in your own software projects.

Table 11-1 shows an inspection process that has been modified to be used with an outsourced project. This script differs from the one in Chapter 5 in that it does not require an inspection meeting. Instead, the inspectors prepare comments and send them back to the moderator, who consolidates them and works with individual inspectors to identify solutions that they all agree on. This requires much more time than a single inspection meeting because instead of having one single discussion about each defect, the moderator must have many different discussions with individual inspectors regarding each defect. It also requires that the selected moderator have extensive familiarity and expertise with the work product being inspected. This may mean that the project manager must serve as the moderator, but that's not always the case.

Table 11-1. Inspection script for multiple organizations

Name	Inspection script for use in multiple organizations
Purpose	To run a moderated inspection (without a meeting) for a team with members in different organizations
Summary	In an inspection, a moderator leads a team of reviewers in reviewing a work product and fixing any defects that are found. The inspectors are from multiple organizations, so they never meet face to face.
Work Products	Input Work product being inspected Output Inspection log
Entry Criteria	A moderator must be selected, as well as team of 3 to 10 people. A work product must be selected, and each team member has read it individually and identified all wording that must be changed or clarified before he or she will approve the work product. A unique version number has been assigned to the work product.
Basic Course of Events	Preparation. The moderator distributes a printed or electronic version of the work product (with line numbers) to each inspector, along with a checklist to aid in the review. Each inspector reads the work product and identifies any defects that must be resolved, compiles those defects into a single document, and returns it to the moderator. Compile the draft inspection log. Each list of defects returned by each inspector must be compared with the others, in order to identify and combine overlapping defects. The moderator compiles a draft of the inspection log that includes all distinct defects found by inspectors. The log does not yet contain any solutions to those defects. Identify conflicts. The moderator searches for any defects reported by different inspectors that contradict each other. For each set of conflicting defects, the moderator holds a discussion (either in person, via teleconference or video conference, or using a collaboration tool like a mailing list or instant message system) between the inspectors who identified those defects, in order to identify the assumptions behind the defects and resolve them into a single defect. The inspection log is updated to reflect the combined defects. Identify solutions. The moderator uses the same means to meet with individual inspectors, to identify solutions to the defects and add those solutions to the inspection log. If more than one person identified the same defect, they must all be involved in creating the solution. Inspectors may also identify additional defects that were not originally found, as well as their solutions. Compile and distribute inspection log. The moderator compiles all solutions identified in Step 4 into the inspection log. Any defects that were not resolved are left as open issues to be resolved by the author. The moderator sends the final inspection log to all inspectors for confirmation. When the inspectors have confirmed that the log is correct, it is sent to the author of the work product. Rework. The author repairs the defects identified in the inspection meeting. Follow-up. Inspection team members verify that the defects were repaired. Approval. The inspection team approves the work product.
Alternative Paths	During Step 5, if one or more team members find errors in the inspection log, the moderator must address those errors before rework can occur. The script returns to Step 2.
Exit Criteria	The work product has been approved.

While not every work product can be inspected, every work product in your project should be reviewed. Use deskchecks to have people in your organization spot-check the work done by vendors. This will help to find errors early on, and it can also generate confidence within your project team. If you are on top of the errors in your project, you can quickly correct problems if things start to go off track. There should be no surprises when delivery time comes.

In addition to inspecting work products, it is important to encourage the team within your organization to mentor their counterparts in the outsource vendor (and vice versa!). The more the two teams learn from each other, the less they will make mistakes or blame one another when things go wrong. The project manager of an outsourced project should try to create a cohesive team for the project that spans both organizations. While your team members may be paid by different organizations, they are all working toward the same goals and should feel comfortable both criticizing and praising each other's work.

11.3.5. Maintain Control of Design and Programming

The challenge of managing the design and programming of an outsourced project is collaborating with your engineering team. Clearly, the programming work is going to be taking place in the vendor's organization; the only question is whether the various aspects of the design take place in your own organization or at the vendor.

There are always design constraints for any piece of software: it may need to work with an existing enterprise infrastructure; you may have legacy code that must be integrated; you or others in your organization may have already decided on a language or platform; there may be user interface or other design standards that are already in place and must be met, etc. Minimally, you must make sure that these are communicated to the vendor.

The easiest way to maintain control over the design of the software is to design it yourself. Sometimes it's easiest to come up with a solution yourself that meets all of your needs, and to provide that as technical direction to the programming team at the vendor. You can do this by writing a technical specification or approach document. If you do this, it's very important that you put checkpoints in place, in order to make sure that the work is really being done according to these documents. The vendor team should inspect the document, and there should be periodic reviews or walkthroughs scheduled throughout the project, in order to verify that the work is being done according to your design.

One of the most common mistakes that project managers make on outsourced projects is to blindly trust that the vendor team fully understands and has properly interpreted all of the design documentation provided. If you do not check in at many points along the way to make sure that your intentions are being properly interpreted, it's very likely that you will end up getting software that technically complies with the specifications but that does not really meet your needs. This is a very frustrating position for a project manager. However, by implementing reviews and inspections (see above), you can avoid this problem.

Once you have put in place controls to ensure that the software design meets your organization's needs, you then need to ensure that the software is being built well. The tools and techniques in Chapter 7 are especially helpful here. Require that the team build unit tests for all code, and that those unit tests are run regularly, with the results of the unit tests published across the teams. Use an automated project monitoring system to create nightly builds and run unit tests, and have the results published across teams. Have in-house software engineers do spot-check code reviews with vendor team memberspreferably using code samples that were already reviewed by the vendor team. (To some people, it may seem like a waste of time to review code that has already been reviewed. In fact, it's especially helpful, because it allows your team to both verify that the code is being built properly and to audit the vendor's code review process.) Use automated code analyzers to enforce widely accepted coding best practices that may slip through a code review.

Some project managers may be hesitant to get this involved with the way the vendor team builds the code. It may seem somehow intrusive that you are second-guessing the abilities of the programmers at the vendor. It's important not to fall into this trap. Almost every vendor will welcome this collaboration. It means that you are sharing responsibility for the quality of the code. You are recognizing the challenges that the team developing the product will face, and you are helping them solve those problems. And it means, ultimately, that there will be no surprises: you won't end up with interminable QA cycles. In other words, both the project manager and the vendor should see this as a win-win situation.

11.3.6. Take Responsibility for Quality

You are responsible for the quality of your products, even when you contract the work out. It's easy to forget this. If you are paying a vendor to build a project, it somehow makes it easy to consider quality just another deliverable. It's notit's a responsibility.

Many times, outside providers will offer "testing" of their products as part of the development estimate. It's not uncommon for this testing to be done not by an independent testing team but by the programmers who wrote it, or by junior members of the programming team with little or no test experience. When this is the case, they generally do not have much test documentation. Do not let this happen to your project. Demand resources allocated exclusively to test activities. Expect to review and inspect all test documentation. Be sure that your test activities are defined in your vision and scope document and, if possible, discussed in your contract.

Just as you would have members of your test team involved in all document inspections in your own organization, schedule your designated test team outside to review all documents and schedule their time researching and writing test plans as well. If you use specific metrics to measure product quality, expect your test team off-site to provide the same data and meet the same standards. Collaborate with them as you would your own teamspot-check the test plans, and test the results to be sure that the testing is being done as you would expect. Defect tracking should be done within your own organization. Test results should be monitored, and redundant tests should be done within your organization to be sure that bugs aren't slipping through.

Don't make decisions that undercut your QA team. When the project has reached its testing phase, there are many points when the project manager has to decide between doing things in less time and doing them thoroughly. As the end date looms closer and closer, it becomes less and less appealing to choose the "thorough" option.

For example, say you have a project 25% through its regression test cycle after a minor bug fix. Everyone expected the test to go well, but instead, a major defect was found. The lead tester at the vendor asks whether they should cut a new build now or if they should finish the regression test. It's possible that there are no more major bugs lurking within the code, and you could potentially cut out 75% of the time it takes to regress the software. Do you do it? If you are taking responsibility for the quality of the product, the answer is "absolutely not": you know that chances you will find another defect when you cut the next build are greater if you haven't completed the current regression test. But it's difficult to make this decision with looming deadlines, clients putting pressure on you, and, most importantly, a compliant vendor willing to cut out work in order to meet your deadline.

It's even harder to keep your commitment to software quality when a small change is made to a product that is already in production and rolled out to clients. It's very tempting to "just make the change" and run a cursory test (for example, testing only the fix itself or smoke testing only a few "core" areas of the software). If you stop to think about it, however, it's even more important to run a full regression test when making even a small change to a product that's already been rolled out. The users already have a good feeling about the software and expect it to keep meeting their needs. It's one thing to deliver a poor quality product from the beginning; it's quite another to replace software that's already working with software that's buggy. There's no easier way to upset your users and make your team look incompetent.

The point is that if you want to release software that will satisfy your users and stakeholders, then your off-site team needs your support, and your commitment to quality must be unwavering. If time is a real constraint, you should use proper planning. People in your organization must review and understand the test approach the vendor will take. Don't ever commit to deadlines that don't include estimates for software testing. (And always assume there will be multiple iterations of testingit's foolish to assume there will be no defects and the first test will be the last!) If the estimated project plan runs past the deadline, cut the scopenot just the testing. Work with your QA team to identify time-consuming activities that can be automated or made more efficient.

Finally, don't just assume that just because the vendor's organization meets certain certifications or has been assessed (CMM Level 5, ISO 9000, Six Sigmasee Chapter 12), it means that they know better than you do how to run your project. Don't assume that the vendor never cuts corners. A good track record for past projects does not necessarily translate to a similar performance on your project. Even when you are working with a certified vendor, you still need to take responsibility for your portion of the work.