Piloting the path to quality

All of the quality control measures that we have discussed so far, depend on one simple assumption: that the designer of the e-learning content knows how their product will be used in the real-world. With that knowledge all they have to do is ensure that the product matches up to spec and release it to an eager public.

Case study: Royal Mail and TATA

Speaking at the 2003 Learning Technologies conference, held in London’s Olympia, Nigel Marsh, e-Learning Lead Project Manager at Royal Mail plc, and his colleague, Jane Deed, outlined the key issues and factors that any organisation must take into consideration if they are to deliver a quality e-learning experience to their employees.

Deed explained: “Our experience with e-learning materials has taught us that the processes that are put in place at the very beginning of the project dictate whether or not the project will be a success. You have to decide on the criteria for success – and how these are to be measured – as well as determining how the deployment of the learning materials will be managed and tracked.”

“Any e-learning materials have to be able to be run on any of the delivery technologies available within the organisation – and, within the Royal Mail, that is a wide range indeed!”, she joked. “If this is not the case, the learner will have a ‘bad experience’ with the e-learning materials and will never want to attempt them again.”

“You also have to be aware of the cost of technical development,” she said. “The cost of re-working e-learning materials can be expensive – so you can’t afford to ‘tinker’ with the materials up to the point at which they are rolled out. It’s vital that you ensure that the initial specification is correct.”

The Royal Mail work with one supplier – TATA Interactive Systems (TIS), who have produced some 30 e-learning courses for them over the last two years. Sambit Mohapatra is TIS’s Vice President for UK, Europe and Middle East: “One of the key reasons why we have been able to enjoy such a long-lasting relationship with Royal Mail, and why we have attracted as many as 30 new customers over the past year, in difficult trading conditions, is our commitment to quality.”

“We observed several years ago that re-work was a big component of total project time and that we had to do something if we were to provide a highly reliable and cost-effective service.” Not content with achieving ISO 9001 quality certification, TIS became – last year - the first company providing custom-built e-learning solutions to be assessed at level five on the Capability Maturity Model (CMM) scale. The CMM is the most rigorous quality standard worldwide and encompasses leading companies such as Boeing, Raytheon, IBM, NASA and Motorola. It was a short step for TIS to investigate Six Sigma - a management philosophy originally developed by the Motorola organisation. The central idea behind Six Sigma is that if you can measure how many ‘defects’ you have in a process, you can work out how to eliminate them and get as close to ‘zero defects’ as possible.

Applying these techniques helped TIS to achieve a remarkable 70 per cent reduction in product defects from 44 per thousand to just 18 per thousand within the first quarter of 2002. Said Mohapatra: “Customers are looking for people who can deliver a reliable product on time. A recent KPMG audit demonstrated that we are completing projects on schedule in 97% of cases, something that we couldn’t have dreamed of without our major commitment to quality processes.”

Interestingly, Marsh revealed that, “What has proved to be most beneficial for our business is not necessarily the quality of what TATA produce – although that is high – but the quality of the relationship we have developed with them.” Before developers disregard this chapter to concentrate instead on customer service, they should reflect that they are unlikely to be given the chance to develop this form of long term relationship if their early work does not fully meet customer quality requirements!

According to Microsoft Partner Readiness Manager, Mark Buckley, this assumption is a dangerous one: “In our experience, it’s more important that a learning product matches what the learner is looking for from the experience than that it conforms to a specification. The process should start with an understanding of the learner’s goals with regard to the learning experience and work from there.”

Brian Sutton is Chief Educator with Microsoft’s training partner, QA: “When you look at how e-learning is used in practice you can be surprised. Not only did we find that many managers failed to deliver on their promises to learners of sufficient time and space to do the learning, we also found that learners used the products in unexpected ways. For example, in many cases learners would gather in small groups to go through learning materials or participate in virtual classroom sessions. This isn’t a problem, it’s actually a bonus; it’s just that the materials need to be optimised to support this process.”

According to Sutton, “Developers need to get out of the mindset of ‘what am I going to tell them’ and think instead of ‘how can I best facilitate the required learning’. That’s how trainers would work in the classroom, so why not online? To make sure you’re really delivering on what the learner requires, there’s only one answer and that’s piloting, with real users in the real job environment.”

Piloting, usability testing, content testing, conformance testing – readers would be forgiven for thinking that life’s too short. In practice, these activities sound like a much more onerous commitment than they really are, when compared to the time spent on research, scripting, graphic design and authoring. The reality is that for every hour spent on quality control, many more hours can be cut from the overall schedule and the better the chance that, once your e-learning product is finally launched, you can sit back and bask in the glory, rather than running for cover.