Real-world testing

Speaking at the 2003 Learning Technologies conference, held in London’s Olympia, Nigel Marsh, e-Learning Lead Project Manager at Royal Mail plc, and his colleague, Jane Deed, outlined the key issues and factors that any organisation must take into consideration if they are to deliver a quality e-learning experience to their employees .

Deed explained: “Our experience with e-learning materials has taught us that the processes that are put in place at the very beginning of the project dictate whether or not the project will be a success. You have to decide on the criteria for success – and how these are to be measured – as well as determining how the deployment of the learning materials will be managed and tracked.”

“Any e-learning materials have to be able to be run on any of the delivery technologies available within the organisation – and, within the Royal Mail, that is a wide range indeed!”, she joked. “If this is not the case, the learner will have a ‘bad experience’ with the e-learning materials and will never want to attempt them again.”

“You also have to be aware of the cost of technical development,” she said. “The cost of re-working e-learning materials can be expensive – so you can’t afford to ‘tinker’ with the materials up to the point at which they are rolled out. It’s vital that you ensure that the initial specification is correct.”

The Royal Mail work with one supplier – TATA Interactive Systems (TIS), who have produced some 30 e-learning courses for them over the last two years . Sambit Mohapatra is TIS’s Vice President for UK, Europe and Middle East: “One of the key reasons why we have been able to enjoy such a long- lasting relationship with Royal Mail, and why we have attracted as many as 30 new customers over the past year, in difficult trading conditions, is our commitment to quality.”

“We observed several years ago that re-work was a big component of total project time and that we had to do something if we were to provide a highly reliable and cost-effective service.” Not content with achieving ISO 9001 quality certification, TIS became – last year - the first company providing custom-built e-learning solutions to be assessed at level five on the Capability Maturity Model (CMM) scale. The CMM is the most rigorous quality standard worldwide and encompasses leading companies such as Boeing, Raytheon, IBM, NASA and Motorola. It was a short step for TIS to investigate Six Sigma - a management philosophy originally developed by the Motorola organisation. The central idea behind Six Sigma is that if you can measure how many ‘defects’ you have in a process, you can work out how to eliminate them and get as close to ‘zero defects’ as possible.

Applying these techniques helped TIS to achieve a remarkable 70 per cent reduction in product defects from 44 per thousand to just 18 per thousand within the first quarter of 2002. Said Mohapatra: “Customers are looking for people who can deliver a reliable product on time. A recent KPMG audit demonstrated that we are completing projects on schedule in 97% of cases, something that we couldn’t have dreamed of without our major commitment to quality processes.”

Interestingly, Marsh revealed that, “What has proved to be most beneficial for our business is not necessarily the quality of what TATA produce – although that is high – but the quality of the relationship we have developed with them.” Before developers disregard this chapter to concentrate instead on customer service, they should reflect that they are unlikely to be given the chance to develop this form of long term relationship if their early work does not fully meet customer quality requirements!