What is working?

Cynics might say that, as trainers, working is an unfamiliar concept, but that would be unfair and missing the point. What I mean is, how do we measure that e-learning is working? In practice, there are many ways of assessing its effectiveness:

Happiness: The reactions of students are important, because happy students come back for more and, just as significantly, tell their friends. So are students happy with e-learning? Well, according to one report from Forrester “enthusiasm for online learning runs high”. On the other hand “a lack of interactivity and resistance from the old guard are primary roadblocks”.

Time online: Learning management systems make it possible to measure how much time students are spending online, but it’s doubtful whether this means very much. As Stephen Bennett of Click2Learn jokes “you don’t know whether a student has been studying assiduously for hours or just forgotten to log off”.

Bums on seats: Quantity is, of course, not the same thing as quality, but important nonetheless. If you have large numbers participating in your e-learning courses you must be doing something right.

Completions: Many would argue that the number of completions – how many people actually get through to the end of the course – is a more a significant measure than the number of starters. For others this is simply missing the point – surely the whole point of e-learning is that you don’t have to sit courses through to the end; you pick the bits that interest you and then you get out.

Learning: Surely we can all agree that it matters whether students learn or not. A successful e-learning project will bring about a change in behaviour, based on new knowledge, skills and attitudes.

Performance change: But learning is not enough. As John Ruskin said: “What we think or what we know or what we believe is, in the end, of little consequence. The only consequence is what we do.” And in the end, it’s performance change, leading to a healthy return on investment, which will ensure the success of e-learning. It’s not hard to see a healthy ROI from e-learning, simply because it saves so much money compared to traditional methods. What will be interesting is whether anyone will be able to demonstrate financial gain from the benefits of better learning.

Case study: Royal Bank of Scotland and Epic Group

The Royal Bank of Scotland commenced their e-learning rollout with the advantage of many years’ experience in delivering learning to their 650 branches through other technology-based training media, including CD-ROM. Having said that, this was a major strategic venture, designed to satisfy, through e-learning, all of the key knowledge requirements of more than 20,000 retail staff.

Working with e-learning developer Epic Group, RBS designed and produced more than 100 hours of online material, based on a clear definition of job roles and competencies, for delivery through its Training and Communications Network. As a result, the bank received a 700% return on its investment and the programme is now being rolled out throughout NatWest.

RBS still uses a variety of non-online methods to meet its skills requirements, but is experimenting with a virtual classroom, named AREL, as a way of meeting needs which don’t justify a workshop. A typical AREL session, lasting about one hour, includes video, multi-way audio and frequent interactivity. Although a virtual classroom takes some getting used to, employees are finding it a useful way to tackle learning needs, in the company of their colleagues throughout the country.