Checking Out - Assessing Learning Online

One thing that computers do really well – and with much less effort than human beings – is to run a test; a certain type of test that is, using highly-structured question formats for which answer judging, scoring and feedback can be readily automated. Because tests are not difficult to put together and deploy online – at least not at a superficial level – practically everyone does it. But what purpose do these tests really serve and do they provide us with the information we need about what students have learned? In this chapter, I explore the opportunities provided by online assessment and, just as importantly, examine the limitations.

Computers love tests

Computers are good at testing learning. They never show bias, they can add up perfectly and they just love to provide you with charts and reports to tell you exactly who has done what to whom, when they did it and where. What’s more, they can run authoring software that makes it a relatively simple form-filling exercise to create new tests that can be delivered securely and consistently across the globe, with not a traditional exam paper in sight. Online assessment looks easy and, superficially, it is. However, there’s more to making a success of online assessment than buying a software package and typing in a set of questions. A great deal of skill and judgment is required to ensure that what you’re testing bears any relationship to your real learning objectives and that online testing is used selectively, often alongside other assessment methods, both online and offline.

Simon Hoyle heads up EBC, a Manchester-based developer of bespoke e-learning solutions: “It’s important to test the impact of learning, but most assessments really only check for knowledge, not skills. You need a follow-on process to look at what people do differently back on the job.” Henry Stewart, MD of e-learning content provider learnfish, agrees: “The key question is whether tests that use multiple choice questions or similar is an effective measure of skills transfer. I don’t believe they are. In teaching software applications, the real test is to have learners carry out tasks on a live document and email them to an e-tutor. This is a genuine test of skills transfer and has the advantage of increasing human contact, which we know increases the effectiveness of e-learning.”

Is this attack on the usefulness of online tests fair? Well only up to a point. There are many situations where knowledge is what you want to be testing, because the transfer of knowledge is the point of the exercise. A large proportion of today’s workforce is made up of knowledge workers – people for whom the knowledge of concepts, principles, rules and procedures is absolutely essential. They are the raw material from which sensible judgments are made and actions initiated. Online assessments may not be able to assess all forms of knowledge adequately, but they can make a major contribution.

To further the case for online assessment, it is also fair to point out that many forms of computerised testing can in fact test for skills. These days, very many of the skills we need are cognitive – they do not involve any manual activity on behalf of the performer. And cognitive skills are important – skills like problem-solving, decision-making, analysis, synthesis and evaluation. Not all cognitive skills can be thoroughly or even adequately tested using tests delivered by computer, but enough can to make the method useful. And that’s not to mention such basic skills as arithmetic, typing and spelling, where the computer can actually generate test items automatically and to precisely fit the learner’s profile.

Lastly, as even Stewart has to admit, tests, or ‘quizzes’ as they are often coined to improve their user-friendliness, “are very effective motivational tools. Learners love completing them”.