Life beyond multiple choice

The most critical quality of an assessment is its ability to accurately test for the achievement of a learning objective. Assuming you’re starting with specific, observable objectives – and that’s a big assumption – this means selecting the right kind of test item and then carefully crafting the item to provide the right level of difficulty and a freedom from ambiguity. The creation of test items is not a trivial task and requires real attention to detail. Critically, you must test your test with typical learners and be prepared to make changes to ensure the test is both valid and reliable.

What happens if you cannot find a suitable test item to test for a particular learning objective? None of the options available really measures the knowledge, skill or attitude change that you’re after. What you absolutely must not do is to revise your learning objective to make it more easily assessable (don’t laugh – it happens all the time), or make do with an inaccurate test. The fact is you do have choices, many of them still within the online domain.

An example is the American Language Program from Columbia Interactive Arts and Sciences. The computer cannot automatically check the quality of your English (or American in this case), but a human can. For each assignment, students are asked to read documents or listen to simulated voicemails or meetings, write a document in response, review their work against a checklist and then submit the document to an online tutor for marking. A similar process is followed on Columbia’s courses for programmers, where this time code is submitted online for assessment.

PricewaterhouseCoopers use a game, developed by EBC, to help clients assess their ability to fill in complex P11D tax forms. As situations arise, users must resolve the issues correctly or risk fines from the Inland Revenue (you can take realism too far!). EBC developed another creative assessment tool for Huthwaite: realising that sales managers do not always devote as much time to coaching as they should, they devised a self-assessment checklist by which salespeople can reflect on a real sales call they have just made and diagnose what went right and wrong.

Online assessment is just another angle on e-learning and, as such, can be too easily treated as a panacea and, just as easily, written off as a poor substitute for traditional methods. The reality is that computers provide their very own blend of advantages and disadvantages that need to be reviewed carefully against all the other options. Chances are that online assessment, in all its forms, has a place somewhere in your training or educational programme.