December 15, 2011 | By: Nick Morris

Aa Aa Aa

Surviving the semester: Designing Online Assessments

Designing online assessments can be tricky, and the first questions that should be asked (and I don't mean in the assessment) are: "Is online assessment suitable?" and "Why is online assessment being used?".

And so, to take the second question first…

Why is online assessment being used?

This is a good question, and you should always ask why online assessment being used in a particular situation and why a traditional paper-based form of assessment is not being used. Quite often the simple answer to this will be class-size, that is, it is a big class and there will be a lot of marking. However, it could also be the case that online assessment provides a more efficient and accurate means of conducting assessment as work can be returned quickly to students and also marked in a consistent manner with consistent high quality feedback.

In addition, some forms of assessment do lend themselves rather nicely to be placed on-line, and do offer significant advantages over paper based assessment.

Is online assessment suitable?

This is a tough question. Can online assessment ever be the best option? Does online assessment really test student understanding, or is it just a test of memory?

This is a very old debate, and one that I don't really wish to revisit here at this time. In broad terms there are two views on this, one is that online assessment can really test student understanding and deep learning, and the other view that online assessment can only test shallow learning, that is the retention of facts, and that it can't ever test deep learning (i.e. understanding).

If care is taken with the design of the online assessment you can avoid just testing fact retention and shallow learning, and you can test understanding and deep learning. It can be difficult to get this right. And quite often this will require the use of a number of different styles of questions such as multiple-choice, multiple answer, the missing word type question, and actual numeric calculations, in the same assessment.

Designing Online Assessment

This is where it really starts to get difficult.

First the correct type of question needs to be identified. Is multiple-choice suitable? Or would a multiple answer or matching items style question, or some sort of numeric calculation, be best? By varying the range of question styles used it is possible to test students understanding and not only their ability to memorise facts.

For example, it is often better to set a numerical type question as a numerical input answer as opposed to multiple-choice style question as numerical input tests the students ability to answer the question, as opposed to just picking a right answer from a list of possibles, some of which may in fact be obviously wrong. For example:

What is 2 + 2 equal to?

The student has a 1 in 4 chance of getting it right. Whereas if this was a numerical input the student would have an infinite number of chances of getting it wrong. That is, the students need to have understanding to get the answer, as opposed to being the owner of a lucky pin.

Another example of where online assessment can be used to gauge understanding is the use of a missing word paragraph as opposed to a series of multiple choice questions on the same subject area, for example:

Western blotting is a powerful and useful technique. Proteins are separated by 1.____-PAGE by molecular weight. The proteins are denatured before running on the gel by heating in them in Sample Buffer. The Sample Buffer contains 2.____, which acts as a buffer to maintain pH, 3.____ to denature and unfold the protein, and 4.____ which breaks disulphide bonds and therefore aids denaturation. Once the gel is run the proteins are transferred to nitrocellulose and probed with anitbodies. One critical step in this process is the blocking of extra protein binding sites on the nitrocellulose sheet. These extra protein binding sites can be blocked with 5.____ or milk in the buffer.

Here the student will have to match the missing words to a list of possible words.

Alternatively, the above 5 questions in the paragraph could be asked as a series of multiple-choice questions. However, that would not test students understanding of a particular subject, and it would not give the student the opportunity to gain further insight in to the subject through the completion of a piece of descriptive text.

Also, you can get sneaky and a bit tricky, and instead of using a multiple-choice style question for a question with a single answer, you can use a multiple answer question where there is in fact only one correct answer. Doing this may force the student to think more deeply about the question and also prevent them from just spotting the correct answer and having a guess.

By using a combination of question styles it is possible to put together an online assessment that tests understanding, as opposed to just testing the student's ability to find and understand and remember facts.

Finally,when designing online assessment there are two things that need to be taken into consideration, one is that student collaborate, and the second is the use of what I call the 'Google attack'.

Student collaboration

This is a truly old problem, and something to which paper-based assessments are highly susceptible. If you have a class of 350+ students doing a paper-based assessment then some of them are bound to swap and share the answers. In this situation online assessment has the advantage over paper-based assessment in that you can generate a wide array of 'random' questions so that different students are highly unlikely to get the same set of questions. This approach effectively removes the risk of collaboration.

'Google Attack'

A 'Google Attack' on your carefully crafted set of questions is more difficult to defend against.

In a 'Google Attack' the student will use the question as a Google search and then just take the first answers to come up on Google as the correct answers to the question. Unfortunately this is a particularly good way of cracking multiple choice questions. Numerical, multi-answer and missing words in a paragraph are less susceptible to this type of approach.

To guard against this the best thing to do is avoid multiple choice questions, or if multiple choice questions are the best possible question format for the assessment, then design the questions so that they don't return the correct answer from Google, and also regularly test the questions against Google to see if they are still immune to attack.

Summary

Yes, online assessment, if used correctly can replace paper-based assessments, but care and thought needs to be put in to the design and execution of the online assessment.

Image Credits: Flickr

0 Comment

Surviving the semester: Designing Online Assessments

share

Email your Friend