Spring Cleaning: More 'fun' with Blackboard (this time assessments and feedback)

The last 2 weeks I've been battling with Blackboard - it really shouldn't take this long to reset a series of courses for the coming academic year.

In my last blog post on Blackboard (see eLearning - The annual spring clean of Blackboard) I wrote about the problems Blackboard has with "normalisation", and displaying meaningful information in summary form. In that post I hinted that I may have found a solution to the lack of information problem. However, unfortunately my solution didn't work. The idea was to use the web clipping functionality of Evernote to record the settings for the individual piece of assessment so that they could be reviewed later. However, the web clipping function of Evernote, which I've never had a problem with before, couldn't handle the Blackboard pages. In the end I solved the "lack of information problem" by saving the pages as PDFs and then reviewing them once I had the module set-up.

Also, in the last 2 weeks I have written around 10,000 questions for Blackboard. This sounds like a lot but it is actually fairly easy to do if you know anything about scripting and how to mash data to form a Blackboard question pool. The type of questions I was writing are to be used as random pool questions in assessments so that the student don't all get the same set of questions and therefore cannot swap answers.

From the student perspective the Blackboard assessment system works very well, with the possible exception of the lack of feedback that is produced. From the instructor perspective the assessment system is a nightmare as not only does it have a lack summary information that allows you to confirm that assessments have been set up correctly with the right deadlines etc., but the assessment system does not 'play well' with the rest of Blackboard.

The problem with Blackboard assessment is there is a lack of control over how and when feedback can be given to the students. For example, if you are running an exam through Blackboard with all the students in a room with the exam open for a a set period of time then you could set the assessment such that when the student submit their answers they get their marks, right and wrong answers, and any feedback. Or, you could have it set up so that they just get their mark and nothing else. However, this approach really doesn't work when it comes to what I call 'open book assessments', that is, an assessment that is on Blackboard that students can do over an extended period of time (say a week or more), and which they can do away from a controlled exam environment.

In an open book assessment there is no easy way to provide marks and feedback to the student after the student submits their work and/or the assessment period is finished (you obviously don't want to release marks and feedback to students whilst the assessment is still running as students may retake the assessment (now armed with all the right answers) or they may give the correct answers to their classmates). The solution is, like a lot of things on Blackboard, not simple or intuitive. (If I'd been designing the system I would have added a tick-box to the assessment system that would allow marks and feedback to be released after the assessment had finished.)

The solution involves the use of the student 'My grades' feature, and the instructor's 'Grade Centre'.

'My grades' - This a is tool that allows students to view their assessment. They can get their marks, and even access feedback.

'The Grade Centre' - This is used by the instructor to control what the students can see in the 'My grades' system.

Now, assuming that you have the assessment set up correctly on Blackboard, that is, you have added it to Blackboard and ticked the box that says the assessment will appear in the 'Grade Centre', and also made sure that none of the boxes for scores, feedback etc. are ticked, then the next thing you need to do is to make sure that in the 'Grade Centre' that the assessment is set so that the students can see it in 'My Grades'. To do this all you have to do is to make sure that in the 'Grade Centre' that there is not a little circle with a red diagonal line through it next to the name of the assessment at the top of the 'Grade Centre' column. If that symbol is present then click on the menu for the column and make the data visible to the students by clicking on 'Show/Hide to users' (make sure you don't click on 'Hide Column' as that will remove the column from the visible part of the Grade Centre). One final adjustment that you need to make is to the way the data is displayed in the 'Grade Centre', if you don't do this then the students will get to see the marks before you want them released.

In the 'Grade Centre' click on the menu icon and select 'Edit Column Information'. In the window that appears find 'Primary Display' and make sure you select 'Complete/Incomplete'. This means that a tick will appear in the 'Grade Centre' when a student submits an assessment, and also that all the student will see in the 'My Grades' system is a tick and no mark.

Now, to get the mark and the feedback to the student you have to do the following once the assessment period has finished:

The students should now be able to view their mark in 'My Grades', and, if you have released right/wrong answers and feedback they should also be able to click on the mark to review the assessment.

The above works fine if you have a class that is just taking one assessment. However, it doesn't work in situations where you have a large class, say 350 students, and you split them into groups with each group taking the same assessment over a different time period.

The first problem you will encounter is that you need to set up each test as a separate item, that is you cannot have the classmates taking the same test. So for example:

Imagine you have two groups of students, 1 and 2, and they do a practical. Group 1 does the practical on Monday. Group 2 does the practical on Tuesday. You want to give both groups the same amount of time to do the assessment - 1 week. Group 1 starts the work on the Monday and submits in on the following Monday. Group 2 starts the work on the Tuesday and submits on the following Tuesday. This can be done in Blackboard by using something called adaptive release and groups.

However, in using adaptive release with assessments you now face the following problem. That is, a given student will not only see the assessment they are supposed to be taking as part of their group in the 'My Grades' section, but they will also see all the other assessments that are being taken by students in the other groups. As you can imagine this would lead to mass confusion and panic amongst the students as they would be worried that they can't complete certain work, or take certain assessments. The bottom line here is that 'My Grades' does not respect the adaptive release settings for a given assessment.

The solution that I have come up with is as follows.

First, I set all the assessments in the 'Grade Centre' so that they do not appear in 'My Grades'. I then create a column in the 'Grade Centre' that takes the marks from the highest attempt for a particular piece of assessment of a given student and returns that mark as a tick. This column I make visible to the students in their 'My Grades'. Effectively what I am doing is blocking access to all the assessment from 'My Grades' and just allowing the students to see a summary.

This is not an ideal solution as students only see work they have submitted, and they don't get to see work that is in progress.

Once the assessment period is over I then make all of the assessment available in 'My Grades' and change the assessment settings as described above so that students can see their mark and get their feedback, and turn off the summary column.

The above is really not an ideal solution as it is very messy, and also create a lot of work for the instructor.

So, where did Blackboard go wrong? How come doing these simple sorts of tasks is so difficult?

In my opinion the underlying model that Blackboard has been created for how courses work at schools and universities is flawed. That is, BlackBoard have not got it right.

Modelling something as complicated as a course at a university, school or college is incredibly difficult, and I know this from my own experience of trying to model two-dimensional gel electrophoresis (see "Gibson*,F., Anderson,L., Babnigg,G., Baker,M., Berth,M., Binz,P.A., Borthwick,A., Cash,P., Day,B.W., Friedman,D.B., Garland,D., Gutstein,H.B., Hoogland,C., Jones,N.A., Khan,A., Klose,J., Lamond,A.I., Lemkin,P.F., Lilley,K.S., Minden,J., Morris,N.J., Paton,N.W., Pisano,M.R., Prime,J.E., Rabilloud,T., Stead,D.A., Taylor,C.F., Voshol,H., Wipat,A., Jones,A.R. (2008) Guidelines for reporting the use of gel electrophoresis in proteomics (MIAPE: Gel Electrophoresis). Nature Biotechnology 26 863-864 (* PhD student in my lab.)"). However, Blackboard is being sold as a cutting edge and leading virtual learning environment for use in education, and yet they do not seem to have grasped some of the fundamentals of assessment.

There is of course the possibility that I've got all the above wrong, and I'm going about things the hard or the wrong way. If that is the case then please feel free to leave your comments below and suggest how I can accomplish what I consider should be a relatively simple task of setting up a series of assessments using Blackboard and giving students timely accesses to marks and feedback.