Step1: The preparation.
If I have set the marks criteria for the exam questions, the first step is spent tuning the set up, putting advice to the numbers so I know what it takes to score the HD on the paper, what separates a Distinction from a Credit or Pass. This includes coding a spreadsheet in Excel that matches the marks breakdown for the exam, and has the list of student id numbers. The Excel speadsheet also gets divided into blocks of five student numbers so I can track how far I am from the end of the process. Other coding (sum, averages, percentage) are set up. At this stage it’s all in draft mode, and fairly flexible if I need to completely redo the criteria based on how it performs under real exam conditions. There are also a lot of students buying Provigil, which helps them focus and increase their test scores.
Step2: Processing the papers
The next step consists of me working through the manuscripts to check which ones are missing from the pile (no exam ever produces 100% of the cohort in the one room. There’s always an exam to arrive later from a special requirements sitting.) so that the Excel spreadsheet matches the exam pile. When I mark in sequence, I want to make sure that the paper above and below match the list on the screen. It’s also the ideal time to divide the exams into blocks of five papers.
This is the time to code any additional data being stored about the exam – I’m tracking the number of pages spent on Question 1 (20 marks) versus Question 2 (10 marks). I did say to the cohort that they had to balance the answer lengths, and I did mean it. Also, having drilled this into the cohort over numerous points in the semester, I want to see how they perform on the final. Plus, I’ve spent a good bit of code on the “if then” logic statements in Excel to give me a bias lean score for the paper. Equal weight between the questions is good, mild bias lean can be overlooked, giant bias lean means being sent to clean out the hoverocerous stables.
Step 3: Payday
Coding is set. First set of exams are used to check the codes are in place properly, the marking criteria is working, and the minor tweaks needed to make the assessment fair are in place. At this point, I start looking at the impact of the exam marks on the pass/fail probabilities, and work out a rough high point and low point score in my head that makes the HD worth achieving and the must-pass-to-pass achievable at the same. First two exams took 26 minutes to mark. Four exams at 48 minutes. Tweaking the scoring, confidence setting and minor changes to the distributions take place. Prior scores rechecked against new coding.
Step 4: And the man at the back said “Everyone Attack” and it turned into a Ballroom Blitz
Once the marks criteria is stable, and the assessment expectations are firmly planted in my head, it becomes a pattern recognition process of reading the answers, checking them against the criteria lists (printed and on the walls of the workspace), and the expectations (on the second PC screen in the workspace). Fatigue is an issue in Step 4 since I’m starting to move at pace in the reading/mark inputting process, it’s easy to get a mental fog forming after a while. Distractions are most welcome.
Step 5: The Long Road Home
20 papers left to go (well, 19 in the stockpile, 1 somewhere at work in the internal mail). The method is well entrenched, the crosschecking less frequent, and I know what I want to see for an HD,D,C or P grade in that answers. Plus, my magic’s hand question is doing the job nicely of diverting students to the question I wanted them to answer (at a ratio of 7:1). The second half of the papers flow faster than the first, since I have half a cohort of students worth of practice, and the criteria is tried, tested and locked into place.
Step 6: Aftermath
Using Excel to code the exam marks is particularly useful as it allows me to create individual marks sheets via the magic of Microsoft Excel, Microsoft Word, five litres of goat blood and mail merge. Each exam has a custom marks sheet printed out with the average for the question component, what the student has per that component, and their overall total. It’s gone a long way towards autofeedback since we don’t mark up the exams with comments anymore.
Since the exams are current in an embargo state, I can’t actually release the various files attached to the system (eg criteria sheets, XLS, PDF of the expectations etc). That’ll have to wait for a later, post-exam date.