Exam grading fiasco down to ‘human decision-making’ not algorithm, former Ofqual head says

‘The mistakes were made by humans, not machines,’ Roger Taylor says

Zoe Tidman
Wednesday 16 June 2021 22:15

Exam grading fiasco last year was 'gross miscalucation', says former Ofqual boss

The former head of England’s exam regulator has said that last year’s grading fiasco was caused by “human-decision making” rather than a controversial algorithm.

The problem was not with the algorithm itself but rather “what we were trying to do with it”, according to Roger Taylor, who led Ofqual through the first set of exam cancellations due to the coronavirus pandemic.

Thousands of A-level results were downgraded last year under the original grading system set up in the absence of exams, which saw an algorithm used to moderate teacher-predicted grades.

Following backlash after A-level results day, a government U-turn allowed students to take their initial grades estimated by teachers.

Mr Taylor, who stepped down as Ofqual chair towards the end of last year, said the government’s initial plan for grading “did not work” in a new report offering a personal reflection on what went wrong last year.

“The mistakes were made by humans, not machines,” he said in an essay published by the Centre for Progressive Policy think-tank.

“The exam grades debacle of 2020 has been blamed on a malfunctioning algorithm. But by blaming the algorithm, we risk missing the most important lessons on mistakes that were made.”

Mr Taylor added: “The problem was not the algorithm, it was what we were trying to do with it: it was human decision making that failed.”

The algorithm used in exam moderation last year has faced heavy criticism, with student protesters holding signs condemning it after moderated results came out last year.

After exams were cancelled this year due to the disruption to education caused by the coronavirus pandemic, the government has said no algorithm will be used in determining grades.

“No-one thought algorithmically moderated grades would be uncontroversial,” Mr Taylor said speaking about last year, adding there was “widespread unease about the chances” of the system working.

The former Ofqual chair said relatively few people saw “obviously” wrong grades but that the “much wider sense of injustice” comes from how the majority of students were affected by moderation using the algorithm, which was by having one or more of their results reduced by one grade.

This problem was known from the outset, with Ofqual raising it publicly on two occasions and explaining how lowering grades moderation through moderation could see many with lower than they would have got in an exam, and others with higher, according to Mr Taylor.

“Unfortunately, there was no way of knowing who they were and so there was nothing that could be done about it. It is true that no algorithm can fix this problem. However, it could have been addressed by adopting a different policy,” he said.

Mr Taylor said it was a “huge ask” of somebody to accept their chances of going to university had been “taken away on the basis of an uncertain prediction on what might have happened if exams had not been cancelled”.

“The pain of disappointment falls on all students who were refused places, not just those incorrectly refused. It must do, since no-one can know the results of exams that never took place”, the ex-Ofqual official said.

He said the algorithm to moderate grades did nothing to solve the problem of how young people can be compensated for policies taking away their ability to "produce the evidence" needed to claim their university place.

“Allowing a much larger number of students to be admitted would limit the number who were wrongly excluded. This option was, to my knowledge, never seriously considered,” Mr Taylor added.

Before stepping down as chair at the end of last year, Mr Taylor told the education select committee that Ofqual had warned that the regulator had warned the algorithm was the “worst-case scenario” for determining grades.

He told MPs Ofqual initially advised against cancelling exams last year and suggested holding exams in a socially-distanced manner and delaying exams ahead of calculating grades.

A Department for Education spokesperson said: “All decisions taken on assessments in 2020 were based on delivering the fairest outcome for students. At all times the department worked closely with Ofqual to find solutions that would allow young people to progress to the next stage of their education or career.”

They added: “We lifted number caps in higher education and provided £20m to increase capacity for university places in the 2020/21 academic year.

“UCAS data showed that more students were placed on to their first choice course in 2020 than in 2019.”

Join our new commenting forum

Join thought-provoking conversations, follow other Independent readers and see their replies

View comments