Does £250 cut it? Picking up the pieces from the SQE1 marking debacle

At 3:39 pm last Friday – generally regarded as a good point in the week to bury bad news – an intriguing email arrived at the Gazette. The Solicitors Regulation Authority was inviting us to a media briefing the following Monday in relation to ‘a matter linked to the SQE’. The regulator gave no further details: all would be revealed on Monday.

Half an hour before the briefing, we received a statement from the SRA and its SQE assessment provider Kaplan. It said that Kaplan had reissued results for the January SQE1 exam to ‘correct an error’. The 6,626 candidates who sat the January exam received their results on 14 March. However, due to a marking error, 175 candidates were told that they had failed either the first or second part of SQE1 when, in fact, they passed.

The debacle arose because Kaplan did not round scores at the point in the results process set out in the SQE’s published policy.

‘The way the results were presented was new for January’s SQE1 – rather than results being shown as a percentage mark, candidates were given a standardised score out of 500. The mistake was made when implementing this change. It was unique to the January 2024 results – no previous SQE assessments are affected. It was discovered by Kaplan through general checks conducted during the appeals period,’ the statement said.

On the call to journalists were SRA chief executive Paul Philip, Andrew Conlan-Trant, vice-president of strategic partnerships at Kaplan International, and Zoe Robinson, managing director of Kaplan SQE.

‘There was a mistake in marking the assessments on the SQE1 January sitting a few weeks ago,’ Philip began. ‘It is hugely disappointing and we are enormously apologetic to the candidates impacted. Our main priority when we found out about it was to put it right as quickly as we can and understand the impact, try and understand it and make sure we did the right thing.’

Conlan-Trant said: ‘I want to apologise for the error to the candidates, to the SRA. We really regret this and the impact this has had on the candidates. We want to do right by the candidates impacted here. We worked as quickly as we could once this issue became known to try and rectify it.’

The ‘issue’ became known on 2 April. Candidates were told on Monday morning. Why the 13-day delay? Conlan-Trant said Kaplan needed to identify what happened and correct the error. ‘Once we ran the calculations again, those results had to be checked further and go to the joint assessment board. Before the results were issued, more checks needed to be done.’

'It is hugely disappointing and we are enormously apologetic to the candidates impacted. Our main priority when we found out about it was to put it right as quickly as we can and understand the impact, try and understand it and make sure we did the right thing'

Paul Philip, SRA

Over the coming weeks, Kaplan will try to determine the ‘root cause’ of the error, Conlan-Trant said. ‘When we finish that, it will identify the changes we probably need to make to processes and procedures and perhaps policies associated with the assessment. We’re going to do right by the candidates. We will do a case-by-case review of each of the impacted candidates so we can support them and whatever their specific circumstances are.’

He added: ‘Of course the SRA are disappointed by this. We understand that. We’re terribly disappointed and upset. When something like this happens, when we make a mistake, we do what we can to fix it as quickly as possible. That’s what we have done here.’

In all there were 182 incorrect outcomes, representing 1.5% of the total results issued, Robinson said. Seven candidates received incorrect outcomes in both functioning legal knowledge assessments; 168 received an incorrect outcome in one of the two assessments. Another 645 candidates saw a slight change in their result – but their overall outcome remained unchanged. Scores, at most, went up or down five marks. Some candidates moved by one quintile – 303 moved up, 164 moved down. No candidates were wrongly told they passed.

Robinson said the January sitting was the first delivery of a week-long assessment window for each of the functioning legal knowledge assessments. ‘Within these week-long windows, multiple papers were used. We scaled scores as an alternative and easier way to present the results to achieve the clearest comparison of candidate results even when the candidates have taken different papers. When employing these new scaled scores, we did not round the pass mark and candidate percentage mark before we scaled the score and that’s what we should have done.’

'We’re going to do right by the candidates. We will do a case-by-case review of each of the impacted candidates'

Andrew Conlan-Trant, Kaplan International

The issue was unique to the January SQE1 sitting and no previous SQE assessments were affected, Robinson said. ‘We’re committed to doing a complete and thorough review once we get through this stage of supporting candidates so we can be absolutely certain this mistake will not happen again.’

Kaplan is offering a £250 ‘goodwill’ payment to those wrongly told they failed. Bookings will be prioritised for those in a position to progress to SQE2.

Many would consider £250 and priority booking pretty meagre compensation given the potential fallout, one journalist suggested. Would it not be more appropriate to give affected candidates a full refund and a free SQE2 booking?

Candidates currently pay £1,798 to sit SQE1 and £2,766 to sit SQE2 (fees will rise by 5% in September).

Robinson said Kaplan focused on what would benefit candidates most. The £250 was ‘blanket recognition’ that the result was wrong and Kaplan would do ‘everything we can to put candidates back in a position had this mistake not happened’.

People may have had training contracts withdrawn, or moved back to their home country. Did Kaplan, whose contract runs to 2032, deserve to have its mandate renewed?

Philip replied: ‘This is the most serious operational mistake that we have made in the last two-and-a-half years. We’re working with Kaplan to reduce the probability of it happening again. In saying that, mistakes happen. You cannot run an operation of delivering an assessment [for thousands of people] and expect it to run infinitely smoothly. I’m not undermining how serious this is. We will be in discussion with Kaplan about the way forward.’

Briefing over, the marking error was made public – and details of the potential consequences began to emerge. Patrick McCann, chair of the City of London Law Society’s training committee, told the Gazette that, apart from the financial consequences, there will be a mental health impact on those who saw their future in the law wiped away.

‘It will have a destabilising effect leading to trust issues with the system,’ he said. ‘But the biggest issue is the employment status. People who either had their employment offers rescinded or had their employment terminated potentially on the basis of them having breached their contract terms by not passing their exam.’

McCann himself knows of a few future trainee solicitors among the 175 whose training contracts were rescinded.

The training committee runs the Social Welfare Solicitors Qualification Fund to boost the pipeline of social welfare lawyers. Two people failed but have now passed and can proceed to SQE2. ‘For them, it’s a massive issue because they cannot afford to fund resits.’

Last year the training committee issued guidance on how law firms might best handle SQE ‘fail’ results and counselled employers to make ‘no sudden movements’. McCann said: ‘It appears some firms made some quick decisions, before the conclusion of any assessment result appeal decisions.’

McCann noted that the marking error came to light during the appeals process. He suggested that, in future, ‘law firms might want to delay any employment decisions until after the appeals process has been exhausted’.

Once the findings of the SRA and Kaplan’s investigation are known, the training committee would welcome the opportunity to discuss the findings ‘and hope it will lead to a robust assessment process which all candidates and their employers can trust’.

The Law Society said it will be engaging closely with the SRA ‘to understand what steps are being taken to ensure any future changes are managed more successfully’. The Society encouraged candidates affected by the marking error to turn to the Junior Solicitors Network, mental wellbeing charity LawCare or the Society’s pastoral care helpline for support.

It remains unclear when the findings of the ‘root and branch’ review will be published. Kaplan acknowledged the marking error will have affected candidates wrongly told they failed in different ways. Candidates who incurred losses as a direct result of the error are encouraged to contact Kaplan’s candidate services team at SQE1Jan24@kaplan.co.uk, outlining their circumstances.

The Gazette asked the SRA if Kaplan’s contract will be reviewed earlier than planned, and whether the contract contains any penalty for performance-related issues.

An SRA spokesperson said the contract provides for penalties in some circumstances. ‘Given the commercial sensitivities around this, we can’t go into the details of the contractual arrangements we have with Kaplan. However, and more importantly, our priority here is recognising and addressing the impact on candidates… We will continue to work with Kaplan to make sure the SQE is implemented effectively – making sure it provides a good experience for candidates, and that everyone can trust qualifying solicitors have met the same, high standard.’

As for the consequences for the SRA, we shall have to wait and see.

 

This article is now closed for comment.