News focus: First SQE results expose worrying attainment gaps

After a few technical glitches (more on that later), the Solicitors Regulation Authority finally revealed the first batch of results for the Solicitors Qualifying Examination last Friday. 

Just over half (53%) of 1,090 candidates passed the first SQE1 assessment, which tests functioning legal knowledge. The SRA pointed out that pass rates for the Legal Practice Course, which the SQE will eventually replace, ranged from 23% to 100% between 2018 and 2020.

The SQE was designed to improve accessibility. A further breakdown of the results shows that those who went to non-selective state schools did slightly better than privately educated candidates, though the regulator said factors such as a top grade at university or prior work experience were indicators of a greater likelihood of success.

Worryingly, though, the attainment gap between white, black and Asian candidates appears to have widened. SQE pass rates were significantly higher for white candidates – 66% compared with 43% for Asian candidates and 39% for black candidates. LPC completion rates for 2019/20 were 65% for white candidates, 52% for Asian candidates and 39% for black candidates.

However, Julie Brannan (pictured), the SRA’s director of education and training, issued a word of caution on interpreting the data. She pointed out that 60% of candidates did not complete the equality, diversity and inclusion (EDI) questions, so the data is based on incomplete information. Also, the first exam was taken by 1,000 candidates. The SRA expects its annual exam cohort to grow to 10,000.

To its credit, the SRA has tried to ensure the SQE does not disadvantage candidates from minority ethnic communities. Those who wrote the exam questions received unconscious bias training and all questions were reviewed for cultural bias. For example, Brannan explained, a question might begin ‘an old client has been in touch’. Is ‘old’ referring to age or that they are a longstanding client? Another example is ‘a client is buying an old water mill’. Some might think of a nice, quaint house ‘but a candidate from India might see that as an industrial building’, she said.

The questions do not mention people by name. They refer to ‘a man’, ‘a woman’, ‘a house seller’ or the person’s role or function. A professional conduct question might refer to a particular characteristic if it is about prejudice or discrimination.

The SQE’s independent assessor, Geoff Coombe, was satisfied that all reasonable steps were taken to avoid bias, but recommended that the SRA and exam provider Kaplan continue to investigate the performance difference by ethnicity.

'It is open to training providers to feed back whatever it is they want to say. EDI will be one of the things we want to look at'

Julie Brannan, SRA

After a pilot of SQE1 found that white candidates generally performed better than black, Asian and other minority ethnic candidates, the SRA appointed the University of Exeter to investigate the reasons behind the attainment gap.

The Law Society says other research on these gaps in higher education and professional exams shows that many of the underlying factors are likely to relate to the wider learning and training environment.

Preparation courses are not compulsory for the SQE but will undoubtedly give candidates who do them an advantage. Will the SRA be looking at who is doing such a course to see if that might explain the attainment gap?

Brannan will be talking to education providers next month ‘to discuss a lot of things’ in relation to the SQE, adding that it is ‘open to training providers to feed back whatever it is they want to say. EDI will be one of the things we want to look at’.

Another notable finding from last week’s results was the disparity between the two functioning legal knowledge tests – FLK1 and FLK2 – that make up SQE1. Candidates had to pass both. The tests were taken three days apart: 67% of candidates passed FLK1; 54% passed FLK2.

The independent assessor said candidates may have become fatigued or had less time to prepare as well for FLK2, suggesting the SRA may want to consider spacing the exams further apart.

Dr Giles Proctor, chief executive of the College of Legal Practice, told the Gazette all training providers will be looking at FLK2 again based on last week’s findings.

Will the SRA space FLK1 and FLK2 apart next time?

Brannan stresses the need to avoid knee-jerk reactions: ‘Next time it could be completely different. One thing I’ve learned in education is you need to take a bit of a long view. You need to monitor very closely – but chopping and changing all the time is not necessarily a good thing in education either.’

However, the SRA will undoubtedly be keen to ensure the results process runs more smoothly next time.

Last week Kaplan had to temporarily close the online portal where candidates can access their results, to deal with ‘technical issues’. The portal reopened at 9.40pm last Thursday. However, eight candidates’ results required ‘further investigation/resolution’ and were not available until Friday morning. Several candidates also saw a name and exam ID number of another candidate. Kaplan says no reportable breach occurred. Nonetheless, the SRA informed the Information Commissioner’s Office.