Examinations Update

The below was sent by email to all registered Intensivists and Anaesthetists in Training on Thursday 3 October 2024. If you did not receive an email or there are issues with your registered contact email address, please let us know.

The RCoA and FICM have announced an adjustment we are making to our standard setting practice for the Primary and Final FRCA and FFICM written examinations (MCQ).  

For all exams from February 2025, we will no longer subtract a Standard Error of Measurement (SEM) from the pass mark. This change does not affect the standard or content of the MCQ exams and therefore will not have any impact on how you prepare.  
 

Why we are making this change  

We are making this adjustment in response to the independent review of our assessment processes, which identified the practice as uncommon and recommended that the ‘practice of subtracting a Standard Error of Measurement should cease.’  

This change, which has been approved by the GMC, therefore brings our exams in line with best practice and with other medical royal colleges, ensuring high quality, robust standard setting. We also believe it is fairer for candidates because our data show that the small number who have been given a statistical ‘benefit of the doubt’ by the subtraction of a SEM go on to experience difficulty with subsequent components of the exam, which can have an impact on their time and wellbeing.   
 

What we will do next 

We are putting in place several steps to ensure continued consistency and fairness for candidates as we make this adjustment.  

• We have conducted and submitted to the GMC an equality and diversity impact assessment. We will continue to monitor for any equality and diversity impact on the exams once the change is introduced in February 2025.   

• We will compare data from exam sittings from February 2025 with benchmarks based on the performance of previous candidate cohorts and the different groups of candidates within the overall cohort. This is an approved process for monitoring pass rates and in line with the Academy of Medical Royal Colleges’ Guidance for standard setting.  

• We will continue to use the Angoff method (see the ‘Standard Setting in the FFICM Examination’ section on the FICM website for an explanation) to determine the pass mark for the exam. We will also apply a secondary standard setting method to provide further assurance that pass rates are correct and appropriate. 

We are not able to quantify the specific number of candidates this change will impact, as SEM outcomes are historically different not only between exams but also between diets of the same examination.  

Further information on our marking and standard setting methods is available on the FICM website.    
 

What else is happening following the independent review? 

The recommendations from the independent review informed a development programme for our exams, of which this adjustment is part. Our goal is to ensure our exams have greater authenticity and alignment to the workplace and more closely reflect best practice in postgraduate medical education.  

When we published the review, we committed to implementing changes with care and to consider the needs of candidates. An important part of this has been to give doctors in training a greater role in our assessment processes, with significant representation on our Examinations Development and Assurance Group. We have also increased capacity within our exams team to implement the development work alongside the delivery of our exams.   

Our progress to date includes reviewing and updating the purpose statements for our exams. We are also redesigning the focus of the FRCA and FFICM exams to reduce the assessment burden, provide greater holistic alignment to the programme of assessment and better replicate clinical scenarios across a range of patient groups.  

In the coming months we will continue trials of new question types and later in the academic year, seek help from anaesthetists and intensivists in training to support the piloting process. We will keep you updated on our progress, and we thank examiner leads, trainee representatives and exam committees for their continued collaboration and support through this process. 

If you are undertaking dual training in anaesthetics and intensive care medicine, you may also receive notification of this change from the Royal College of Anaesthetists, and I apologise for any duplication of emails.  

I hope this information is helpful. We are here to support you – please contact exams@rcoa.ac.uk if you have any questions.   

Fiona Daniels   
Head of Examinations