FFICM Examination Report October 2020

This FFICM oral examination was rather different, as it was the first sitting of the remote oral exam, and took place when London was in Tier 2 COVID restrictions and the College building was mostly closed. The Spring FFICM oral examination and July MCQ had been cancelled due to COVID restrictions, and a lot of effort went into devising and testing a format that would be able to take place in uncertain circumstances. All candidates and examiners were in their homes / workplaces with only the exams administration staff, the Chair and Deputy actually (socially distanced) in the College building. A huge amount of work was done in advance by the exams team and examiners, to convert all the questions to a suitable format and set up the remote system. Many hours of testing and training were carried out, including 7 half-days of examiner training. As a result, the first sitting ran successfully and all planned candidates were examined. The ‘Zoom’ platform was chosen for this exam, as it was the most suitable for this exam format.

For candidates, examiners and exams staff this was a new experience. Some candidates were happy not to have to make a long expensive journey to London, and sitting an exam in the familiar environment of home/work etc was less stressful, while others found it strange and off-putting. 

The remote exams took longer than the face-to-face version even though the actual examining time was the same. This was due to additional time being needed for candidates to read station instructions, to ‘move’ candidates between stations, for an environment security check for each candidate and an extra allowance to compensate for any internet issues. For this reason, the 14 station OSCE was split into two halves to allow a break between. The exam ran over 4 days (instead of the usual two days), with 1 more day allocated in case of repeat exams being needed.

In order to assist candidates with their exam preparation, a free webinar was held at the beginning of August, along with guidance and FAQs published on the FICM website. These resources were supplemented in September with additional guidance, a presentation of example OSCE artefacts, a video of two remote OSCE stations and remote exam regulations.

Some technical and internet issues were to be expected. A number of mitigations were put into the planning of the remote exams. These were: allowing extra time on each station to be used in case of technical issues, time to run a station again at the end of the exam,   the ability to put a candidate in for a completely new exam on a different day (if needed) and all exams were videoed. Extra examiners were available and prepared, so as to be able to substitute at short notice should an examiner lose their internet connection.

Inevitably there were a number of brief internet ‘drop outs’ (almost all due to candidates’ internet connections) and each day a number of examiners utilised the extra 1 minute at the end of the station to compensate for technical issues, allowing all candidates to have the full 7 minutes on each   question.  

Numerous testing and training sessions for all the examiners using the remote format were carried out, so there were very few examiner internet and technical problems. A small number of examiners had an internet problem during the exam, such as a few seconds of ‘drop out’. One SOE examiner (2% of examiners) lost connection completely in one station and the other stepped in immediately. Whenever an examiner had any sort of internet problem, a spare was ‘sent’ into their station, in order to substitute in if needed. There were three occasions (0.02% of stations) where a technical issue occurred eg a slide failed to display or a candidate received less than the expected time for a question. In all these cases the videos were reviewed and appropriate extra marks were awarded to the candidate to compensate.

However, 15% of exams had significant candidate internet issues such as being disconnected then reconnected during the exam. Some of them needed to change to using their 4G phone signal (often an administrator or spare examiner would need to talk them through how to do this) to improve internet connectivity. Some candidates disconnected more than once during an exam. One had a laptop failure mid exam and two dialled into the wrong ‘meeting’ so appeared to be absent at the start of the exam. The fourth day of the exam had a particularly large number of candidate internet issues. Rather surprisingly, quite a number of candidates ‘dialled in’ to the exam meeting 5-10 minutes later than the expected time, which delayed the start of some exams.

Several additional strategies were implemented and considerable effort was made to ensure all candidates did complete their exam, such as pausing the exam briefly and adding additional stations at the end, however, this additional time for internet problems did lead to other candidates having longer gaps between some stations, and extended the duration of some exams. On some days the afternoon SOE was delayed to allow candidates to have a suitable lunch break between sessions. All candidates did complete their exams. The four days of examining included 1417 OSCE ‘stations’ and 376 SOE ‘tables’.

The examiners’ workload was much greater than usual, with examiners needing to download and print questions, score sheets and PowerPoint presentations of artefacts in advance. They then had to  complete score tables, upload score sheets, videos and report forms after each exam. 

The task of uploading, checking and collating marks in order to establish who had passed was much larger in the remote format, and all manual on this occasion. This led to the pass list being slightly delayed, and it was issued one week after the last day of examining.

There were no visitors on this occasion, and no examiner appraisals, due to the complexity of managing the remote format.

All the questions were from the usual exam question banks, some with adaptations for the remote format. These questions worked well over the remote format, and it was felt that the breadth of curriculum covered and standard of the exam were the same as the face-to-face version.