MCA deck oral exams ‒ to fail or not to fail?

The UK’s move to online oral examinations during the pandemic is a good opportunity to introduce transparency and structure into a highly subjective process, says marine consultant and former MCA examiner Ulrich Jurgens

Maritime and Coastguard Agency (MCA) oral exams are a key element of the MCA’s seafarer training and certification structure. Oral exams can trigger anxiety, fear, anger, frustration, exuberance, joy and every feeling in between – just depending on who and where you are.

Preparation for orals is offered by nautical colleges which primarily deliver MCA-audited long courses by professional lecturers. Unregulated private providers make money out of specific oral exam training, which attempts to prepare for particular examiners’ questions who in turn try to protect ‘their’ questions. Candidates sacrifice their souls to fear – and pay.

Once passed, MCA oral exams may give applicants an initial advantage in the job market. Candidates are forced (at least) three times in their career from officer of the watch (OOW) to master to study and update their understanding and knowledge each time before facing an MCA examiner.

The latter experience may be good or bad. Candidates who pass will usually provide positive feedback about the exam. Reports about failures, if feedback is provided at all, are more likely to have a negative tone.

Feedback from candidates

Exam feedback reports from candidates represent a very subjective view by the relevant examinee. The following candidate feedback is illustrative of how candidates can react to an examiner.

Positive reports from recent candidates in 2020/21 mention examiners to be ‘very practical’ or ‘thorough and fair’. The examiner may have been ‘good …, and often quite forceful’. One respondent even reported that it was ‘a pleasure to be examined by X’.

However, there was also some criticism of the process from candidates. Examinees report that ‘questions were not as clear as they could be’. The 105-minute exam was thought to be ‘very long’, leaving the candidate exhausted. The same candidate reported pressure from an examiner who did not allow questions to be written down.

Examiners' reported behaviour was at times questionable. One candidate asked: ‘Why was X so apparently bored, uninterested and unpleasant during the process’, while another examiner suggested the candidate 'fuck off back to yachts'. One candidate (in 2016) suggested his fellow examinees should speak a bit louder because the examiner doesn’t ‘listen to you when he is writing’.

Conflicting feedback also exists that examiners allegedly told candidates not to write a report after the exam or the opposite, encouraging examinees to do so.

Examiners may also make questionable assessments of candidates’ answers, impacting disadvantageously on the candidate.

Clandestine operation

Exams exist to demonstrate meeting the ‘standard of competence’ required by the International Convention on Standards of Training, Certification and Watchkeeping for Seafarers (STCW). However, MCA oral exams are still clouded by a veil of secrecy and may leave the candidate baffled as to why they passed or failed.

An oral exam is a clandestine operation. There is no reliable evidence of what has been asked, what has been answered or what behaviour either the examiner or the candidate displayed. The only witnesses to any exam other than, for example, those with another MCA employee sitting in for training or other purposes, are the candidate and the examiner.

Participants when challenged about alleged errors may remember an alternative truth. Memory is of a fragile nature and, according to the Marine Accident Investigators’ International Forum investigation manual, declines rapidly after 24 hours, following a traumatic event, which failure in an exam can be for some. Consequently, recollection has to be treated with caution.

For example, two candidates who passed their orals in 2020 and 2021 reported ‘I’m sure I’ll miss plenty as it’s all a bit of a blur’, and ‘a pretty accurate report … minus some blurred memory’.

Unsurprisingly, individual recall by examiner and examinee can differ, particularly when days or weeks have passed. Conflicting interests may add to that. A complaining candidate will want to prove unfair treatment and the examiner the opposite.

Hence, without transparency and introduced objectivity it seems impossible that fairness – said to be one of the cardinal requirements in rule-of-law systems – can be verified.

To compare, the UK Quality Code for Higher Education (QAA) defines as one of the core practices of an education provider to use ‘assessment and classification processes that are reliable, fair and transparent’. Demands on the MCA, representing the state, should be even higher.

The emperor’s new clothes

As of 1 November 2021, MIN 653, which outlines the deck oral exam syllabus, replaces MGN 69 – at least until 1 May 2022.

STCW requires that flag states shall issue certificates to candidates who to the satisfaction of the administration meet the requirements for qualification and examinations.

The Convention leaves the type of examination to the discretion of the MCA. There is no legal requirement either in STCW or the relevant UK legislation for an oral exam.

STCW only stipulates persons assessing seafarers to have an appropriate level of knowledge and be qualified for the assessment of competence.

One element of the latter is the act of the MCA examiner. According to MIN 653. ‘the examiner is expected to base the assessment on the competence and relate them to tasks, responsibilities and duties considered necessary for ship operations, safety of life at sea and the protection of the marine environment’.

The MCA does not specify the length of an oral exam which according to candidate’s feedback may reportedly vary between 30 minutes and two hours.

As in MGN 69, measurable criteria to assess a pass or fail do not exist. Criteria provided in MIN 653 (e.g. for masters) for evaluating competence are a literal copy of the relevant part of the STCW Code, which was not specifically written for MCA oral exams.

UK particularities, such as the Code of Safe Working Practices for Merchant Seafarers (COSWOP), which details guidance for health and safety onboard ships, or the harsh UK strict criminal liability system only sneak in through the ‘knowledge’ or ‘further guidance’ column. MIN 653 does not state whether guidance or national legislation form part of the criteria for the single examiner to decide a pass or fail.

The MCA determines that the level of answers shall be enough in the ‘examiner's professional judgment’ that, for example, the OOW candidate can carry out their watchkeeping and operational duties and responsibilities or, for masters, serve in that capacity.

To be reliable, fair and transparent ought to be the trademark of a successful exam system

Professional judgment

‘Professional judgment’ is not defined in MIN 653 and leaves room for interpretation. English courts have dealt with the term on many occasions.

In one case, for example, the professional judgment of a ship’s master was based upon a high degree of skill and many years of experience. A medical doctor’s professional judgment meant drawing upon medical knowledge and experience. Another master of a cargo ship was only allowed a professional judgment within the ‘navigational area’, as opposed to the right for a decision ‘to economise on bunkers for no good maritime reason’.

The decisions suggest that professional judgment requires skill, experience and knowledge.

But applying professional judgment also accepts that the decision will be unavoidably subjective. The exam’s questions and answers are always limited and never known to anybody outside the exam room. The professional judgment of examiners may also vary. Different examiners have different experience, knowledge and skills. Professional judgment is no guarantee for error-free decisions.

In the recent Colreg (Convention on the International Regulations for Preventing Collisions at Sea) case about the application of Rule 15 in a narrow channel (see Nautilus Telegraph, May 2021), for example, the UK Supreme Court overturned decisions by both the Admiralty Court and the Court of Appeal. It can hardly be said that the latter courts were staffed with judges who did neither have nor apply their professional judgment. Their decisions were simply considered wrong by the Supreme Court.

According to the MCA, it currently has 44 nautical examiners. Six are employed at MCA headquarters and hold a UK Master’s Unlimited Certificate of Competence (COC). COCs held by the other examiners are not known as the MCA states that ‘we do not hold this information as definitive list [sic]’. The MCA also holds no data about whether its examiners have sailed as master on voyages which are ‘not short international voyages’.

Oral exams, however, can be undertaken by any examiner trained within the MCA to the respective level, i.e. for OOW they must witness four and perform four supervised exams, plus two each for chief mates’ (CM) exams and another three each for master.

That suggests the MCA lacks knowledge about the extent to which, and the area in which, its examiners can apply professional judgment. Unless, of course, one would say that the MCA training to become an examiner will provide the required skill, experience and knowledge. Experience as a master, however, can hardly be achieved by conducting oral exams, which is the final hurdle for a candidate’s career progression.

Examiners have the power to make candidates resit the exam for currently £220 several months after a failed exam or, worse even, set examinees free to become a danger to the seafaring industry.

To fail or not to fail

As an MCA examiner for approximately 12 years, I failed candidates. I probably made errors. I passed candidates. I probably made errors.

Would I ever have been able to decide whether a candidate was to make a competent OOW, CM or master? Hardly. I could only judge whether the answer given was in my assessment right or wrong.

When asked how an examiner decides whether to pass or fail the MCA simply repeated the wording provided in MIN 653. That appears to be a very long way from how the QAA wants assessments marked – and, of course, the MCA examines for a ship operating licence and not for a degree.

The QAA requires that ‘there is clear guidance about how borderline marks or grades are defined and treated, both in individual assessments and in overall results for a module or a programme’.

The MCA examiner sets the borderline for an oral test. Although, according to the MCA, ‘there is no leeway for safety critical questions such as Colregs, IALA Buoyage systems, emergencies, or enclosed spaces. Non-safety, non-Colregs areas are more often an assessment of whether the candidate knows enough for their grade’. But the MCA does neither define where ‘safety critical’ begins nor what knowledge is enough.

Missing evidence

A ship must be detained when it is not equipped with a functioning voyage data recorder (VDR).

To have a functioning VDR onboard is one of the many requirements a master has to ensure compliance with, in addition to hundreds of strict criminalisation opportunities under UK Merchant Shipping Regulations.

Strict criminal liability turns the presumption of innocence on its head. However, the system at least requires evidence that the unlawful event occurred.

MCA surveyors/inspectors are the enforcers of the system. Collecting evidence through photos and interviews including using a VDR lies within their powers. Without evidence there is no enforcement. During a port State control (PSC) inspection, for example, the inspector must first find evidence of a defect (‘clear grounds’) before professional judgment can be applied. That judgment can afterwards be legally challenged.

That does not apply for oral exams. Evidence of the conduct or contents of the exam is missing. Clear grounds only exist in the mind of the examiner. Each of them applies their personal system to decide their clear grounds for a pass or fail.

Thus, one of the principles of natural justice that anyone who has an adverse decision made against them by a state official should have a right to be heard, is completely undermined.

The end of the pandemic orals

Due to the coronavirus pandemic MCA oral exams are currently conducted online. They were thankfully not abandoned.

This offers the golden opportunity of introducing objective evidence. Recorded exams – online or in-person – would not leave room for speculation as to what was said. Whether any judgment was professional or not would become a question of objectivity – if an exam outcome was ever questioned.

To be reliable, fair and transparent ought to be the trademark of a successful exam system delivering competent and trusted mariners to an industry fraught with high operational and legal risks.