Guest: Professor Anthony G Gallagher Host: Patrick Kiely
Episode focus: two landmark studies, the 2004 JAMA carotid stenting paper and the Copernicus arthroscopy trial

Episode summary

Professor Tony Gallagher, the founder of Proficiency-Based Progression (PBP), joins Patrick Kiely to revisit two studies that changed how we think about surgical competence.

The first is the 2004 JAMA paper describing a closed-door meeting at which the US Food and Drug Administration agreed, for the first time, that simulation training should form part of how doctors are approved to perform a procedure. The second is the Copernicus trial in shoulder arthroscopy, which showed that a simulator only improves training when it is paired with validated metrics and a clear proficiency benchmark.

Together they make a simple, evidence-led case: competence should be measured by the skill a clinician can demonstrate, not by years served or cases counted.

Chapters

0:00 Introduction
0:50 Inside the 2004 closed-door FDA meeting on carotid stenting
4:43 Why carotid stenting forced the conversation
7:25 Skill over specialty: ending the turf war
9:44 The FDA precedent: simulation becomes part of credentialing
12:03 Why procedure volume is a crude proxy for competence
14:17 Why the argument had to appear in JAMA
16:22 A homogeneous skill set, devices, and patient safety
20:42 The Copernicus Initiative: a paradigm shift in training
22:53 Three groups, one lesson: a simulator alone is not enough
25:44 The results: 56 per cent fewer errors and the 7.5 times finding
27:53 The trainees who did not pass, and distributed training
30:37 Pass the cognitive exam before the skills lab
32:33 Task deconstruction: 45 steps and 77 possible errors
36:04 Errors versus sentinel errors: why minor errors matter most
39:09 Why fidelity is not the point
41:59 Why a multi-site trial mattered
44:20 Where to start: begin with the metrics

Key points

• Carotid artery stenting is high risk and crossed three specialties, so the FDA needed a way to be sure each clinician was safe to perform it. PBP simulation let credentialing rest on demonstrated skill rather than on specialty or case numbers.
• In 2004 the FDA accepted virtual reality simulation as part of the training package for a new device. This was the first time a regulator tied device approval to a training standard.
• Procedure volume and hours logged are weak indicators of skill. Demonstrated proficiency is a far better one.
• In the Copernicus arthroscopy trial, traditional training performed worst, adding a simulator alone helped only slightly, and PBP plus the simulator produced the strongest and safest performance.
• The PBP group made roughly 56 per cent fewer errors, and residents who met every benchmark were 7.5 times more likely to reach the final standard.
• Minor errors, not only critical ones, predict poor outcomes, so trainees are taught to avoid every avoidable error.
• To build PBP: find people who are genuinely good at the task, define and validate the metrics, choose simulations that let trainees practise the key steps, train faculty on the metrics first, and require a pass on the online didactic before anyone enters the skills lab.

Studies referenced

• Gallagher AG, Cates CU. Approval of virtual reality training for carotid stenting: what this means for procedural-based medicine. JAMA. 2004;292(24):3024-3026. Read on JAMA Network
• Angelo RL, Ryu RKN, Pedowitz RA, et al. A Proficiency-Based Progression Training Curriculum Coupled With a Model Simulator Results in the Acquisition of a Superior Arthroscopic Bankart Skill Set. Arthroscopy. 2015;31(10):1854-1871. Read on the Arthroscopy journal

Connect and follow

• Professor Tony Gallagher on LinkedIn
• Professor Tony Gallagher on Google Scholar