『Show Me The Evidence』のカバーアート

Show Me The Evidence

Show Me The Evidence

著者： Anthony G. Gallagher Flux Learning Ltd
無料で聴く

Most training is sold on confidence. Show Me The Evidence is built on data. In every episode we take a single study, clinical trial, or systematic review and work through what it found, how it was designed, and what it means for the way we teach and assess skill. We focus on metrics-based training and proficiency-based progression, the approach that asks learners to demonstrate measurable competence before moving on, and we trace its results across surgical, medical, and professional education. This is a podcast for learning professionals and medical educators who want more than opinion. Expect plain-language breakdowns of the research, honest discussion of what the evidence does and does not support, and conversations with the people behind the studies. If you make decisions about how people are trained, we think you deserve to see the evidence first.2026 OGC Metrics and Anthony G Gallahger. 社会科学 科学
エピソード
  • Dr. Richard Angelo: From Apprenticeship to Proficiency — Rethinking How We Train Surgeons

    Dr. Richard Angelo: From Apprenticeship to Proficiency — Rethinking How We Train Surgeons
    2026/05/29
    Episode 4 — Dr. Richard AngeloGuestDr. Richard (Rick) Angelo — Arthroscopic surgeon based in Seattle; former President of the Arthroscopic Association of North America (AANA). Holds a PhD in proficiency-based progression training.HostTony (relationship with Rick spans ~15 years, originating from a chance meeting at a conference in Sweden)Episode OverviewA deep-dive conversation on the fundamental failures of traditional surgical training and how proficiency-based progression (PBP) training offers a scientifically rigorous alternative. The discussion centres on the landmark Copernicus Study — the first study in medicine to use proficiency demonstration as an outcome measure.Key Topics Covered1. Limitations of the Traditional Apprenticeship ModelThe "see one, do one, teach one" model lacks objective assessmentDespite decades of training and significant investment, AANA could not verify whether skill acquisition was actually occurringComplication rates and suboptimal outcomes weren't improving with existing training efforts2. The Founding QuestionRick, during his time in the AANA presidential line, asked: "Is there a better way to train surgical skills?"This led to engagement with Tony's work on proficiency-based progression training3. Proficiency-Based Progression (PBP) Training — Core PrinciplesDefine a clear target: what does quality performance of a procedure look like?Deconstruct tasks into discrete, trainable componentsDevelop objective, binary metrics (did it occur or not?) rather than global rating scalesEstablish inter-rater reliability between assessorsTrainees must demonstrate a benchmark at each stage before progressing (including cognitive pre-course material — 83% threshold)Errors and deviations from optimal performance are trained explicitly — not just steps4. The Bankart Repair — Why It Was ChosenCommon procedure with a broad, transferable skill setSuited to task deconstruction and partial task simulationChosen by Rick and endorsed by the AANA core group5. Curriculum Before SimulationA critical insight: the curriculum and metrics must be developed first; simulation is chosen to match, not the other way aroundContrast with the wider medical field's focus on "eye candy" VR simulators that lack meaningful metricsThe FAST model (Fundamentals of Arthroscopic Surgery Training) was developed with Rob Pedowitz for knot tying — a low-cost, highly accurate partial task trainerEven a simple conical nail punch from a garage became an effective tool for measuring loop elongation6. The Copernicus Study — Design & ResultsThree study groups:Group A (Traditional): Lectures, open-access knot-tying lab, cadaver session — standard AANA approachGroup B (Simulator only): Access to the simulator without the PBP curriculum or metricsGroup C (PBP): Proficiency benchmarks at every stage — cognitive, knot-tying, and shoulder modelResults:Group B was 1.4× more likely than Group A to meet the benchmark (marginal)Group C participants (assigned to PBP, even without passing all benchmarks): 5.5× more likely than Group AGroup C participants who met all proficiency benchmarks: 7.5× more likely to meet the final benchmarkError reduction: ~56% decrease in Bankart errors; ~58% for rotator cuff repairIn one follow-up weekend cohort of 18 trainees: 89% demonstrated proficiency in Bankart repair; 83% in rotator cuff repair7. Key Finding: The Deficiency is in Training, Not TraineesPre-study concern about a "weed-out process" proved unfoundedWith quality training, almost all trainees can master the required skillsReferenced Frank Lewis (former Chair, American Board of Surgery) sharing the same observationStefano Pogliani's study demonstrated near-universal proficiency is achievable8. The Role of Errors in Surgical TrainingDistinguishing novice from expert performers is best predicted by error enactment, not step completionEach deviation from optimal performance creates a cascade risk — even if consequences aren't immediateUpcoming study expected to show errors are the best predictor of patient outcomes9. Broader Applicability to Procedure-Based MedicinePrinciples apply across disciplines — cardiology, robotics, and beyondContrast drawn with VR simulator manufacturers at the European Heart Rhythm Association Conference (Paris), where most simulations had no metricsChicken tissue models used successfully in robotic surgery training at €5 per chicken — effective without being high-tech10. Credentialing and Quality AssuranceDiscussion of whether PBP methodology could or should underpin credentialing for new procedures or devicesDevice failures in the field often attributable to inadequate clinician preparation, not device defectsPractical challenges for societal credentialing (procedure selection, remediation pathways, cost of metric development, legal defensibility)European Commission is moving toward micro-credentials for technical skills — awarded by universities, recognised across EU member ...
    続きを読む 一部表示
    48 分
  • From the FDA to the operating theatre: how proficiency rewrote the rules of surgical training

    From the FDA to the operating theatre: how proficiency rewrote the rules of surgical training
    2026/05/25


    Guest: Professor Anthony G Gallagher Host: Patrick Kiely
    Episode focus: two landmark studies, the 2004 JAMA carotid stenting paper and the Copernicus arthroscopy trial


    Episode summary

    Professor Tony Gallagher, the founder of Proficiency-Based Progression (PBP), joins Patrick Kiely to revisit two studies that changed how we think about surgical competence.

    The first is the 2004 JAMA paper describing a closed-door meeting at which the US Food and Drug Administration agreed, for the first time, that simulation training should form part of how doctors are approved to perform a procedure. The second is the Copernicus trial in shoulder arthroscopy, which showed that a simulator only improves training when it is paired with validated metrics and a clear proficiency benchmark.

    Together they make a simple, evidence-led case: competence should be measured by the skill a clinician can demonstrate, not by years served or cases counted.


    Chapters

    0:00 Introduction
    0:50 Inside the 2004 closed-door FDA meeting on carotid stenting
    4:43 Why carotid stenting forced the conversation
    7:25 Skill over specialty: ending the turf war
    9:44 The FDA precedent: simulation becomes part of credentialing
    12:03 Why procedure volume is a crude proxy for competence
    14:17 Why the argument had to appear in JAMA
    16:22 A homogeneous skill set, devices, and patient safety
    20:42 The Copernicus Initiative: a paradigm shift in training
    22:53 Three groups, one lesson: a simulator alone is not enough
    25:44 The results: 56 per cent fewer errors and the 7.5 times finding
    27:53 The trainees who did not pass, and distributed training
    30:37 Pass the cognitive exam before the skills lab
    32:33 Task deconstruction: 45 steps and 77 possible errors
    36:04 Errors versus sentinel errors: why minor errors matter most
    39:09 Why fidelity is not the point
    41:59 Why a multi-site trial mattered
    44:20 Where to start: begin with the metrics


    Key points

    • Carotid artery stenting is high risk and crossed three specialties, so the FDA needed a way to be sure each clinician was safe to perform it. PBP simulation let credentialing rest on demonstrated skill rather than on specialty or case numbers.
    • In 2004 the FDA accepted virtual reality simulation as part of the training package for a new device. This was the first time a regulator tied device approval to a training standard.
    • Procedure volume and hours logged are weak indicators of skill. Demonstrated proficiency is a far better one.
    • In the Copernicus arthroscopy trial, traditional training performed worst, adding a simulator alone helped only slightly, and PBP plus the simulator produced the strongest and safest performance.
    • The PBP group made roughly 56 per cent fewer errors, and residents who met every benchmark were 7.5 times more likely to reach the final standard.
    • Minor errors, not only critical ones, predict poor outcomes, so trainees are taught to avoid every avoidable error.
    • To build PBP: find people who are genuinely good at the task, define and validate the metrics, choose simulations that let trainees practise the key steps, train faculty on the metrics first, and require a pass on the online didactic before anyone enters the skills lab.


    Studies referenced

    • Gallagher AG, Cates CU. Approval of virtual reality training for carotid stenting: what this means for procedural-based medicine. JAMA. 2004;292(24):3024-3026. Read on JAMA Network
    • Angelo RL, Ryu RKN, Pedowitz RA, et al. A Proficiency-Based Progression Training Curriculum Coupled With a Model Simulator Results in the Acquisition of a Superior Arthroscopic Bankart Skill Set. Arthroscopy. 2015;31(10):1854-1871. Read on the Arthroscopy journal


    Connect and follow

    • Professor Tony Gallagher on LinkedIn
    • Professor Tony Gallagher on Google Scholar
    続きを読む 一部表示
    46 分
  • The VR-OR Study — Proof That Simulation Training Transfers to the Operating Room & The Methodology of Proficiency-Based Progression

    The VR-OR Study — Proof That Simulation Training Transfers to the Operating Room & The Methodology of Proficiency-Based Progression
    2026/05/25
    Guest: Professor Anthony G GallagherTopic: The VR-OR Study — Proof That Simulation Training Transfers to the Operating Room & The Methodology of Proficiency-Based ProgressionEpisode SummaryIn this episode, Patrick Kiely sits down with Professor Tony Gallagher to examine two landmark papers that transformed simulation-based surgical training. The first — the 2002 Yale VR-OR study — provided the first prospective randomised blinded proof that virtual reality simulator training transfers directly to improved operating room performance. The second — a 2005 Annals of Surgery paper — provided the field with the recipe for how to actually implement it. Together, they form the scientific and methodological backbone of Proficiency-Based Progression. Tony explains why the design decisions that made these studies credible — blinding, objective metrics, proficiency benchmarks, construct validity — are the same decisions most training programs still fail to make today.Key Topics Covered1. The Problem VR Training Was Designed to Solve — 0:00The apprenticeship model and why laparoscopic surgery broke itThe fundamental cognitive challenge of moving from direct vision to a monitorThe fulcrum effect: why instrument manipulation on a monitor creates a proprioceptive conflict the brain must automateRick Satava's proposal: acquire basic skills outside the OR, on simulators2. The Simulator That Changed Things — 3:21Johnson & Johnson's Ethicon simulator: an emulator, not a physics-based modelWhy abstract psychomotor tasks work better than tissue simulationThe surgical community's scepticism — and why Yale provided the opportunity to test it properly3. The Proficiency Benchmark: How It Was Set — 4:51Rejecting time and trial number as training endpointsUsing objectively assessed performance of experienced (not world-class) surgeons as the benchmarkMean vs. median performance, and how to handle outlier experts (>2 SD from mean are excluded)Frank Lewis (American Board of Surgery) on why the benchmark is deliberately high — and why that's fine4. The Results: What Happened in the OR — 6:57VR-trained residents: six times fewer errors in the ORControl group: nine times more likely to fail to progress during a procedure5. Failure to Progress: What It Reveals — 7:23Defining the metric: instruments moving but the procedure not advancingWhy it indicates the person was not ready to perform the task independentlyHow it predicted the need for online didactic preparation before the skills lab6. Why the Study Had to Be Prospective, Randomised, and Blinded — 13:11The gold standard language clinicians understandWhy senior figures in surgery said it wasn't doable — and why they were wrongHow double-blinding protected the integrity of intraoperative assessmentThe study design that subsequently became the default methodology for evaluating simulation tools in medicine7. Objective Metrics vs. Likert Scales — 15:22Why Likert scales fail for technical skill assessmentInter-rater reliability below .8 invalidates any assessment tool by defaultThe subjectivity problem: two surgeons from the same year, same school, scoring the same video differentlyWhy errors are the most sensitive measure of change as a result of trainingSteps vs. errors: trainees learn what to do; what they don't learn systematically is what not to do8. The 2005 Annals Paper: The Recipe for PBP — 27:33Why the VR-OR paper alone wasn't enough — Randy Halleck: "You assume we know how to use the methodology"What the 2005 paper added: how to develop metrics, who to involve, how to set the benchmark, how to validateThe core principles of PBP that remain unchanged todayPublication: Gallagher, A.G. & Seymour, N.E. (2002). Virtual reality training for laparoscopic surgery. Annals of Surgery, October 2002.https://journals.lww.com/annalsofsurgery/abstract/2002/10000/virtual_reality_training_improves_operating_room.8.aspx9. Education vs. Training: Why the Distinction Matters — 29:05Education = knowledge transmission; Training = skill acquisitionWhy medicine has done excellent education for centuries but apprenticeship-based training no longer fits the 21st centuryThe online didactic benchmark: trainees don't enter the skills lab until they've demonstrated knowledge to the level of experienced practitionersWhat this saves in skills lab time — and what it tells supervisors about where to direct help10. The Pre-Trained Novice and Attentional Capacity — 31:31Chunking: how the brain compresses discrete information units into automated sequencesWhy unautomated technical skills consume attentional capacity that should be available for situational awarenessThe bicycle analogy: looking at the handlebars vs. seeing the potholeWhy automation must occur outside the OR — stress in the operating room compounds cognitive load11. Case Volume as a Surrogate for Skill — 37:04Why procedure numbers are a weak and noisy predictor of surgical competenceThe Berkmar study:...
    続きを読む 一部表示
    54 分
adbl_web_anon_alc_button_suppression_t1
まだレビューはありません