⏱️ Slide Timing: 1 min - Welcome and frame the session: a data paradox that can quietly flip life-or-death decisions - Promise: by the end, everyone will know to ask "what's hiding inside this average?" - Bridge: open with a scenario that feels like an easy, obvious choice

⏱️ Slide Timing: 2 min - Anchor the audience in a relatable, high-stakes decision — nobody picks a surgeon casually - 86% vs 79% looks like a 7-point gap, which feels decisive - Hold the blockquote question — the next slides will overturn the obvious answer ❓ Ask: "Based on these two numbers alone, who would you pick?"

⏱️ Slide Timing: 1 min - Good slide to leave on screen for anyone who wants to rewatch the walkthrough after the session - The TED-Ed video is the shorter pre-read; its hospital example mirrors the surgeon story here - Its closing point sets up the "Split or Combine?" slide near the end of this deck

⏱️ Slide Timing: 3 min - The reveal: Smith is better with the hard cases AND the easy cases - This is the moment that should feel logically impossible — flag it explicitly - Let the contradiction sit for a beat before explaining it on the next slide

⏱️ Slide Timing: 3 min - Same data as the last slide, now side by side — the pink bars (Smith) beat the blue bars (Jones) in both subgroups - Then the rightmost pair flips: Jones's overall bar is taller than Smith's - This single chart is the entire paradox in one image — everything after explains why it happens

⏱️ Slide Timing: 4 min - Dr. Smith is a specialist — 80 of her last 100 patients were high-risk cases - Dr. Jones handles mostly routine surgery — 80 of his last 100 patients were low-risk - The two doctors aren't being compared on the same mix of cases at all

⏱️ Slide Timing: 4 min - Walk through the arithmetic slowly — this is the "aha" slide where the paradox stops feeling magical - Jones's overall number is pulled up by a large, easy caseload, not superior skill - Smith's overall number is pulled down by a large, difficult caseload, not inferior skill

⏱️ Slide Timing: 3 min - Formal definition, now that the audience has already felt the paradox firsthand - Emphasize: this isn't a rare edge case — it's a structural risk in any aggregated comparison - The named phenomenon dates to statistician Edward Simpson's 1951 paper, though the effect was noted earlier by others including Karl Pearson

⏱️ Slide Timing: 3 min - This is the general shape of Simpson's Paradox beyond the surgeon example — it applies to any X-Y relationship, not just success rates - Left panel is what a naive analysis sees; right panel is what stratifying by group reveals — same data, opposite conclusion - The mechanism is the same as the surgeon case: groups differ in their typical X position, and that positioning — not the true within-group relationship — dominates the aggregate

⏱️ Slide Timing: 3 min - This slide is the conceptual core — everything else in the deck is an application of this one idea - A lurking variable doesn't have to be exotic — patient mix, timing, sample size, or geography are common culprits - Rule of thumb: whenever you compare rates across groups, ask what else differs between those groups

⏱️ Slide Timing: 3 min - Direct real-world echo of the surgeon example, now at institutional scale - This is why healthcare quality metrics increasingly report "risk-adjusted mortality," not raw mortality - Ties back to policy: a naive public ranking could steer patients away from the centers best equipped to save them

⏱️ Slide Timing: 3 min - Same structure as the surgeon example, with real data: the lurking variable is stone size - Large stones are harder to treat AND were steered toward open surgery — so open surgery's overall rate is dragged down by its harder caseload - Randomization is the standard defense: if a coin decides who gets which treatment, stone size can't line up with treatment choice - Observational data — patient records, product analytics, most dashboards — has no such protection ❓ Ask: "Where in your own data does someone choose which group a case lands in?"

⏱️ Slide Timing: 4 min - One of the most famous real applications of Simpson's Paradox, published by Bickel, Hammel & O'Connell in Science (1975) - The lurking variable here is choice of department, not applicant sex — competitive departments admit fewer applicants of any gender - Important nuance to state clearly: this doesn't prove there was zero bias anywhere in the system, just that the aggregate number alone was misleading as evidence ❓ Ask: "If you only saw the aggregate 44% vs 35% figures, what would you have concluded?"

⏱️ Slide Timing: 3 min - Age is the most common lurking variable in practice: it drives outcomes and it differs between almost any two populations you might compare - Smoking: the smokers in the sample were younger, so fewer of them died in 20 years — the aggregate made smoking look protective - COVID: early cross-country comparisons of fatality rates were largely comparisons of age structure and testing - The smoking case also appears in the TED-Ed video on the Watch slide

⏱️ Slide Timing: 2 min - Transition slide — moving from "here's the paradox" to "here's what to do about it" - Reconnect explicitly to the surgeon math slide: the weights differed, so the comparison was not apples-to-apples - A/B tests: if traffic splits shift between segments mid-test (e.g. mobile vs desktop), the blended conversion rate can reverse the per-segment result

⏱️ Slide Timing: 3 min - Encourage a habit: whenever a metric compares two groups (doctors, campaigns, cohorts), ask what's unevenly distributed underneath it - The counterweight: hunting through dozens of segments until one flips is its own way to mislead - A split justified by how the data was produced (patient risk, stone size, age) is far more credible than one found by searching

⏱️ Slide Timing: 4 min - This slide stops the audience over-learning "always stratify" — the subgroup view is not automatically the true one - Confounder (split): a common cause of both the grouping and the outcome - Mediator (combine): a step on the path from the thing being compared to the outcome — conditioning on it removes part of the effect you want to measure - This is the same conclusion the TED-Ed video ends on: sometimes the overall number gives the more accurate picture ❓ Ask: "In the surgeon example, could the surgeon have caused the patient's risk level? Why does that matter?"

⏱️ Slide Timing: 2 min - Workflow version of everything covered — a checklist to run before presenting or signing off on any comparative metric - Every item here is a check the surgeon example, by construction, would fail

⏱️ Slide Timing: 1 min - Closing summary before references — the three lines here compress the entire deck - If nothing else survives from this session, this slide should

⏱️ Slide Timing: 1 min - Simpson (1951) is the formal statistical origin, though the pattern was observed earlier by Pearson and Yule - Charig et al. (1986) is the classic kidney-stone treatment case, textbook example of Simpson's Paradox in a clinical trial - Krumholz et al. (2006) underlies the risk-standardized mortality methodology CMS uses for hospital rankings - Bickel, Hammel & O'Connell (1975) is the canonical UC Berkeley case study referenced earlier in the deck - Pearl (2014) gives a modern causal-inference framing of why and when the paradox occurs — the basis of the "Split or Combine?" slide - Appleton et al. (1996) is the Whickham smoking study; von Kügelgen et al. (2021) analyses the Italy vs China COVID-19 reversal