At a Glance

  • Session: Plenary Lecture 2 (around 09:52-10:30)
  • Slide title: The End of Average Medicine: How AI Identifies the Right Treatment for Every Patient
  • Speaker: Professor and Chair, Department of Biostatistics, Vanderbilt University Medical Center; Harold L. Moses Chair in Cancer Research; voting member of an FDA advisory committee (self-described)
  • Note on these notes: the tail end of the talk (the second half of the cfDNA case and the conclusion) wasn't fully captured live, so that part is filled in from the slides only

Outline and the Five-Year Outlook

The talk moved through four themes: AI for smart biomedical research, AI for pragmatic clinical trials and real-world data, individualized treatment effects and machine learning, and then the conclusion.

Smart precision medicine rests on three things: computing power (from GPUs to TPUs to NPUs, including edge computing), new analytical tools (adaptive multimodal models), and high-quality data. Her vision: 'To achieve true AGI our models must be adaptive multimodal... they will all combine into a unified intelligence.'

She offered an analogy: when the data changes, the decision should change with it. Deciding whether to bring an umbrella shouldn't be based on yesterday's forecast - you look out the window today. That's the difference between a static model and one that adapts to real-time data.

Patient Heterogeneity and Pragmatic Trials

Patients differ along many dimensions at once - genetics, clinical status, lifestyle, environment - so we need statistical methods flexible enough to handle that complexity.

An efficacy (explanatory) trial asks whether a treatment can work under ideal conditions. An effectiveness (pragmatic) trial asks whether it does work in the real world (citing Ford and Norrie, 'Pragmatic Trials,' NEJM 2016;375:454-63).

Pragmatic trials have loose inclusion criteria, track patient-centered practical outcomes, are embedded in routine care, allow flexible interventions, and pull outcomes straight from EHR, claims, or registry data. A cluster-level design can even waive individual informed consent and traditional case report forms, cutting costs substantially.

Case Study: ICU Oxygen Targets

Act One: The PILOT Trial (a Negative Result)

About 2,500 mechanically ventilated patients were enrolled, with entire ICUs cluster-randomized and crossed over between SpO2 targets of 90%, 94%, and 98% (ranges 88-92, 92-96, and 96-100). The primary endpoint, ventilator-free days, came out almost identical across the three groups (about 20, 21, and 21 days, p is about 0.8). (Semler MW et al., NEJM 2022;387:1759-69)

Act Two: Machine Learning Overturns the Verdict (JAMA 2024)

The team went back to that same data and estimated individualized treatment effects instead. They tested six machine learning algorithms with k-fold cross-validation and selected the best one using the Qini statistic (an XGBoost-based model won), then validated externally against the Australia-New Zealand ICU-ROX trial. The result: some patients actually benefited from the high target, others from the low target - and those opposite effects canceled out into what looked like 'no difference' on average. (Buell KG et al., JAMA 2024;331(14):1195-1204)

Act Three: Back Into the Clinic (Vanderbilt)

The model is now built into Vanderbilt's EHR system (Epic). It tracks patient variables day by day, and once they cross a threshold it prompts the physician to adjust the oxygen target - high, medium, or low. Early observations suggest patients whose care followed the model's suggestion did better, though she noted that, as the study's designer, she may be biased, and the paper is forthcoming.

The Core Method: The Qini Statistic

Definition from the slides: if you use a model to target the people most likely to benefit, how much more outcome do you gain compared to targeting people at random? That's the Qini statistic - a performance metric for uplift modeling used across marketing, policy evaluation, and precision medicine. The point is finding who benefits most, not predicting risk.

She gave an e-commerce analogy: if your budget only covers 100 phone calls, you want to pick the customers who will actually change their decision because of the call. The original Qini formulation even includes a penalty - some customers who would have bought anyway end up not buying because you called them. Applied to medicine, this becomes the extra benefit a new treatment gives this particular patient over the standard treatment, and cost can be folded into that individualized treatment effect too. (Qini reference)

Closing Case (Reconstructed From Slides)

A prenatal cfDNA screening example: in a routine first-trimester screening population, the Harmony test analyzes only specific chromosomal fragments rather than whole chromosomes, and does so with high throughput and accuracy. This corresponds to the NEXT study (Norton ME et al., NEJM 2015;372:1589-97), which found a 100% detection rate for trisomy 21 versus 78.9% for traditional screening.

Quote

The end of average medicine - a zero average treatment effect doesn't mean the treatment had no effect for anyone.