Potential for Economic Inefficiencies in Human-AI Hybrid Screening: An Analysis of the Sybil AI on NLST Data

Abstract

Background: Lung cancer screening using low-dose chest CT (LDCT) can be enhanced by artificial intelligence algorithms such as Sybil. While autonomous AI models demonstrate superior predictive accuracy and cost-effectiveness compared to conventional radiologist evaluations, attempts to heuristically modify or augment these algorithms with non-complementary human methodologies can yield unintended economic and clinical consequences. Methods: The study incorporated a cost-effectiveness analysis utilizing an unpublished foundational dataset encompassing 11,424 participants from the National Lung Screening Trial (NLST), of whom 99 were diagnosed with lung cancer. The study evaluated three diagnostic workflows: Baseline (radiologist as sole reader), Sybil Cost-Saving (Sybil first reader; radiologist reviews positives), and Sybil Safeguard/Hybrid (Sybil first reader; radiologist reviews negatives). Performance metrics and total medical costs, based on the 2024 Medicare fee schedule, were compared. Findings: The Sybil Cost-Saving Strategy reduced average per-patient costs from $421 to $324, realizing a total cohort savings of $1,110,594. It identified seven additional true positives and 13 fewer false positives than the Baseline. Conversely, the Sybil Safeguard/Hybrid Strategy triggered a substantial surge of 561 false positives. This near-doubling of false positives erased the cost savings, increasing the overall total medical cost by $1,211,086 over the Sybil Cost-Saving strategy (an incremental cost of $173,012 per additionally identified true positive). Conclusions: Modifying autonomous AI algorithms through piecemeal human adjudication subverts economic efficiency and severely inflates false-positive rates due to divergent human-machine evaluation criteria. Policymakers and clinicians must carefully weigh the marginal sensitivity benefits against the exponential surge in false positives and profound opportunity costs.