Development and Evaluation of American College of Surgeons NSQIP Pediatric On-Demand Reporting of Risk-Adjusted and Smoothed Rates

BACKGROUND: The NSQIP Pediatric Semiannual Report (NSQIP Ped SAR) provides hospitals with risk-adjusted benchmarked results for comparative performance based on 1 year of data. These data are 6 to 18 months old due to requirements for data processing and modeling, and this delay potentially limits their usefulness for hospital surgical quality improvement efforts. A timelier reporting mechanism is needed. STUDY DESIGN: The "On-Demand" methodology uses 5 years of historical data to construct logistic model prediction equations for outcomes, which support the construction of smoothed and risk-adjusted benchmarking reports in real time. In addition, custom filters allow users to build models for time periods and unique patient subgroups of interest, allowing comparisons of hospital performance across different time periods to a fixed population outcome rate. A rate-based metric (risk-adjusted rates) along with CIs graphically describe hospital performance.SAR and On-Demand statistical benchmarking methodologies were compared for mortality, morbidity, and surgical site infection across all surgeries, gastrointestinal surgeries, and general surgeries, for patients with operation dates from January 1, 2022, to December 31, 2022. Differences in outlier detections and logged odds ratios from benchmarked results were evaluated using weighted kappa statistics and Pearson correlation coefficients. RESULTS: On-Demand predictive equations were built using data from January 1, 2018, to December 31, 2022. Comparisons of benchmarking results from the SAR and On-Demand from January 1, 2022, to December 31, 2022, demonstrated near-perfect agreement; weighted kappa statistics (excluding mortality) averaged 0.865, and correlations averaged 0.978. CONCLUSIONS: The newly introduced NSQIP Ped On-Demand application will augment SAR reporting, providing earlier information, on a model-by-model basis, for assessing hospital performance to improve surgical outcomes at NSQIP Ped hospitals.