Inter- and intra-rater reliability and accuracy of Sanders Skeletal Maturity Staging System when used by surgeons performing vertebral body tethering

Purpose: Pediatric orthopedic surgeons must accurately assess the skeletal stage of adolescent idiopathic scoliosis (AIS) patients for selection and timing of optimal treatment. Successful treatment using vertebral growth modulation is highly dependent on skeletal growth remaining. We sought to evaluate the current-state use of the Sanders Skeletal Maturity System (SSMS) in regard to precision and accuracy. We hypothesized that pediatric orthopedic surgeons currently use SSMS with moderate precision and accuracy. Methods: Eight practicing pediatric orthopedic surgeons who perform vertebral body tethering surgery without specific training in SSMS were asked to assign the SSMS stage for 34 de-identified hand radiographs from AIS patients. Precision was evaluated as inter-rater reliability, using both Krippendorff’s α and Weighted Cohen’s kappa statistics, and as intra-rater reliability, using only Weighted Cohen’s kappa statistics. Surgeon accuracy was evaluated using Weighted Cohen’s kappa statistics with comparison of surveyed surgeons’ responses to the gold standard rating. Results: Inter-rater reliability across the surveyed surgeons indicated moderate to substantial agreement using both statistical methods (α = 0.766, κ = 0.627) with the majority of discord occurring when assigning SSMS stages 2 through 4. The surveyed surgeons displayed substantial accuracy when compared to the gold standard (κ = 0.627) with the majority of inaccuracy involving the identification of stage 3B. When re-surveyed, the surgeons showed substantial intra-rater reliability (κ = 0.71) with increased inconsistencies when deciding between SSMS stage 3A and stage 3B. Conclusion: The current-state use of SSMS across pediatric orthopedic surgeons for evaluation of AIS patients displays adequate but imperfect precision and accuracy with difficulties delineating SSMS stages 2 through 4, which correlate with adolescent growth periods germane to scoliosis growth modulation surgery. Centralized assessment of hand-bone age may help ensure standardized reporting for non-fusion scoliosis research.