Intervention to improve interobserver agreement in the assessment of children with pharyngitis

Objective: The objective of this study is to examine the effectiveness of an intervention to improve interobserver reliability of clinical evaluation of children with pharyngitis. Methods: This was a sequential trial performed in an academic children's hospital emergency department. Two sets of 100 children aged 2 to 18 years with a complaint of sore throat were independently evaluated by emergency department attending staff and residents for coryza, scarlatiniform rash, tonsillar enlargement, tonsillar erythema, tonsillar exudates, cervical adenopathy, and palatal petechiae. The first 100 subjects were evaluated according to the clinician's routine practice; the second set was evaluated using a visual-aid sheet showing pictorial gradations of each characteristic. All findings were scored dichotomously and analyzed by κ statistics. Results: Both sets of patients were similar with respect to age, sex, race, and prior episodes of group A β-hemolytic streptococcus. Interobserver reliability without the visual aid was poor in all areas except tonsillar enlargement and tonsillar exudates which had κ values of 0.50 and 0.58, respectively. The use of the visual-aid sheet did not significantly improve reliability in any of the clinical evaluations, and all κ scores remained below 0.50. Conclusion: Interobserver reliability in evaluating clinical signs of pharyngitis is fair to poor. The use of pictorial prompts does not improve interobserver reliability.