| |
An Operative Performance Rating System for Urology Residents
Aaron Benson, M.D.,1 Stephen Markwell, M.A.,2 Tobias Köhler, M.D., M.P.H.,1 Thomas H. Tarter, M.D., Ph.D.3
Purpose
An operative performance rating system (OPRS) for Urology residents was developed
for 6 sentinel urological procedures. This study tests the reliability, validity, and
feasibility of an OPRS for Urology residents.
Materials and Methods
Each procedure OPRS contained a 3-point case difficulty scale, 4 to 6 procedure specific
items, 3 general items, and an overall performance item. A Likert scale of 1 to
5 was used for each item. A single video/audio record of each procedure was
evaluated by the faculty, and single-item inter-rater agreement was measured by
comparison of the observed variance and random measurement-error variance.
Resident operative performance evaluations were completed on-line. Internal
consistency reliability was measured using Cronbach's alpha. Overall scale scores by
PGY level were compared using one way analysis of variance (ANOVA).
Results
Faculty evaluation of video/audio records showed an inter-rater agreement range of
0.71 to 0.92. Faculty evaluations of resident operative performance showed an
internal consistency reliability range of 0.91 to 0.95. Significant differences in overall
scale scores between PGY levels were demonstrated for 3 of the 6 procedures (p <
0.0016).
Conclusions
An OPRS for Urology residents is feasible using an internet based resident
management system. Inter-rater agreement and internal consistency reliability meet
threshold limits for checklist evaluation instruments. The OPRS can discriminate PGY
level of resident training. A validated OPRS can offer residents immediate and
objective feedback on surgical performance, and enable program directors to monitor
progress in resident operative performance.
1Department of Surgery, Division of Urology Southern Illinois University School of Medicine, Springfield, IL
2Department of Surgery, Southern Illinois University School of Medicine, Springfield, IL
3Cancer Care Specialists of Central Illinois, S.C., Decatur, IL
Sentinel Procedure OPRS Forms
(The manuscript for this study has been accepted for publication in The Journal
of Urology with anticipated publication date of November 2012. Above are links
to example OPRS forms developed for each of the six sentinel procedures.)
|
