Journal of Surgical Education
Volume 66, Issue 5 , Pages 255-263 , September 2009

Continuous, Data-Rich Appraisal of Surgical Trainees' Operative Abilities: A Novel Approach for Measuring Performance and Providing Feedback

  • Paul B. Roach, MD

      Affiliations

    • Department of Surgery, Naval Medical Center Portsmouth, Portsmouth, Virginia
    • ,
  • Kevin K. Roggin, MD

      Affiliations

    • Department of Surgery, the University of Chicago, Chicago, Illinois
    • ,
  • Gene Selkov Jr, MD

      Affiliations

    • Department of Surgery, the University of Chicago, Chicago, Illinois
    • ,
  • Mitchell C. Posner, MD

      Affiliations

    • Department of Surgery, the University of Chicago, Chicago, Illinois
    • ,
  • Jonathan C. Silverstein, MD

      Affiliations

    • Department of Surgery, the University of Chicago, Chicago, Illinois
    • Corresponding Author InformationCorrespondence: Inquiries to Jonathan C. Silverstein, MD, MS, Computation Institute, 5735 S. Ellis Avenue, Chicago, IL 60637; fax: (773) 834-8400

References 

  1. Debas HT, Bass BL, Brennan MF, et al. American Surgical Association Blue Ribbon Committee Report on Surgical Education: 2004. Ann Surg. 2005;241:1–8
  2. Pellegrini CA, Warshaw AL, Debas HT. Residency training in surgery in the 21st century: A new paradigm. Surgery. 2004;136:953–965
  3. Bass BL. Fundamental changes in general surgery residency training. Am Surg. 2007;73:109–113
  4. Thomas WE. Teaching and assessing surgical competence. Ann R Coll Surg Engl. 2006;88:429–432
  5. Brennan MF, Debas HT. Surgical education in the United States: Portents for change. Ann Surg. 2004;240:565–572
  6. Request for proposal for a comprehensive literature review and analysis of residency training and duty hours experience. http://www.acgme.org/acWebsite/home/acgme_nascaletter_RFP.pdf
  7. Mestres CA, Revuelta JM, Yankah AC. The European Working Time Directive: quo vadis? (A well-planned and organized assassination of surgery). Eur J Cardio Thorac Surg. 2006;30:571–573
  8. Kelly A, Canter R. A new curriculum for surgical training within the UK: context and model. J Surg Educ. 2007;64:10–19
  9. Canter R, Kelly A. A new curriculum for surgical training within the UK: the first stages of implementation. J Surg Educ. 2007;64:20–26
  10. Bell RH. Surgical council on resident education: a new organization devoted to graduate surgical education. J Am Coll Surg. 2007;204:341–346
  11. ACGME. Accreditation Council for Graduate Medical Education (ACGME) General competencies. In: ACGME; September 28, 1999; Chicago, IL; 1999.
  12. Delaney PV, Hennessy TP, Quill RD, et al. Assessment of operative surgical skills. Ir Med J. 1978;71:438–441
  13. Sidhu RS, Grober ED, Musselman LJ, et al. Assessing competency in surgery: where to begin?. Surgery. 2004;135:6–20
  14. Sachdeva AK. Invited commentary: educational interventions to address the core competencies in surgery. Surgery. 2004;135:43–47
  15. Aggarwal R, Grantcharov TP, Darzi A. Framework for systematic training and assessment of technical skills. J Am Coll Surg. 2007;204:697–705
  16. In:  Watts F editors. Assessment of Technical Skills. New York: Springer; 1985;p. 259–274
  17. Datta V, Bann S, Beard J, et al. Comparison of bench test evaluations of surgical skill with live operating performance assessments. J Am Coll Surg. 2004;199:603–606
  18. Moorthy K, Munz Y, Sarker SK, et al. Objective assessment of technical skills in surgery. BMJ. 2003;327:1032–1037
  19. NOTSS handbook. http://www.abdn.ac.uk/iprc/notss
  20. Yule S, Flin R, Paterson-Brown S, et al. Development of a rating system for surgeons' nontechnical skills. Med Education. 2006;40:1098–1104
  21. Roach PB, Roggin KK, Selkov E, et al. Use of a novel, web-based educational platform facilitates intraoperative training in a surgical oncology fellowship program. Ann Surg Oncol. 2009;16:1100–1107
  22. Cleveland WS. Robust locally-weighted regression and smoothing scatterplots. Journal of the American Statistical Association. 1979;74:829–836
  23. Kopta JA. An approach to the evaluation of operative skills. Surgery. 1971;70:297–303
  24. In:  Crosby PB editors. Quality is Free: the art of making quality certain. New York: McGraw-Hill Book Company; 1979;
  25. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84:273–278
  26. Vassiliou MC, Feldman LS, Andrew CG, et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005;190:107–113
  27. Deming WE. Out of the Crisis. Cambridge, MA: MIT Press; 2000;
  28. Winslow F, Taylor ME. Scientific Management. New York: Harper & Row; 1911;
  29. Ishikawa . Guide in Quality Control. Japan: Asian Productivity Organization; 1986;
  30. Juran J. Quality Control Handbook. 3rd ed.. New York: McGraw-Hill; 1974;
  31. Epstein RM. Assessment in medical education. N Engl J Med. 2007;356:387–396
  32. Reznick R, Regehr G, MacRae H, et al. Testing technical skill via an innovative “bench station” examination. Am J Surg. 1997;173:226–230
  33. Darosa DA. It takes a faculty. Surgery. 2002;131:205–209
  34. Ward M, MacRae H, Schlachta C, et al. Resident self-assessment of operative performance. Am J Surg. 2003;185:521–524
  35. Newble DI, Jaeger K. The effect of assessments and examinations on the learning of medical students. Med Education. 1983;17:165–171
  36. Spencer JA, Jordan RK. Learner centered approaches in medical education. BMJ. 1999;318(7193):1280–1283
  37. ACGME Outcomes Project. http://www.acgme.org/Outcome/
  38. Williams RG. The 10 commandments of performance appraisal provide a foundation for the surgical training process. http://www.facs.org/education/rap/williams.html

 Sources of funding: Department of Surgery, the University of Chicago; Computation Institute, the University of Chicago; Educational Research Grant, the University of Chicago Pritzker School of Medicine.

 Special thanks to the residents, fellows and staff of the University of Chicago, Department of Surgery, who have participated in STAT's development, and also to Stephen A. Roach for his contributions with respect to Managerial Science and quality management.

PII: S1931-7204(09)00159-7

doi: 10.1016/j.jsurg.2009.10.001

Journal of Surgical Education
Volume 66, Issue 5 , Pages 255-263 , September 2009

Article Tools

* Image Usage & Resolution