Significance of experts’ overall ratings for medical student competence in relation to history-taking
Keywords:
Clinical competence, Medical history taking, Educational measurement, Medical students, Medical educationAbstract
CONTEXT AND OBJECTIVE: Overall ratings (ORs) of competence, given by expert physi- cians, are increasingly used in clinical skills assessments. Nevertheless, the influence of specific components of competence on ORs is incompletely understood. The aim here was to investigate whether ORs for medical student history-taking competence are influenced by performance relating to communication skills, completeness of questioning and asking content- driven key questions. DESIGN AND SETTING: Descriptive, quantitative study at Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo. METHODS: Thirty-six medical students were examined in a 15-station high-stake objective structured clinical examination (OSCE). At four stations devoted to history-taking, examiners filled out checklists covering the components investigated and independently rated students’ overall performance using a five-point scale from 1 (poor) to 5 (excellent). Physician ratings were aggregated for each student. Nonparametric correlations were made between ORs. RESULTS: ORs presented significant correlations with checklist scores (Spearman’s rs = 0.38; p = 0.02) and OSCE general results (rs = 0.52; p < 0.001). Scores for “communication skills” tended to correlate with ORs (rs = 0.31), but without reaching significance (p = 0.06). Nei- ther the scores for “completeness” (rs = 0.26; p = 0.11) nor those for “asking key questions” (rs = 0.07; p = 0.60) correlated with ORs. CONCLUSIONS: Experts’ overall ratings for medi- cal student competence regarding history-taking is likely to encompass a particular dimension, since ratings were only weakly influenced by specific components of performance.
Downloads
References
Harden RM, Stevenson M, Downie WW, Wilson GM. Assess- ment of clinical competence using objective structured clinical examination. Br Med J. 1975;1(5955):447-51.
Vu NV, Barrows HC, Marcy ML, Verhust SJ, Colliver JA, Travis T. Six years of comprehensive, clinical, performance-based as- sessment using standardized patients at the Southern Illinois University School of Medicine. Acad Med. 1992;67(1):42-50.
Newble DI. Assessing clinical competence at the undergraduate level. Med Educ. 1992;26(6):504-11.
Petrusa ER, Blackwell TA, Ainsworth MA. Reliability and validity of an objective structured clinical examination for as- sessing the clinical performance of residents. Arch Intern Med. 1990;150(3):573-7.
Reznick RK, Blackmore D, Cohen R, et al. An objective structured clinical examination for the licentiate of the Medical Council of Canada: from research to reality. Acad Med. 1993;68(10 Suppl):S4-6.
Proceedings of the AAMC’S Consensus Conference on the Use of Standardized Patients In the Teaching and Evaluation of Clinical Skills. Washington, D. C., December 3-4, 1992. Acad Med. 1993;68(6):437-83.
Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Acad Med. 1999;74(10):1129–34.
Vu NV, Marcy MM, Colliver JA, Verhust SJ, Travis TA, Barrows HS. Standardized (simulated) patients’ accuracy in recording clinical performance check-list items. Med Educ. 1992;26(2):99-104.
Wilkinson TJ, Fontaine S. Patients’ global ratings of student competence. Unreliable contamination or gold standard? Med Educ. 2002;36(12):1117–21.
Hodges B, Turnbull J, Cohen R, Bienenstock A, Norman G. Evaluating communication skills in the OSCE format: reliability and generalizability. Med Educ. 1996;30(1):38-43.
Rothman AI, Cusimano M. A comparison of physician examin- ers’, standardized patients’ and xommunication experts’ ratings of international medical graduates’ English proficiency. Acad Med. 2000;75(12):1206-11.
Newble D, Dawson B, Dauphinee D, et al. Guidelines for assessing clinical competence. Teach Learn Med. 1994;6(2):213-20.
Troncon LE. Clinical skills assessment: limitations to the introduction of an “OSCE” (Objective Structured Clinical Examination) in a traditional Brazilian medical school. Sao Paulo Med J. 2004;122(1):12-7.
Barrows HS. Simulated (standardized) patients and other hu- man simulations. Chapel Hill. North Carolina: Health Sciences Consortium; 1987.
Noel GL, Herbers JE Jr, Caplow MP, Cooper GS, Pangaro LN, Harvey J. How well do internal medicine faculty mem- bers evaluate the clinical skills of residents? Ann Intern Med. 1992;117(9):757-65.
Sloan DA, Donnelly MB, Schwartz RW, Felts JL, Blue AV, Strodel WE. The use of objective structured clinical examina- tion (OSCE) for evaluation and instruction in graduate medical education. J Surg Res. 1996;63(1):225-30.
Hodges B, McIlroy JH. Analytic global OSCE ratings are sensi- tive to level of training. Med Educ. 2003;37(11):1012-6.
MacRae HM, Vu NV, Graham B, Word-Sims M, Colliver JA, Robbs RS. Comparing checklists and databases with physicians’ ratings as measures of students’ history and physical-examination skills. Acad Med. 1995;70(4);313-7.
van der Vleuten CP, Swanson DB. Assessment of clinical skills with standardized patients: state of the art. Teach Learn Med. 1990;2(1):58-76.
van der Vleuten CP, Newble DI. How can we test clinical reasoning? Lancet. 1995;345(8956):1032-4.
