Clinical skills assessment
limitations to the introduction of an “OSCE” (Objective Structured Clinical Examination) in a traditional Brazilian medical school
Keywords:
Assessment, Clinical skills, University hospitals, Medical students, Undergraduate medical educationAbstract
CONTEXT: Assessment of clinical skills has a central role in medical education and the selection of suitable methods is highly relevant. The OSCE (Objective Structured Clinical Examination) is now established as one of the most valid, reliable and effective tests for the assessment of clinical skills. OBJECTIVE: To describe student and faculty perceptions of an OSCE introduced in a traditional Brazilian medical school. TYPE OF STUDY: Descriptive, semi-quantitative study. SETTING: Faculdade de Medicina de Ribeirão Preto, Universidade de São Paulo. PARTICIPANTS: 258 junior medical students finishing an introductory course on basic clinical skills and six faculty members deeply involved with the OSCE administration. PROCEDURES: Over a period of three consecutive years, student perceptions on the examination were evaluated using a structured questionnaire containing several five-point scales; faculty members’ opinions were collected using a structured questionnaire plus a personal interview. MAIN MEASUREMENTS: Student satisfaction or dissatisfaction with aspects of OSCE administration and positive or negative opinions from faculty members. RESULTS: Students were comfortable with cases and tasks, but nearly half (48%) of them criticized organizational aspects of the OSCE. Substantial proportions of students reported difficulties with both time management (70%) and stress control (70%). Improvement of several aspects of exams reduced criticism of organization to a minority (5%) of students, but the proportions of students reporting difficulties with time management (40%) and stress control (75%) during the exam remained virtually unchanged. Faculty members acknowledged the accuracy of the OSCE, but criticized its limitations for assessing the integrated approach to patients and complained that the examination was remarkably time and effort-consuming. The educational impact of the OSCE was felt to be limited, since other faculty members did not respond to the communication of exam results. CONCLUSIONS: In addition to shortage of resources and organizational difficulties, local cultural aspects and the absence of a more favorable educational climate may hinder lasting improvements in assessment methods in traditional medical schools.
Downloads
References
Newble DI. Assessing clinical competence at the undergradu- ate level. Med Educ 1992;26(6):504-11.
Lowry S. Assessment of students. BMJ 1993;306(6869):51-4.
Schuwirth LW, van der Vleuten CP. ABC of learning and teaching in medicine: Written assessment. BMJ 2003;326(7390):643-5.
Harden RM, Stevenson M, Downie WW, Wilson GM. Assess- ment of clinical competence using objective structured exami- nation. Br Med J 1975;1(5955):447-51.
Harden RM, Gleeson FA. Assessment of clinical competence using an objective structured clinical examination (OSCE). Med Educ 1979;13(1):41-54.
Proceedings of the AAMC’S Consensus Conference in the Use of Standardized Patients in the Teaching and Evaluation of Clini- cal Skills. Washington, D. C., December 3-4, 1992. Acad Med 1993;68:437-83.
Troncon LEA, Foss NT, Voltarelli JC, et al. Avaliação de habilidades clínicas por exame objetivo estruturado por estações, com emprego de pacientes padronizados: uma aplicação no Brasil. Rev Bras Educ Med 1996;20(2/3):53-60.
Troncon LEA. Restruturação curricular do curso médico na Faculdade de Medicina da Universidade de São Paulo — Ribeirão Preto. Rev Fac Ciências Médicas UNICAMP 2000;8(Edição Especial):33-7.
Newble D, Dawson B, Dauphinee D, et al. Guidelines for assess- ing clinical competence. Teach Learn Med 1994;6(3):213-20.
Smee S. Skill based assessment. BMJ 2003;326(7391):703-6.
Barrows HC. Simulated (standardized) patients and other hu- man simulations. Health Sciences Consortium. North Caroline: Chapel Hill; 1987.
Van der Vleuten CPM, Scherpbier AJJA, Dolmans DHJM, et al. Clerkship assessment assessed. Med Teach 2000; 22(6):592-600.
Krasilchik Avaliação do ensino. In: Anais do Encontro Avaliação e Ensino; 1992. São Paulo; Feusp; 1992. p.6 - 11.
Shatzer JH, Darosa D, Colliver JA, et al. Station-length require- ments for reliable performance-based examination scores. Acad Med 1993;68(3):224-9.
Cohen R., Reznick RK, Taylor BR, et al. Reliability and valid- ity of the objective structured clinical examination in assessing surgical residents. Am J Surg 1990;160(3):302-5.
Sloan DA, Donnelly MB, Schwartz RW, et al. The use of objec- tive structured clinical examination (OSCE) for evaluation and instruction in graduate medical education. J Surg Res 1996;63(1):225-30.
Wilson BE. Performance-based assessment of internal medi- cine interns: evaluation of baseline clinical and communication skills. Acad Med 2002;77(11):1158.
Reznick RK, Blackmore D, Cohen R, et al. An objective struc- tured clinical examination for the licentiate of the Medical Coun- cil of Canada: from research to reality. Acad Med 1993;68 (10 Suppl):S4-6.
Ziv A, Ben-David MF, Sutnick AI, Gary NE. Lessons learned from six years of international administration of the ECFMG’s SP-based clinical skills assessment. Acad Med 1998;73(1):84-91.
Mavis BE, Henry RC, Ogle KS, Hoppe RB. The emperor’s new clothes: the OSCE reassessed. Acad Med 1996;71(5):447-53.
Vu NV, Barrows HS, Marcy ML, Verhulst SJ, Colliver JA, Travis T. Six years of comprehensive, clinical, performance-based as- sessment using standardized patients at the Southern Illinois University School of Medicine. Acad Med 1992;67(1):42-50.
Duffield KE, Spencer JA. A survey of medical students’ views about the purposes and fairness of assessment. Med Educ 2002;36(9):879-86.
McMannus IC. How will medical education change? Lancet 1991;337(8756):1519-21.
Biggs JS, Price DA. Sustaining and rewarding clinical teaching. Med Educ 1992;26(4):264-8.
Vianna HM. Avaliação educacional nos cadernos de pesquisa. Cad Pesq São Paulo 1992;80:100-5.
Ministério da Educação. Conselho Nacional de Educação. Diretrizes curriculares nacionais do curso de graduação em Medicina. Disponível em: http://www.mec.gov.br/sesu/ftp/pareceres/113301EnfMedNutr.doc. Acessado em 27/10/03.
Des Marchais JE. From traditional to problem-based curricu- lum: how the switch was made at Sherbrooke, Canada. Lancet 1991;338(8761):234-7.
Lowry S. Making change happen. BMJ 1993;306(6873):320-2.
Krackov SK, Mennin SP. A story of change. Acad Med 1998;73(9 Suppl):S1-3.
Mennin SP, Kalishman S. Student assessment. Acad Med 1998;73(9 Suppl):S46-54.
van der Vleuten C. Improving medical education. BMJ 1993;306(6873):284-5.
