Longitudinal evaluation of the Ophthalmology residents in Brazil
an observational prospective study
Keywords:
Ophthalmology, Education, Medical, Internship and residencyAbstract
BACKGROUND: The longitudinal evaluation of students seems to be a better way to assess their knowledge compared with that of the traditional methods of evaluation, such as modular and final tests. Currently, progress testing is the most consolidated type of longitudinal testing method. However, despite being well consolidated as an assessment tool in medical education, the use of this type of test in residency programs is scarce. OBJECTIVES: This study aimed to investigate residents’ knowledge growth regarding residency training and to describe the implementation of a longitudinal evaluation test in ophthalmological residency training across several medical schools in Brazil. Finally, the study aimed to check whether performance in the tests can be used as a predictor of the results of the specialist title test. DESIGN AND SETTING: This was a prospective observational study. This study was conducted using an online platform. METHODS: Online tests were developed following the same pattern as the Brazilian Ophthalmology Council specialist tests. All the residents performed the test simultaneously. The tests were conducted once a year at the end of the school year. RESULTS: A progress test was conducted across 13 services with 259 residents. Our results demonstrated that resident scores improved over the years (P < 0.0001) and had a moderate correlation with the Brazilian Opthalmology Council specialist test (P = 0.0156). CONCLUSION: The progress test can be considered a valuable tool to assess knowledge, meaning their knowledge increased over residency training. In addition, it can be used as a predictor of the result in the specialist title test.
Downloads
References
Van Der Vleuten CP. The assessment of professional competence: Developments, research and practical implications. Adv Health Sci Educ Theory Pract. 1996;1(1):41-67. PMID: 24178994; https://doi.org/10.1007/BF00596229.
Cecilio-Fernandes D, Cohen-Schotanus J, Tio RA. Assessment programs to enhance learning. Physical Therapy Reviews. 2018;23(1):17-20. https://doi.org/10.1080/10833196.2017.1341143.
Wood T. Assessment not only drives learning, it may also help learning. Med Educ. 2009;43(1):5-6. PMID: 19140992; https://doi.org/10.1111/j.1365-2923.2008.03237.x.
Norcini J, Anderson B, Bollela V, et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach. 2011;33(3):206-14. PMID: 21345060; https://doi.org/10.3109/0142159X.2011.551559.
Verhoeven BH, Snellen-Balendong HA, Hay IT, et al. The versatility of progress testing assessed in an international context: a start for benchmarking global standardization? Med Teach. 2005;27(6):514-20. PMID: 16199358; https://doi.org/10.1080/01421590500136238.
Nouns ZM, Georg W. Progress testing in German speaking countries. Med Teach. 2010;32(6):467-70. PMID: 20515374; https://doi.org/10.3109/0142159X.2010.485656.
Dijksterhuis MG, Scheele F, Schuwirth LW, et al. Progress testing in postgraduate medical education. Med Teach. 2009;31(10):e464-8. PMID: 19877854; https://doi.org/10.3109/01421590902849545.
Wrigley W, van der Vleuten CP, Freeman A, Muijtjens A. A systemic framework for the progress test: strengths, constraints and issues: AMEE Guide No. 71. Med Teach. 2012;34(9):683-97. PMID: 22905655; https://doi.org/10.3109/0142159X.2012.704437.
Cecilio-Fernandes D, Bicudo AM, Hamamoto Filho PT. Progress testing as a pattern of excellence for the assessment of medical students’ knowledge: concepts, history, and perspective. Medicina (Ribeirão Preto). 2021 Jul 16;54(1):e173770. Available from: http://hdl.handle.net/11449/229239. Accessed in 2022 (Jul 5).
Rutgers DR, van Raamt F, van Lankeren W, et al. Fourteen years of progress testing in radiology residency training: experiences from The Netherlands. Eur Radiol. 2018;28(5):2208-15. PMID: 29196854; https://doi.org/10.1007/s00330-017-5138-8.
McHarg J, Bradley P, Chamberlain S, et al. Assessment of progress tests. Med Educ. 2005;39(2):221-7. PMID: 15679690; https://doi.org/10.1111/j.1365-2929.2004.02060.x.
Cecilio-Fernandes D, Kerdijk W, Jaarsma AD, Tio RA. Development of cognitive processing and judgments of knowledge in medical students: Analysis of progress test results. Med Teach. 2016;38(11):1125-9. PMID: 27117670; https://doi.org/10.3109/0142159X.2016.1170781.
Hamamoto Filho PT, Silva E, Ribeiro ZMTet al. Relationships between Bloom’s taxonomy, judges’ estimation of item difficulty and psychometric properties of items from a progress test: a prospective observational study. Sao Paulo Med J. 2020;138(1):33-9. PMID: 32321103; https://doi.org/10.1590/1516-3180.2019.0459.R1.19112019.
Verhoeven BH, Verwijnen GM, Scherpbier AJJ, et al. An analysis of progress test results of PBL and nonPBL students. Med Teach. 1998; 20(4):310-6. https://doi.org/10.1080/01421599880724.
Albano MG, Cavallo F, Hoogenboom R, et al. An international comparison of knowledge levels of medical students: the Maastricht Progress Test. Med Educ. 1996;30(4):239-45. PMID: 8949534; https://doi.org/10.1111/j.1365-2923.1996.tb00824.x.
Cecilio-Fernandes D, Aalders WS, Bremers AJA, Tio RA, de Vries J. The Impact of Curriculum Design in the Acquisition of Knowledge of Oncology: Comparison Among Four Medical Schools. J Cancer Educ. 2018 Oct;33(5):1110-1114. PMID: 28374229; https://doi.org/10.1007/s13187-017-1219-2.
Cecilio-Fernandes D, Aalders WS, de Vries J, Tio RA. The Impact of Massed and Spaced-Out Curriculum in Oncology Knowledge Acquisition. J Cancer Educ. 2018;33(4):922-5. PMID: 28194581; https://doi.org/10.1007/s13187-017-1190-y.
Verhoeven BH, Verwijnen GM, Scherpbier AJ, van der Vleuten CP. Growth of medical knowledge. Med Educ. 2002;36(8):711-7. PMID: 12191053; https://doi.org/10.1046/j.1365-2923.2002.01268.x.
Blake JM, Norman GR, Keane DR, et al. Introducing progress testing in McMaster University’s problem-based medical curriculum: psychometric properties and effect on learning. Acad Med. 1996;71(9):1002-7. PMID: 9125989; https://doi.org/10.1097/00001888-199609000-00016.
Ferreira RC. Relação entre o desempenho no teste de progresso e na seleção para residência médica [thesis]. Campinas: Universidade Estadual de Campinas; 2019.
Bicudo AM, Hamamoto Filho PT, Abbade JF, Hafner MLMB, Maffei CML.Teste de Progresso em Consórcios para Todas as Escolas Médicas do Brasil. Rev Bras Educ Med. 2019;43(4):151-6. https://doi.org/10.1590/1981-52712015v43n4RB20190018.
Conselho Federal de Medicina. VII Fórum Nacional de Ensino Médico [Internet]. Available from: http://www.eventos.cfm.org.br/index.php?option=com_content&view=article&id=21090. Accessed in 2022 (Jul 5).
Ravesloot C, van der Schaaf M, Haaring C, et al. Construct validation of progress testing to measure knowledge and visual skills in radiology. Med Teach. 2012;34(12):1047-55. PMID: 22931139; https://doi.org/10.3109/0142159X.2012.716177.
Dijksterhuis MG, Scheele F, Schuwirth LW, et al. Progress testing in postgraduate medical education. Med Teach. 2009;31(10):e464-8. PMID: 19877854; https://doi.org/10.3109/01421590902849545.
Dijksterhuis MG, Schuwirth LW, Braat DD, Scheele F. An exploratory study into the impact and acceptability of formatively used progress testing in postgraduate obstetrics and gynaecology. Perspect Med Educ. 2013;2(3):126-41. PMID: 27023455; https://doi.org/10.1007/s40037-013-0063-2.
Tso MO, Goldberg MF, Lee AG, et al. An international strategic plan to preserve and restore vision: four curricula of ophthalmic education. Am J Ophthalmol. 2007;143(5):859-65. PMID: 17452171; https://doi.org/10.1016/j.ajo.2007.01.055.
ICO Residency Curriculum 2nd Edition and Updated Community Eye Health Section.. San Francisco, CA: International Council of Ophthalmology; 2016. Available from: https://dansk-oftalmologisk-selskab.dk/wp-content/uploads/2021/05/161216-updated-ico-residency-curriculum.pdf. Accessed in 2022 (Jul 8).
Conselho Brasileiro de Oftalmologia [Internet]. Available from: https://www.cbo.net.br/. Accessed in 2022 (Jun 5).
Chern KC, Saidel M, editors. Review questions in ophthalmology. 3rd ed. Philadelphia: Wolters Kluwer; 2015.
Heidary F, Rahimi A, Gharebaghi R. Clinical Optics and Refraction 313 Key Questions Answered. Shiraz, Iran: CreateSpace Independent Publishing Platform; 2013
Chua CN. Self-tests in optics and refraction. Kelana Jaya: Alco Malaysia; 2007.
Tomic ER, Martins MA, Lotufo PA, Benseñor IM. Progress testing: evaluation of four years of application in the school of medicine, University of São Paulo. Clinics (Sao Paulo). 2005;60(5):389-96. PMID: 16254675; https://doi.org/10.1590/s1807-59322005000500007.
Dournes G, Bricault I, Chateil JF. Analysis of the French national evaluation of radiology residents. Diagn Interv Imaging. 2019;100(3):185-93. PMID: 30527527; https://doi.org/10.1016/j.diii.2018.11.006.
Ray JJ, Sznol JA, Teisch LF, et al. Association Between American Board of Surgery In-Training Examination Scores and Resident Performance. JAMA Surg. 2016;151(1):26-31. PMID: 26536059; https://doi.org/10.1001/jamasurg.2015.3088.
Mauro GP, Najas GF, Carvalho HA, Villar RC. Prospective validation of a core curriculum progress assimilation instrument for radiation oncology residentship. Rep Pract Oncol Radiother. 2020;25(6):951-5. PMID: 33100910; https://doi.org/10.1016/j.rpor.2020.09.003.
Hamamoto Filho PT, de Arruda Lourenção PLT, do Valle AP, Abbade JF, Bicudo AM. The Correlation Between Students’ Progress Testing Scores and Their Performance in a Residency Selection Process. Med Sci Educ. 2019;29(4):1071-5. PMID: 34457585; https://doi.org/10.1007/s40670-019-00811-4.
Al-Mohammed A, Al Mohanadi D, Rahil A, et al. Evaluation of Progress of an ACGME-International Accredited Residency Program in Qatar. Qatar Med J. 2020;2020(1):6. PMID: 32300550; https://doi.org/10.5339/qmj.2020.6.
