Assessment and evaluation in healthcare
16th Ottawa Conference, Transforming Healthcare Through Excellence in Assessment and Evaluation
David Birks (Senior Lecturer at SRH Latrobe Valley & West Gippsland) joined around 2,000 other medical educationalists from around the world at this year’s Ottawa conference.
David presented a paper (co-authored by Dr Cathy Haigh and Professor Brian Jolly of the University of Newcastle) about their nine-year experience of MiniCase Records (MCRs) as an assessment tool for Monash medical students. The MCR is a variant of the miniCEX (mini Clinical Evaluation Exercise) which is used widely for trainee physicians around the world. The paper he presented was included in a session of six papers, four of which focussed on the miniCEX.
Other sessions attended included a workshop on feedback as well as plenary sessions and symposia.
Plenary sessions
John Norcini (USA) gave the opening address in which took the view of improving practice by integrating assessment into practice. The data shows that doctors who passed postgraduate exams (which are not necessary for registration in the USA) performed better than their peers who did not take the exams but had otherwise undergone the same training. He also suggested that doctors don’t effectively apply new information that appears after they complete training, but remain competent in areas covered when they trained.
Des Gorman(NZ) delivered a stimulating session on planning the disposition and training of healthcare workers for an uncertain future. I later had the opportunity to hear Prof Gorman in more detail when he conducted a workshop at NOSM, Sudbury with Prof R Strasser. At these forums he emphasised the need for better leadership and planning of workforce. He outlined a different approach to planning which included
- a change in the recognition, reward and remuneration schema for doctors from one that has a recruitment and retention bias to one that has a productivity and (outcome) quality bias and
- a demographic, geographic and disciplinary redistribution of the medical workforce to better meet health need.
Brian Hodges(Canada) spoke about Scylla and Charybdis : caught between examination and reflection in medical education. It seems 17-20 per cent of medical students cannot reflect.
Trudie Roberts (UK) discussed cheating in assessments. This is more common than expected amongst students in general.
Glen Regehr (Canada) addressed the concept of “What if they aren’t playing our game” suggesting that the rigidity of the medical school system encourages students to play the game of focussing on the goals of the system rather than allowing them to become partners in the whole enterprise.
Symposia
David also attended three symposia which focussed on validity and reliability. The first of these is a difficult concept and nonmedical experts in the field outlined a detailed approach to achieve validity in assessment. This seemed to be quite complicated and probably not achievable by the average working clinician in the field. The session of reliability was more practical but emphasised the inherent idiosyncrasy in our judgements which probably cannot be removed. The sense was that diversity is important in assessment.
McGill University ran a symposium on professional development in assessment for clinicians and medical educators. This professional development is a worthwhile exercise which David recommends should be adopted locally.
An additional Educational Innovation Symposium focussed on putting EPAs (Entrustable Professional Activities) into action in the competency based educational environment. Professor Ten Cate from Utrech explained the rationale and process for EPAs. These seem a logical approach to assessing our students and trainees and determining their fitness(trustworthiness) to move onto independence in any particular area. He suggested trust was based on competence, honesty, reliability and vulnerability. (Academic Medicine, Vol. 89, No. 4 / April 2014 691 AM Last Page) David says EPAs should be considered in our approach to MCRs as well as the new development of the Continuous Clinical Skills program.
Posters
The posters on display covered many areas of medical education. One that struck David was a small randomised trial that found students who wrote notes during a lecture had better retention a month later than those that use a keyboard.