Benchmarking medical school assessment and graduate outcomes – developing a national approach to demonstrating quality graduate outcomes
What is driving this current project?
In the last three years in Australia and New Zealand there has been much debate, and quite considerable activity, about whether and how to work collaboratively across medical schools in assessment. Most assessment of learning outcomes, in most medical schools occurs in isolation with little, if any, external moderation. However, the narrative is changing with most schools recognising the value of collaboration in assessment. There is significant momentum building amongst regulators, medical educators, clinical training providers and funding bodies that all medical schools should be able to demonstrate that their medical graduates meet an agreed set of standards. The public would expect the same.
The first stage in achieving the aims of the Benchmarking project was to gain a commitment from all Australian and New Zealand medical schools to participate in the Project, and this has been achieved.
The second stage was to develop, collaboratively, a collection of national standard assessment items for delivery to final year medical students across all Australian and New Zealand medical schools, and this has been achieved – see update from the Chair of the Steering Committee, Professor Nicholas Glasgow, 24 March 2014.
The final stage was to pilot the items in penultimate and final year assessments – the Project has received all completed items from participating schools and this data has been analysed.
What is the aim of the project?
Numerous reports – from Australia and internationally – indicate that collaborative assessment can deliver efficiencies in resource use and assist in improving medical course assessments and improve our collective skills and understanding of assessment.
While there are currently a number of highly effective assessment consortia assessing medical students in various ways and at various stages of a medical degree, none delivers a set of comprehensive test items that assess the clinical years in a comprehensive and nationally consistent way. The aim of this project therefore is to benchmark medical school assessment and develop a collaborative national approach that demonstrates quality medical graduate outcomes and provides comparisons of outcomes.
Further the project aims for a consensus position from medical schools about a framework and a mechanism that is inclusive, open and collaborative and enables schools to work together on assessment. In any such framework schools would retain autonomy and responsibility for assessment but would share assessment materials in a way that provides anonymous, quantitative benchmarking data. Having a robust, comprehensive item bank and benchmarking data means we can start the conversations about performance and about ensuring high quality graduates from our medical schools
Why Medical Deans as leads?
Medical Deans, as the peak body for the first stage of medical training, support the basic principles, objectives and values of collaboration and is well placed to lead a collaborative assessment benchmark project. The recent development of the Medical Deans Medical Education Collaboration Committee (MECC), which includes Deans of Medicine and senior medical educators representing all 21 Australian and New Zealand medical schools, places Medical Deans in a position to achieve outcomes which are not able to be met by the individual consortiums.
FAQs about the project.
A final report on the outcomes of the project were presented at the Quality Assurance Workshop in April 2015. Medical Schools have received their confidential and de-identified psychometric reports. A collaborative project with the AMC will be the next stage of Assessment Benchmarking in 2016.
For more information on the project, contact Mr Dan Dumbrell, Project and Research Officer on (02) 8084 6557 or firstname.lastname@example.org.
This project was possible due to funding made available by Health Workforce Australia