Putting together the pieces of the puzzle: Defensibility in admissions and in-program assessment

2 December 2020 - 2 December 2020
 Putting together the pieces of the puzzle: Defensibility in admissions and in-program assessment

ASME BITESIZE Website Event Banner

ASMEBITESIZE: Putting together the pieces of the puzzle: Defensibility in admissions and in-program assessment
Wednesday 2nd December 2020, 16:00 GMT 

This session was run by Dr Kelly Dore and Jillian Derby from Altus Assessments with host Dr Eliot Rees (TASME Chair) and was intended for faculty and staff who are involved at all levels of assessment in both trainee selection or in-program assessment. The topic is appropriate to all who work in undergraduate, graduate and postgraduate training.

This session explored how to ensure that pieces of your assessment process, either in-program or during admissions, are put together in a way that aligns to your program mission as well as the competencies which make up a good physician (i.e. the GMC guide to Good Medical Practice). Common mistakes and practical solutions were discussed in the construction of assessment processes across both the academic and non-academic attributes. Real case examples in admissions and selection as well as in-program were used to walk through the principles and theory discussed.

This session aimed to help participants: identify what they are trying to measure (constructs), ensure they are including and weighting assessment metrics appropriately, and learn how to bring together multiple assessments to ensure they are reflective of the broad constructs of physician practice. Key topics were reviewed to minimize validity threats to your assessment program including: construct identification, blueprinting (within assessments and across programs of assessment) in admissions and in-program, construct over/under representation & construct irrelevant variance.

Please click on the button below to access the webinar video

Video access

If you are unable to watch the video a transcript is available by clicking here

Kelly and Jillian have shared the following resources from their session...

Webinar slides

Implicit Bias Handout


Dr Kelly Dore

Kelly Dore Photo

Dr. Kelly Dore is an adjunct Associate Professor in the Departments of Medicine at McMaster University and Co-Founder of Altus Assessments. 

Dr. Dore completed her PhD in Health Research Methodology at McMaster University with a focus on Health Professions Education Research and Cognitive Psychology.  Dr. Dore was the Inaugural Director of the Masters Science in Health Science Education program at McMaster University – an online MSc Program for Health Professionals Internationally, developing and coordinating the core course on assessment and evaluation. She has also assisted in the development and teaches in the Royal College of Physicians and Surgeons of Canada Principles of Assessment in Simulation (PASS) course. Dr. Dore has held multiple grants from both the Medical Council of Canada, the National Board of Medical Examiners (Stemmler Grant), and the Royal College of Physicians and Surgeons of Canada. She has received the Department of Medicine Internal Career Award for Quality in Patient Safety, Graduate Teaching Award, and was the recipient of the Canadian Association of Medical Education Award of Merit. 

She is the Co-Creator of Casper, an online selection tool used internationally for selection into health professions and teacher education. In 2015, Altus Assessments Inc. was formed with Dr. Dore and Dr. Reiter as founders, offering the admissions screening test internationally.  

Jillian Derby

Jillian Derby Photo

Jillian possesses a demonstrated history of item and assessment development, as well as an extensive pedagogical background. Previously a consultant for curriculum development, she currently researches both education and assessment, affording her an informed perspective on the various complexities of SJT development. Her award-winning research on the nature of human emotion, alongside her expertise in philosophy, literature, and theatre, makes her intricately aware of what is essential for realistic and effective assessment content. At Altus, Jillian makes use of her unique background to facilitate, research, and guide all test content development from scenarios, to question sets, to compiled tests.

Dr Eliot Rees - TASME Chair, Lecturer in Medical Education at Keele University & Academic Clinical Fellow in General Practice at UCL

Eliot Rees Photo