Research Methods Series: Lies, damned lies, and surveys: Designing better surveys for education and research
Wednesday 23rd June, 18:00-19:00 BST
Mark Twain famously expressed his disdain for statistics when he said, “There are three kinds of lies: lies, damned lies, and statistics.” Beyond referencing his own difficulties with maths, Twain was making the point that statistics can have persuasive power, even when used inappropriately.
In short, statistics often are used to bolster weak arguments.
The same can be said of surveys—results from a poorly designed and/or poorly executed survey can have considerable persuasive power. Flawed survey results can send educators off searching for fixes to non-existent problems, negatively influence policy decisions, and fill the scientific literature with unsubstantiated knowledge claims that take years to correct, if they are ever corrected.
In this presentation, Prof Anthony Artino will discuss the challenge of designing and implementing high-quality, self-administered surveys. Using empirical data and a series of examples from health care, education, and politics, he will describe how survey results can be used to shape science and policy, often inappropriately. In addition, he will provide insight into the cognitive processes that guide how respondents make sense of survey questions, and demonstrate what happens when these cognitive processes are ignored by educators and researchers. Finally, he will propose several design principles meant to improve surveys and introduce a seven-step process that front loads the survey-design process by focusing heavily on item development and pretesting. Taken together, the goal of this talk is to demonstrate that Mark Twain’s lament, when applied to surveys, is often correct—but it certainly doesn’t have to be.
Learning Objectives:
By the end of this session, participants will be able to…
Recognize the elements of a survey;
Describe how cognitive processes and motivation guide the way people understand and respond to survey questions;
Identify poorly written survey items and other design pitfalls;
State several design principles; and
Identify the importance of conducting expert reviews and cognitive interviews
Handouts:
Developing Questionnaires for Educational Research: AMEE Guide No. 87
The Survey Checklist (Manifesto)
Guidelines for Reporting Survey-Based Research Submitted to Academic Medicine
Designing Better Surveys for Education and Research
Speaker Details
Anthony R. Artino, Jr., PhD, is tenured professor and Interim Associate Dean for Evaluation and Educational Research at the George Washington University (GW) School of Medicine and Health Sciences. Trained as an educational psychologist, he has been a leader in military medicine, serving 23 years in the U.S. Navy and retiring as a Captain in 2020. He is currently leading GW’s efforts to advance educational research and innovative scholarship within medicine and health sciences, and he is internationally recognized for his work in health professions education and research. His research focuses on various aspects of human motivation, learning, and assessment in medicine. He is deputy editor for the Journal of Graduate Medical Education, associate editor for Perspectives on Medical Education, and assistant editor for Academic Medicine. He is also a fellow of the Association for Medical Education in Europe and has published more than 200 scientific articles and book chapters. Previously, he was Deputy Director of the Center for Health Professions Education at the Uniformed Services University of the Health Sciences, where he was responsible for graduate level teaching, research, faculty and student mentoring, and administrative leadership for the graduate programs in health professions education.
Please click on the button below to register*
Register here
*By registering for our Zoom webinar you will be sent a confirmation email with instructions of how to join the session. If you don’t receive these instructions please email: Tasme-Info@asme.org.uk and the confirmation email will be resent
If you have any questions regarding the Research Methods Series please email: Tasme-info@asme.org.uk