1. Power, politics and practicalities in the evaluation of specialist medical training

The evaluation of a specialist training programme is fraught with issues of power (specialist/novice), politics (service/learning/funding) and practicalities, for example, which standards should apply — local, national, international? This workshop enabled participants to consider these issues in their evaluations. The objectives of the workshop were as follows:

• to discuss their experiences with evaluations
• to produce a personal action plan for further work on evaluations
• to make improvements to postgraduate specialist medical education.

The workshop was run on the following basis: Cycles of plenary and small group work; drawing on the facilitators quantitative and qualitative research; and based on discussions and activities framed around the following questions:

1. What are the competencies of medical specialists?
2. What should the training be seeking to achieve?
3. Why should the training be evaluated?
4. How should the evaluation be carried out?

Techniques included — brain storming, snow balling, World Café, nominal group technique as appropriate depending on the discussion flows.

The workshop

1. enabled the participants to share their experiences of good practices and pitfalls in evaluating specialist training
2. empowered the participants to facilitate further evaluations in order to improve training.

PowerPoint slides to set the background including our literature review and separately to present the results from our research. Each participant will have drafted a personal action plan to take away for discussion at their workplace.

Facilitators: Mick Button & Iain Robbe, Cardiff University, Wales, UK

2. Managing doctors in difficulty

This workshop used the NACT UK document "Managing Trainees in Difficulty" to help participants gain confidence in managing and supporting trainees with difficulties.

Small group work was used to explore scenario examples of doctors in difficulty, followed by plenary discussion of learning points. Participants had the opportunity to explore current arrangements and utility of the new guidance.

Liz Spencer is an experienced Director of Medical Education and co-ordinated the NACT UK working group on Managing Trainees in Difficulty. She is an experienced teacher and has been involved in Train the Trainer Programmes locally, regionally and nationally (for NACT UK).

The NACT UK document was given to all participants. The framework enabled prompt management of trainees in difficulty. Standard paperwork enables concerns and an action plan to be documented. Managing issues early and appropriately improves training satisfaction and service delivery, while maintaining patient safety.

Facilitator: Liz Spencer, Gloucestershire NHS Foundation Trust, UK

3. Leadership and management for medical students

At the end of the workshop, participants were able to:

1. Knowledge:
   1. demonstrate awareness of the need for leadership and management education at the undergraduate level.
   2. demonstrate awareness of the national policy emphasis on leadership and management education at the undergraduate level.
   3. demonstrate awareness of the NHS Institution for Innovation and Improvement Medical Leadership Competency Framework.
2. Skills:
   1. approach a clinically-based problem from a different perspective.
   2. approach a problem from a ’management’ perspective and translate this into curriculum innovation.
   3. solve a management problem.
3. Attitudes:
   1. to take a positive, proactive view towards management and leadership education within the undergraduate curriculum.
   2. to take a positive view towards collaboration with those involved in the provision of leadership and management education in the formulating of possible inputs to medical education.

The workshop was interactive and took the following form:

1. Introduction to workshop: Outlining the structure of the workshop.
2. Participants were divided into 6 groups: junior doctor, junior nurse, senior nurse, consultant, patient 1, operations manager. Each group was assigned to one of the above, and was given a short description of the person, their roles and responsibilities. Participants were asked to observe a video and record their observations from that person's perspective.
3. Video: watched by all participants. This would be a created video, using actors to illustrate a common scenario encountered by junior doctors which raises important issues about management: In A&E, a junior doctor is being told that he has to leave a medically complex case - patient 1 to attend to a simpler medical case because the latter patient is about to breach the 4 hour target. The hospital is applying for Trust status and is currently failing to meet A&E 4 hour targets.
4. Group work. Each group was asked to arrive at a consensus perspective for their person in terms of: A] Implications of this problem for their person. B] Decide what their person wanted the junior doctor to do or be able to do from his/her perspective. Each educational need was written on separate Post-it notes.
5. Group work activity. Around the room were 5 paper sheets representing each of the 5 domains of the Medical Leadership Competency Framework (NHS Institute for Innovation and Improvement). Each group was asked to attach the educational needs identified in Phase 4 within these 5 domains.

By the end of this phase, participants came to a view of all the competences which the workshop has identified would have been useful in this case.

6. Group work. Participants were asked to identify: A] What opportunities exist in their current undergraduate curriculum where these issues are addressed B] What new opportunities would have to be provided to address these issues.
7. Summary and Feedback - Outline suggestions for curriculum innovation and reinforce why it is relevant to contemporary medical education.

The workshop co-ordinators agreed to collate results of the day and email to participants.

Participants identified the importance and range of management issues relevant to their own courses. Brain storm opportunities in own course where management issues could fruitfully be addressed. The increase in awareness of the policy emphasis on leadership and management for undergraduate medical students should lead to curriculum innovation to enable future doctors to better lead and manage the health service for the benefit of patient care.

Facilitators: Mark Abbas, John Benson, Steve Gillam, Thelma Quince & Diana Wood, University of Cambridge, UK

4. Tracking and monitoring professionalism

The emphasis of this workshop was on the undergraduate curriculum but not confined to this setting.

Participants explored:

• The historical background relevant to our understanding and interest in professionalism in healthcare.
• The current context and drivers for assessing and tracking professionalism
• The purposes of assessing professionalism i.e. the inferences and decisions we want to draw from the results: the literature to support these.
• Key components of professionalism and criteria for judgements.
• The nature of assessment of professionalism and how to improve the construct validity of the decisions.
• How to address students’ objections and worries about assessment and tracking of professionalism.
• The balance between supporting the student and protecting the patient.
• IT systems and processes to track assessments.
• How the university Boards of Examiners might deal with accumulated concerns
• How Boards of Examiners should relate to Fitness to Practise committees.

The workshop took the following format:

1. Presentations on background and context.
2. Discussions on purposes of assessment of professionalism and key components.
3. Small group discussions tackling key issues such as
   • Taking into account the students perspective
   • Supporting students or protecting patients
   • Who decides: Board of Examiners or Fitness to Practise Committees
   • National v local decision-making processes and committee
4. Demonstration of Edinburgh’s IT system for collecting and accessing data

Material for distribution included PowerPoint slides and reference list.

Facilitators: Helen Cameron & Karen Simpson, University of Edinburgh, UK

5. Can we identify key risk areas and reduce failure rates in a high-stakes summative OSCE? Setting up a formative clinical skills OSCE to identify 7 specific learning needs of underperforming candidates

This workshop enabled participants to understand the assessment of clinical skills in postgraduate medicine, using the MRCGP Clinical Skills Assessment (CSA), now part of the new licensing exam for UK general practice as an example of an integrated simulated surgery OSCE and to learn what distinguishes the CSA from standard OSCEs. It aimed to help participants understand how the cases are developed and marking schedules devised.

It explored the reasons for and challenges faced by educational institutions regarding differential pass rates of the CSA, e.g. International Medical Graduates/UK University of graduation and helped participants develop skills to make judgements about underperforming candidates. It aimed to teach participants about a method piloted in SE Scotland for the early identification of candidates at risk of failure in the CSA using a CSA-style OSCE to inform teaching and learning strategies for the trainee.

Presentation: the background to the CSA, its format and purpose, assessment methodology, validity and reliability, differential pass/fail rates. Small group work, including the challenges of the CSA as an assessment, possible reasons for candidate failure and differential fail rates.

Feedback and Q and A. Small group work looking at filmed material of GPRs; making decisions on grading and identifying specific learning needs of GPRs Outline of SE Scotland approach to early identification. Training of the formative OSCE facilitators, setting up and running the OSCE, immediate and longer term evaluation Interactive whole group discussion- lessons learned, implications for use of OSCE style assessments.

Handouts relating to the workshop were provided for participants along with a stepby- step guide to setting up a formative CSA-style OSCE.

Facilitators: MeiLing Denney, Drummond Begg, NHS Education Scotland, Edinburgh & Richard Wakeford, University of Cambridge, UK

6. Ultrasound for anatomists

This workshop demonstrated the use of an ultrasound machine on living subjects to normal cadaveric learning and will show that modern ultrasound machines are easy to use. We used the ultrasound machine to link surface anatomy to living anatomy and will show how dynamically viewing the interior of the living can be linked to applied anatomy.

The neck was been selected as the area of interest for the purposes of this workshop. This region is easy to access and has a number of vital anatomical structures that can be easily seen on an ultrasound machine — these structures can be linked to clinical moulages. We used diagrams and photographs to show the anatomy of the neck. We demonstrated the use of a Sonosite Micromaxx ultrasound machine to show the internal anatomy of the neck. We asked participants to use the ultrasound machine on the neck of the subject so that they get hands on experience of using an ultrasound machine on a living subject. We asked participants to identify structures on the ultrasound image.

The workshop contained a PowerPoint presentation; participants were given a handbook that will show them how to use the ultrasound machine in anatomy teaching and learning. The workshop aimed to show that ultrasound is useful in the teaching of anatomy; that the technique is easy to do and that hands-on experience engages learners in the teaching session.

Facilitator: Donny Shanahan, Northumbria University, UK

7 Teaching and learning a clinical skill

This workshop aimed to explore how educational theory can inform teaching within medical education. By the end of this interactive workshop, participants were able to:

• Understand the key choices required in planning clinical skills teaching
• Review the major factors affecting student learning
• Apply these factors affecting learning in order to critique traditional and more modern approaches to teaching a clinical skill.

This interactive workshop had two main components. Participants taught a simple skill in small groups (no prior knowledge required) and observers within each group were asked to deconstruct the main choices that were made in doing so. From this participants identified some of the key principles to consider when planning and delivering clinical skills training, and these will be related to the literature. These included aspects from the competency model (Purnell, 1998), level of skill (Simpson, 1972); instructor behaviour including scaffolding, just in time information (van Merrienboer and de Croock, 2002), timing of feedback etc; and training design choices including Part task vs Whole task training, overtraining, spacing and duration (Patrick, 2002)

The second half of the workshop tasked the participants with identifying the key factors affecting student learning through a snowballing exercise. We used this framework to critique four models for clinical skills teaching and learning — the four stage/ACLS approach (Peyton, 1998), Cagne s instruction events (O’Connor, 2002), the Evans- Brown approach (Evans and Brown, 2009 (in press)) and the Michels framework (Michels et al., 2009)

The workshop was of interest to those with some experience of teaching clinical skills who would like to know more about how educational theory might validate and help them improve their practice. It was also relevant to those involved in clinical skills curriculum design and in staff development around clinical skills.

The workshop generated a list of key aspects to consider in relation to skills teaching, and the relevant literature on these aspects. A prioritised list of factors affecting learning was also generated, alongside a critique of some of the main methods for teaching clinical skills. These will be disseminated to the participants.

The workshop provided development for skills teachers, and we hope that it also stimulated ongoing collaboration between clinical skills teachers, building new partnerships between institutions.

Facilitators: Kerry Boardman & Dason Evans, St George s, University of London, UK Martina Michels, University of Applied Science, Bern-Basel, Switzerland

8. It ain t what you say it s the way that you say it: moving beyond thematic analyses of interview data in medical education research

Interview data like focus group discussions and individual interviews are commonplace in medical education research, with researchers typically employing thematic analyses to analyse data. Although important in determining the whats of talk (i.e. contentrelated themes), thematic analysis tends to emphasise objective and factual aspects of the data, ignoring the hows of talk such as linguistic and paralinguistic features. However, by analysing linguistic and paralinguistic features of interview participants talk (e.g. metaphors, laughter etc) 2-4, researchers can reveal hidden insights about their topic of inquiry; insights not apparent from thematic analysis alone.

Using qualitative data from the facilitators own medical education research studies, this workshop aimed to help qualitative researchers in medical education develop their analytical toolkit. By the end of the workshop, delegates were able to:

• Identify a variety of important linguistic and paralinguistic features of interview participants talk
• Appreciate how the analysis of interview participants talk can deepen their understanding, not only of their research questions, but also of other important issues in medical education research such as identity, power etc.
• Analyse one or two linguistic and/or paralinguistic features of interview participants talk (e.g. humour)

The format of the session was a 3-hour interactive workshop. It began with a brief overview presentation about the analysis of linguistic and paralinguistic features of interview participants talk. The workshop continued by giving workshop participants hands-on experience analysing real data. Participants discussed their analyses with each other, comparing their findings firstly in small groups. The workshop ended with a large group discussion about the analyses, with comparisons about the findings between the small groups and the facilitators analyses of the data; and a discussion about the strengths and weaknesses of the analysis, and how participants can use such analysis with their own medical and healthcare education research. The facilitators suggested pre-reading to workshop delegates; provided participants with handouts such as PowerPoint slides and a list of follow-up literature.

It is anticipated that the workshop will help disseminate good practice regarding the analysis of qualitative data, promote collaboration between qualitative researchers in medical and healthcare education, and develop the qualitative analytic skills of medical and healthcare education researchers, thereby promoting and improving the quality of educational qualitative research.

Participants were requested to complete an evaluation form at the end of the workshop to elicit their views about whether the workshop activities met the specified learning outcomes.

Facilitators: Lynn Monrouxe, Cardiff University, Wales, UK & Charlotte Rees, University of Sydney, Australia