|
| |
| SHORT COMMUNICATION |
|
| Ahead of print publication |
|
|
Workplace-based assessment in the era of competency based medical education: Perception and experiences as a medical teacher
Jarina Begum1, Tanvir K Sidhu2, Anushi Mahajan3, Sinjita Dutta4, Anuj Singhal5, Gagandeep Kwatra6
1 Department of Community Medicine, Great Eastern Medical School and Hospital, Srikakulam, Andhra Pradesh, India
2 Department of Community Medicine, Adesh Institute of Medical Sciences and Research, Bathinda, Punjab, India
3 Department of Periodontology, Christian Dental College, CMC, Ludhiana, Punjab, India
4 Department of Community Medicine, Institute of Post Graduate Medical Education and Research (IPGME&R), Kolkata, West Bengal, India
5 Department of Medicine, Army Hospital Research and referral, New Delhi, India
6 Department of Pharmacology, Christian Medical College and Hospital, Ludhiana, India
| Date of Submission | 28-Jan-2022 |
| Date of Decision | 12-Apr-2022 |
| Date of Acceptance | 24-Apr-2022 |
| Date of Web Publication | 19-Jul-2022 |
Correspondence Address:
Jarina Begum,
Address: Professor in Department of Community Medicine, 2nd Floor, College Building, Great Eastern Medical School and Hospital, Srikakulam - 532 484, Andhra Pradesh
India
 Source of Support: None, Conflict of Interest: None DOI: 10.4103/mjdrdypu.mjdrdypu_79_22
Context: Workplace-based assessment (WPBA) is an essential component of competency-based assessment addressing the “does” level. The study was planned to understand the perceptions and experiences of medical faculties with WPBA for its formal adoption under CBME. Aim: To obtain faculty perceptions and their experiences with WPBA as a medical educator. Settings and Design: A cross-sectional survey was conducted as a part of an online web discussion on WPBA among 38 faculties. Materials and Methods: The information was collected through an online semi-structured questionnaire and data were analyzed using percentages, proportions, and themes. Results: The majority were 31–50 years, females and professors. Directly observed procedural skills (DOPS), case-based discussion (CBD), and logbooks were commonly used tools. The challenges in WPBA practice were hesitancy, time constraints, lack of resources, complexity of tools, and pandemic situation. Conclusions: practice of WPBA tools was perceived positively, however a huge gap was noticed between knowledge and practice of WPBA tools.
Keywords: CBD, CBME, DOPS, feedback, KAP, medical faculty, mini-CEX, WPBA
How to cite this URL:
Begum J, Sidhu TK, Mahajan A, Dutta S, Singhal A, Kwatra G. Workplace-based assessment in the era of competency based medical education: Perception and experiences as a medical teacher. Med J DY Patil Vidyapeeth [Epub ahead of print] [cited 2023 Mar 20]. Available from: https://www.mjdrdypv.org/preprintarticle.asp?id=351336 |
| Introduction | |  |
The competency-based medical education (CBME) curriculum has been introduced in India through the Graduate Medical Education Regulations (GMER) guidelines 2019.[1] Even before this, the CBME curriculum for postgraduation was also adopted.[2] One of the most important pillars of CBME is the assessment. Workplace-based assessment (WPBA) offers the assessment at the highest level of Miller's pyramid, i.e., at the “Does” level.[3],[4],[5],[6] However, WPBA has been underutilized and almost non-existent in the assessment methodology of most of the medical colleges throughout India. The study was planned to get the baseline data of the faculty perceptions and experiences with WPBA to characterize faculty needs to support the adoption of WPBA under CBME.
| Aim | | |
To obtain faculty perceptions and their experiences with WPBA as a medical teacher.
| Objectives | | |
- To assess the knowledge, attitude, and practices (KAP) of the faculty regarding WPBA.
- To evaluate the faculty perception towards the application of WPBA tools in relation to CBME curriculum and pandemic situations.
| Subjects and Methods | | |
A cross-sectional survey was conducted among the faculties who enrolled for a fellowship in the advanced international medical education and research (FAIMER) program in November 2020 as part of the Mentor Learner Web discussion. A semi-structured questionnaire, validated by the subject experts, was used as the study tool. It consisted of three sections: the demographic details (multiple-choice questions (MCQs)), the KAP of WPBA (MCQs and 5-point Likert scale), and open-ended questions on their views and suggestions towards the practice of WPBA in relation to CBME and pandemic. Data thus collected were analyzed using descriptive statistics and thematic analysis.
| Results | | |
A total of 38 responses were recorded which showed that the majority (81.6%) were 31 to 50 years of age group, females (71.1%), and professors (55.3%), having more than 10 years of teaching experience (78.9%), and belong to pre-para clinical department (44.8%) followed by clinical and dental sciences [Table 1].
Half of the participants (19) had previous training on WPBA and the majority (84%) are practising these methods already with directly observed procedural skills (DOPS) 62%, case-based discussion (CBD) 58%, and logbooks 54% being the most common followed by mini-clinical examination (mini-CEX), multi-source feedback (MSF), and portfolios.
Undergraduates (67%) were mostly the target learners for these tools, followed by postgraduates (42%) and interns (37%).
The majority either strongly agreed or agreed that for the practice of WPBA tools, the important requirements were multiple assessors, multiple methods, training of assessors, and more resources. Similarly, the essential characteristics of WPBA tools were giving feedback, working in a team, documenting, using a standardized checklist, and learner-initiated process.
More than 90% of participants strongly agreed or agreed that WPBA can be used for teaching and assessing skills and it ensures entrusted professional activity as per CBME (86%). More than half agreed to its use in pandemics despite limited feasibility. However, the use was perceived as mostly formative (72%) and few (28%) believed its implication in a longitudinal program only.
Almost all the participants agreed on the high educational impact of this method. Around 80% agreed to their high validity and reliability. The indicators low on the utility of WPBA were acceptability and feasibility.
The qualitative data in terms of views expressed in response to open-ended questions were analyzed by thematic analysis. If given an opportunity, the various preferred WPBA tools to be used by the participants with reasons are tabulated in [Table 2].
The various themes that emerged on difficulties anticipated/faced in implementing WPBA were time constraints, lack of resources, the complexity of the tool, and the pandemic crisis with few faculty and student factors. The probable solutions suggested by the participants were time management, creating/reallocating resources, faculty training by faculty development programmes, sensitizing students and adopting small batches, the adaptation of WPBA tools as per the Indian context, and developing a conducive environment through policies and guidelines for effective practice of WPBA tools [Figure 1]. | Figure 1: Challenges anticipated and solutions suggested in WPBA implementation
Click here to view |
As far as the effect of the coronavirus disease 2019 (COVID-19) pandemic on WPBA implementation at their institute was concerned, a majority (96%) perceived it as a difficult task. Few (7.8%) participants reported a positive effect in terms of getting an opportunity to small group CBDs. The participants had divided views on using alternatives like simulation; the maximum responses indicated simulation to use as an adjunct but not as a replacement.
Overall, the study participants were skeptical towards WPBA tools, their utility, and practice in relation to the CBME and COVID-19 pandemic along with the role of simulation [Figure 2].
| Discussion | | |
In the present study, most of the participants were in the 31–50 years of age range, females, and professors with teaching experience of more than 10 years in the field of medical education showing familiarity with various educational tools in their practices. Pre/para and clinical faculties were involved in the study attributing uniformity of data and unbiased responses. Studies have shown the assessment strategies currently in place for international medical graduates seeking certification are DOPS, MSF, reflections, and portfolios. Similarly, the development of any assessment system depends on direct observation and feedback making it valid and reliable at the same time using multiple modalities, multiple samples, and multiple assessors.[7],[8]
The present study found that WPBA tools are mostly formative. Many studies have mentioned the formative aspect of WPBA tool. However, its role as summative tool is considered as unfair and still needed to be explored more.[9],[10],[11]
Although the application of WPBA tools in the Indian context was perceived as feasible by the participants, actual practice can be enhanced by making appropriate choices of tools, advance planning, building mutual trust, and training assessors. Another study showed a high level of satisfaction among trainers and trainees, indicating successful integration of WPBA tools in an Asian postgraduate training program despite systemic, cultural, and language barriers.[12]
The challenges perceived during the effective practice of WPBA were highlighted in the study along with the solutions. Similarly, few other studies had revealed that CBME was viewed as a positive change in the light of WPBA, though there was a huge gap. The barriers to implementation of CBME and WPBA included lack of training in assessment and feedback, financial implications, low familiarity with WPBA tools, lack of proactiveness, and time constraints.[13],[14]
The present study depicts that the pandemic has adversely affected the process of WPBA, and simulation can just act as an adjunct. Similar findings were noticed in another study where longitudinal coaching, replacing a few training experiences with simulation, a strategic pairing of residents and faculties along with using assessment tools that quantify a faculty member's entrustment were suggested as the answer to the uncertainties of workplaces during the pandemic.[15]
| Conclusions | | |
Practice of WPBA tools at workplace was perceived positively by all faculties in terms of feasibility and availability.
Although participants were aware there is a large gap between knowledge and practices of WPBA tools attributed to challenges in the Indian scenario, which were more during the pandemic suggesting innovation, simulation, and appropriate use of WPBA tools.
Limitations
A small sample size (38) was taken, and the survey was conducted online, in a small-time frame.
Acknowledgement
We sincerely acknowledge the guidance of Dr. Dinesh Badiyal, Dr. Monika Sharma, Dr. Jyoti Nath Modi, Dr. Aroma, and the support of other faculties, of CMC-Ludhiana, a regional center for Fellowship in Advanced International Medical Education and Research (FAIMER program) for being the constant source of inspiration during the study. Furthermore, we acknowledge all the faculties and fellows who participated in the study and consented to publication.
Financial support and sponsorship
Source(s) of support: FAIMER CMC- Ludhiana, Regional Institute.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | |
| 2. | |
| 3. | |
| 4. | Norcini J, Burch V. Workplace-based assessment as an educational tool: AMEE guide no. 31. Med Teach 2007;29:855-71.  |
| 5. | Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach 2010;32:676-82. |
| 6. | Challis M. AMEE medical education guide No. 11 (revised): Portfolio-based learning and assessment in medical education. Med Teach 1999;21:370-86. |
| 7. | |
| 8. | Singh T, Sood R. Workplace-based assessment: Measuring and shaping clinical learning. Natl Med J India 2013;26:42-6. |
| 9. | Parry-Smith W, Mahmud A, Landau A, Hayes K. Workplace-based assessment: A new approach to existing tools. Obstet Gynaecol 2014;16:281-5. |
| 10. | Nesbitt A, Baird F, Canning B, Griffin A, Sturrock A. Student perception of workplace-based assessment. Clin Teach 2013;10:399-404. |
| 11. | Joshi MK, Singh T, Badyal DK. Acceptability and feasibility of mini-clinical evaluation exercise as a formative assessment tool for workplace-based assessment for surgical postgraduate students. J Postgrad Med 2017;63:100. [ PUBMED] [Full text] |
| 12. | Tan J, Tengah C, Chong VH, Liew A, Naing L. Workplace based assessment in an Asian context: Trainees' and trainers' perception of validity, reliability, feasibility, acceptability, and educational impact. J Biomed Educ 2015;2015:1-8. |
| 13. | Tannenbaum E, Furmli H, Kent N, Dore S, Sagle M, Caccia N. Exploring faculty perceptions of competency-based medical education and assessing needs for implementation in obstetrics and gynaecology residency. J Obstet Gynaecol Can 2020;42:707-17. |
| 14. | Wangchuk P, Tenzin K, Tshering S, Zangmo S. Evaluation of work place based assessment implementation in postgraduate medical education at a medical university in Bhutan. Bhutan Health J 2020;6:12-8. |
| 15. | Kealey A, Alam F, McCreath G, Matava CT, Bahrey LA, Walsh CM. Real-world impact of the COVID-19 pandemic on the assessment of anaesthesiology residents. Br J Anaesth 2020;125:e430-2. |
[Figure 1], [Figure 2]
[Table 1], [Table 2]
|
