Emergency Medicine In-Training Examination Scores are Not Associated with Burnout and Not Affected by the Introduction of a Wellness Curriculum

reported for 296/383 (77.5%) residents for 2017 and 304/386 (78.8%) residents for 2018. There was no association between change in mean EM-ITE scores at the intervention sites compared to the control sites. In the subset of 172 residents who completed the 2017 and 2018 MBI, there was no correlation between burnout and changes in EM-ITE scores. Conclusion: In this study of EM residents, burnout was not associated with resident medical knowledge acquisition and change in EM resident medical knowledge was not affected by the introduction of a wellness curriculum. ABSTRACT


INTRODUCTION
Consequences of resident well-being and burn-out are well documented, including effects on psychological health, career satisfaction, and even patient outcomes [1][2][3][4].Yet it is unclear if the negative consequences of burnout extend to acquisition of medical knowledge.
The Emergency Medicine In-Training Examination (EM-ITE) is a tool to assess EM residents' medical knowledge.Performance on the EM-ITE is one of the sole metrics shown to predict likelihood of passing the American Board of Emergency Medicine qualifying examination [5].Other methods of assessment, including direct observation and conference attendance, have not been found to correlate with objective measures of medical knowledge [6].Educators struggle to accurately appraise the medical knowledge and skills of their residents [7][8].
Previous investigations evaluating the relationship between the acquisition of medical knowledge and measurements of burnout have yielded inconclusive results [9][10].To our knowledge, no studies have assessed whether wellness curricular innovations aimed at improving resident burnout can also affect knowledge acquisition and scores on standardized medical examinations.
The objective of this study was to determine if EM resident performance on the EM-ITE is associated with burnout.We also sought to determine if the implementation of a wellness curriculum is associated with a change in resident medical knowledge acquisition.

Study Design
This study was a component of a larger, multi-institution prospective education intervention trial conducted at 10 ACGME-accredited EM residency programs in the United States [11].At all sites, members of the Emergency Medicine Education Research Alliance (EMERA) were core faculty when the study was initiated.Prior to the initiation of the study, all sites received IRB approval at their respective institutions.

Subjects
PGY1-4 EM residents at the participating programs during the study period of February 2017 through February 2018 were eligible to participate.Participation in the study was voluntary.Informed consent was obtained from all subjects.

Study Protocol Survey Instrument:
Eligible participants at all sites were sent the survey instrument at three different time points during the study: February 2017, August 2017, and February 2018.The survey was administered either on paper or via online proprietary software (SurveyMonkey) per the preference of the study site leader.Follow-up for nonresponders was program-specific, either in-person or via email.
The survey instrument was designed for completion in 15 minutes and consisted of 34 questions including demographic information as well as several tools used in physician wellness research.These included the Maslach Burnout Inventory, a widely utilized and well-recognized tool in burnout assessment that examines the domains of emotional exhaustion, depersonalization, and personal accomplishment [12].The survey instrument also included the Primary Care Evaluation of Mental Health Disorders Patient Health Questionnaire 2 question screen (PRIME-MD PHQ-2) and three additional published wellness assessments: a quality of life instrument, a rating of career satisfaction, and an appraisal of work life balance [13][14][15][16].
The EM-ITE is a standardized online test consisting of 225 multiple choice questions that is administered at all EM residencies in the United States annually during the last week of February.The EM-ITE is considered a valuable tool in the assessment of EM resident medical knowledge.A higher score on the ITE is associated with a greater likelihood of passing the American Board of Emergency Medicine written qualifying examination [5,17].

Curriculum Intervention:
Of the 10 participating sites, five self-selected into the intervention group based on resources necessary to institute the multi-faceted wellness curriculum and the other five sites comprised the control group.The control sites agreed not to introduce new wellness initiatives during the study period.A year-long comprehensive wellness curriculum was introduced at the five intervention sites in March 2017 and concluded in February 2018.This multifaceted curriculum and its detailed components have been previously published [18][19].In summary, the curriculum included standardized bimonthly didactic lectures, individualized interactive instruction (III) assignments, additional internet-based resources, and mobile application software [18].

Analysis
In addition to administering the MBI and other assessments, basic demographic information was obtained including respondent age, gender, ethnicity, and PGY classification.
Descriptive statistics are presented as total number (n) and percentages with 95% confidence intervals for categorical variables.Continuous variables are displayed as either means with standard deviation for normally distributed variables or as medians with interquartile ranges (IQR) for non-normally distributed variables.Univariate analyses were performed using chi-square, or Student's t-test as appropriate for continuous or categorical variables.Linear regression was performed to assess the relationship between change in ITE score and burnout while controlling for age, gender, ethnicity, quality of life, level of support, career satisfaction, and depression screening.Analysis was performed using a statistical package program (R version 3.3.1).
Consistent with Dr. Maslach's definition, burnout was defined as both emotional exhaustion >26 and depersonalization > 12 [20].In this study, burnout ranged from 17.1% to 26.8% (Table 2).The only statistically significant difference between the intervention and control sites was a slightly higher mean EM-ITE score at the control sites for the PGY1 residents in 2017 (Table 3).There were no significant changes in mean ITE scores from 2017 to 2018 between the intervention and control groups (Table 4).There remained no significant differences between the control and intervention site EM ITE scores when controlling for age, gender, ethnicity, quality of life, level of support, career satisfaction, or depression screening using linear regression analysis.
The subset of 172 residents who completed both the Feb 2017 and Feb 2018 ITE was well-matched to the group that did not have an ITE score reported for both years.The median age for those who completed both ITEs was 28 (IQR: 27-30), 33% women (CI 26.7-36.3%),and 5.7% (3.3-8.2%)who identified as under-represented minority in medicine (URIM).For the group that did not have both 2017 and 2018 scores reported, the median age was 30 (IQR: 28-32), there was 31% women (CI 26.7-36.3%),and 7.6% (CI: 5.4-10.1%)identified as URIM.As well, there was no correlation between burnout and changes in ITE score.There was also no correlation between any individual component of burnout and change in ITE score.For emotional exhaustion, the Pearson's coefficient was 0.019 (95% CI -0.17-0.21).For depersonalization, the Pearson's coefficient was 0.78 (95% CI -0.11-0.26).For personal accomplishment, the Pearson's coefficient was 0.05 (95% CI -0.14-0.23).In a linear regression analysis of this group of residents, there was no association between components of burnout and change in ITE scores when controlling for the previously mentioned variables.

DISCUSSION
In this year-long national study, burnout did not appear to affect the development of medical knowledge in a cohort of EM residents, as evidenced by the lack of association between burnout and change in ITE score.In addition, EM resident medical knowledge acquisition was not affected by the introduction of a multi-faceted wellness curriculum.This study represents the first EM multi-center educational intervention trial to assess the effects of resident burnout on change in medical knowledge.
While the associations are well-established between physician burnout, physical health, mental health, as well as effects on patient care, the relationship between resident well-being and medical knowledge has not been well-delineated [1][2][3][4].Medical knowledge, an ACGME core competency, is essential to the delivery of appropriate clinical care.The authors hypothesized that burnout could negatively affect the acquisition of medical knowledge, especially since emotional exhaustion relates to feelings of being emotionally overextended and exhausted by one's work, potentially affecting knowledge retention or one's willingness to study outside clinical hours.
Stressful aspects of the medical student learning environment have been related to burnout [21].A study of internal medicine (IM) residents concluded that resident well-being did not correlate with medical knowledge as assessed by web-based courses or the IM-ITE and a study of family medicine residents determined no relationship between self-reported burnout and medical knowledge as measured by ITE performance [9,22].In addition, a national study of pediatrics residents demonstrated that PGY1 residents who were positive for burnout performed lower in all competency assessments except medical knowledge [23].However, in a more recent study involving three quarters of all IM residents training in the United States from 2008-2009, increased frequency of burnout symptoms was associated with lower scores on the IM-ITE.This was particularly true for those residents experiencing high emotional exhaustion and low quality of life [10].Our study found no correlation with the components of the MBI and the EM-ITE score.
One confounder that may help explain the lack of correlation between burnout and EM-ITE scores in our study is the role that stress plays in learning.Previous studies have demonstrated that stress can either augment or impede learning depending on the type and severity [24,25].While high levels of stress have been correlated with worse academic performance, stress has also been shown to benefit memory, speed up processing times, enhance motivation and increase work effort [25][26][27].Emotional exhaustion and stress are significantly associated and have been demonstrated to be causative factors to one another [28].Future research on burnout and medical knowledge specifically aimed at considering the stress level of subjects at the time of study would be beneficial.
There are several additional factors that may have contributed to our findings.Our study population reported lower baseline burnout rates of 17.1 to 25.9% compared to previously published studies that determined emergency medicine burnout rates to be some of the highest among all specialties at 59% [29].This may be at least partially due to the fact that we defined burnout according to Dr. Maslach's definition of having both   an emotional exhaustion score >26 and depersonalization score > 12, rather than elevation of only one of the domains [20].In addition, it is becoming increasingly clear that burnout exists as a continuum and discrete cutoffs are likely not the most accurate way to describe burnout.Also, during the study period, physician wellness became an increasing focus within medical education and there was a significant increase in literature, podcasts, and national initiatives promoting physician wellness.While the control sites agreed not to introduce new programmatic wellness initiatives during the study period, resident exposure to the topic by other means may have influenced our results.We were also unable to control for whether programs introduced new qualifying examination study materials from 2017 to 2018.In addition, there is now increased understanding of systems effects on burnout and the contribution of the learning environment to resident well-being.Our curriculum was directed towards program and personal interventions and did not address systems or institutions.Finally, participation in the varying elements of the curriculum was highly encouraged but not mandated, which led to variable compliance [19].
There are several important limitations to our study.The 10 participating sites comprised a convenience sample of residents and the results were not subject to power analysis.Additionally, due to program-specific resources, the participating sites self-selected into the intervention and control groups.All programs continued wellness initiatives that were already in place before introduction of the curriculum, which may have affected the results of the study.Finally, we used the EM-ITE to measure resident medical knowledge.While the EM-ITE is an assessment tool for medical knowledge, other factors beyond burnout may influence resident performance on the EM-ITE.

CONCLUSION
In this study of EM residents, burnout was not associated with the acquisition of medical knowledge as measured by the EM-ITE.In addition, change in EM resident medical knowledge was not affected by the introduction of a multi-faceted wellness curriculum.
RESULTSA total of 285/382 (75%) residents participated in the February 2017 data collection; 247/386 (64%) residents participated in August 2017; and 228/386 (59%) participated in February of 2018.EM-ITE scores were reported for 296/383 (77.5%) residents for February 2017 and 304/386 (78.8%) residents for February 2018.A subset of 172 residents completed the February 2017 and February 2018 MBI as well as reported both 2017 and 2018 EM-ITE scores.There were no significant differences in age, gender, ethnicity, or PGY training year distribution between the control and the intervention sites (Table

Table 1 :
Demographics of Respondents

Table 2 :
Proportion of Respondents Screening Positive for Global Burnout by Study Group *as of February 2017

Table 3 :
Mean ITE Scores

Table 4 :
Mean Change in ITE Score 2017 to 2018 *as of February 2017