The Burnout Assessment Tool (BAT): A Contribution to Italian Validation with Teachers’

Abstract

This study aims to validate the Burnout Assessment Tool (BAT) adapted to the Italian education sector. Teacher burnout is physical and emotional pain, due to prolonged exposure to school-related stress factors. Previous research has abundantly proven that preventive assessment of teachers’ risk level for burnout may reduce adverse outcomes. In this regard, new assessment tools, able to bring together evidence from fifty years of research on this topic, were mainly used to monitor burnout-risk levels in the school context. For the present work, 846 Italian teachers (Female, 91.1%; M age = 47.52; SD = 9.94) were involved in the study. Confirmatory factor analysis supported a four-factor structure for the core dimensions (BAT-C; exhaustion, mental distance, emotional impairment, cognitive impairment), and a two-factor structure for the secondary dimensions (BAT-S; psychological distress, psychosomatic complaints). The Italian version of the BAT-C and BAT-S has shown good internal consistency (respectively, α = 0.900 and ω = 0.913; α = 0.845 and ω = 0.857) and validity (all correlations between variables showed a p value < 0.01). Our findings support the Italian adaptation of the original version of the BAT as a valid instrument for measuring teachers’ burnout through principal and secondary symptoms.

1. Introduction

1.1. Teachers and School-Related Burnout

Teacher burnout is the experience of physical and emotional pain towards job tasks, characterized by a loss of positive attitude, engagement, and commitment to one’s work [1]. Due to a prolonged exposure to school-related stress factors, teachers feel unable to face job demands [2]. Recently, Skaalvik and Skaalvik defined teacher’s stress as follows: “(it) is typically conceptualized as unpleasant emotions resulting from aspects of the work as a teacher” ([3], p. 1251). Since the early investigations on teachers’ burnout, this topic has received considerable attention globally, becoming a central element in the latest teacher assessment and learning survey [4]. In the final report, teachers’ stress emerged as a pervasive phenomenon among European countries, impacting the working environment in several manners.

Today, there is broad agreement regarding the effect of teachers’ burnout on their psychological and subjective well-being [5]. Several events may enhance teachers’ risk of the burnout experience. Among these, more common results referred to the following sources: students misbehavior, students with special education needs, conflicts with colleagues, time pressure, and workload [6,7,8,9]. Simultaneously, scholars have investigated whether and to what extent teachers’ burnout impacts students’ school adjustment [10]. Findings have shown that classroom climate, and student motivation and achievement were significantly affected by teachers’ burnout (e.g., [11]). Additionally, burned-out teachers show low performance, low job satisfaction, high risk of illness, and absenteeism (e.g., [12]). Furthermore, burned-out teachers report psychosomatic symptoms and cognitive impairments, which, in turn, significantly impact their general well-being [13].

Likewise, in recent decades, research focused on what factors may boost teachers’ resources (i.e., personal and school-related) to support their emotional exhaustion. Resilience, emotional intelligence, instructional practices, cooperative work-related behaviors, and adaptive coping strategies were the core of this approach (e.g., [14,15,16,17,18]). In this regard, many training programs for pre-service and in-service teachers have been developed and proven to reduce burnout risk [19,20,21]. In addition, a fundamental advancement in burnout research refers to the development of assessment tools that may reveal several levels of burnout risk associated with a myriad of personal and professional variables. The most studied variables have been: teachers’ age, gender, career, and education level (e.g., [22,23,24,25]). However, even considering individual differences (e.g., [26]), cross-culture studies support a shared idea that school-related burnout is a phenomenon profoundly linked to the helping professions, such as teaching (e.g., [27]).

Finally, extensive research on burned-out teachers revealed high social costs on at least three levels: the personal and psychological, the working environment, and the social and economic [4]. Furthermore, in the last two years, teachers faced new and challenging tasks regarding their practice, digital skills, and their relationships with students, which dramatically changed due to the transition from face-to-face to distant learning. Consequently, the European and various national policy agendas have focused on teachers’ well-being at work [13]. The central core of this perspective is the promotion of teaching sustainability. The more good practices in human resource management are employed to assure the health of school staff, the more teachers may efficiently serve as educators, which positively impacts student well-being. In this regard, it is expected that scholars, practitioners, and policymakers maintain interest in updated and clever tools to continue monitoring teachers’ burnout levels.

1.2. Two Assessment Generations

Two main instruments have been developed for teaching professions in the last fifty years: the Maslach Burnout Inventory Educator Survey (MBI-ES, [28]) and the Copenhagen Burnout Inventory (CBI, [29]). Even though the second is less frequently used than the MBI survey, the two tools respond to different concepts of burnout assessment, which deserves a deeper analysis.

The MBI-ES is considered the gold standard for evaluating occupational burnout [30,31]. Inspired by this tool, a myriad of other self-report questionnaires have been developed around the globe (see, for a review, [32]). However, the original version of the MBI remains ubiquitous in empirical research. It comprises three independent sub-scales, to evaluate emotional exhaustion, depersonalization (recently renamed cynicism), and reduced personal accomplishment (recently renamed professional efficacy) [2]. The first refers to negative emotions towards job tasks; the depersonalization sub-scale refers to the detachment from context, other people, and the working environment, in general. Finally, reduced-accomplishment sub-scale addresses the extent to which teachers feel inadequate to face challenging school–life events. However, the three-factor structure of the MBI-ES is far from definitively confirmed by evidence, as recently underlined by some scholars [33]. According to Kristensen and colleagues, depersonalization and personal accomplishment are erroneously incorporated in occupational burnout assessment [29].

Contrarily, findings worldwide have supported the central core of the emotional-exhaustion dimension of the MBI [34]. The emotional dimension of burnout captures the feeling of being overwhelmed, emotionally impaired, and depressed because of work routine [2]. Furthermore, with specific regard to teachers, several studies have confirmed that the earlier symptoms of burnout were those compounding within the emotional exhaustion scale, as also revealed by long-term research [35,36]. To sum up, findings of teachers’ emotional exhaustion report this dimension to be: (a) the first signal of teachers’ maladaptive feelings in the school context; (b) the predictive factor of teachers’ later experience of cynicism and reduced personal accomplishment; and (c) frequently correlated with other positive and negative dimensions (e.g., self-efficacy and absenteeism) [26,37].

The CBI is a recent self-report questionnaire, translated into several languages, and adopted to evaluate teachers’ burnout risk [29,38]. It consists of three sub-scales, which focus on emotional exhaustion, fatigue in maintaining positive relationships, and well-adjustment, in three contexts, private life, relationships with students, and school-related tasks. The personal burnout dimension addresses physical and psychological weakness and emotional exhaustion experienced by teachers in daily life, independently from their work environment. The great advantage of this sub-scale is the bringing together of two pieces of evidence from burnout research. First, evaluations of teachers’ experience of burnout as a generic and context-free occurrence, as it leads to understanding emotional exhaustion in different dimensions of life other than those related to work related [39]. Second, monitoring the intersecting effects between burnout and depression symptoms may account for the overlapping influence between the two experiences of suffering (e.g., [40]). The other two CBI dimensions specifically refer to the school context. The work-related burnout sub-scale aims to evaluate emotional and physical exhaustion in the working environment. Finally, with student-related burnout, the emphasis is on the teachers’ fatigue at maintaining positive and constructive relationships with their students.

The fundamental approach of the CBI is to overcome limits resulting from the extensive use of the MBI. More specifically, earlier research has underlined that reduced personal accomplishment clusters in an independent dimension of burnout assessment, as an effect of teachers’ state of suffering in their work context [41,42]. Other studies supported accounting for teachers’ depersonalization or cynicism as a coping strategy to protect themselves during emotionally challenging events [29]. Evidence concerning emotional exhaustion as the central core of the MBI tool led several scholars to use this sub-scale as the prominent and, indeed, only assessment of teachers’ burnout (e.g., [43,44]). Secondly, the CBI approach addressed the need to analyze emotional exhaustion related to the school context separately from that experienced in the life contexts, or of other workers.

To sum up, Shoman and colleagues have found that both the MBI and CBI tools have shown the best psychometric validation indexes compared with other existing occupational burnout measures [45], confirming that these remain two valid instruments to assess occupational burnout.

1.3. A New Complex Tool: The BAT

The Burnout Assessment Tool (BAT, [46]) was developed to bring together recent evidence on the complex picture of workers’ burnout experience. More specifically, the authors of the BAT underlined three flaws with the MBI. First, the fact that MBI does not include reduced cognitive performance, such as diminished concentration, attention, and working memory. As reported by Deligkaris and colleagues [47], reduced cognitive performance is indeed attributable to burnout.

Second, the answers to some questionnaire items (e.g., “I don’t really care what happens” from some recipients” and “I feel that I am treated as if I were an impersonal object” from others) are misrepresented, thus lessening the reliability of the MBI [32]. Specifically, participants are more likely to tick “sporadically” or “never”, instead of “very often” or “always” due to the extreme wording of these items. More among these psychometric problems, items associated with personal accomplishment are positively worded, whereas depersonalization and exhaustion scales are negatively worded. As a consequence, a low professional efficacy score underscores burnout syndrome. Moreover, the correlations between exhaustion and depersonalization are usually higher than the association between either the former or the latter and professional efficacy [48]. On the other hand, the correlations are higher when accomplishment is measured through negatively worded items [49].

Furthermore, the results reported by De Beer and Bianchi [50] suggest a two-factor model, such that the first factor combined depersonalization and exhaustion, and the second included personal accomplishment factors. According to the authors, the two-factor model best fit the data. This finding, in turn, brought into question the role of personal accomplishment in the burnout construct. These results are consistent with those obtained by Maslach and colleagues [51], as exhaustion and cynicism are negative factors, whereas personal accomplishment is positive.

Third, the MBI does not provide a single burnout score because it was not designed to identify people with acute burnout symptoms, but rather peoples’ connection with their profession at various levels [52]. Moreover, cut-off values have not been clinically validated in most countries [53], thus preventing the employment of this questionnaire as a personal assessment tool for clinical diagnosis. Hence, the MBI cannot be used to differentiate burned-out from non-burned-out subjects.

To overcome these shortcomings, Schaufeli and colleagues [54] developed the BAT, a questionnaire consisting of four core subdimensions: exhaustion, characterized by a loss of energy that is affective both mentally and physically; emotional impairment, referring to strong emotional reaction and a sense of being overwhelmed by one’s feelings; cognitive impairment, including both issues in remembering information and a lack of attention; and, finally, mental distance, characterized by work reluctance due to psychological distance from one’s work.

Likewise, the BAT includes two secondary subdimensions: first, psychological complaints, refers to the non-physical consequences of psychological problems. Second, psychosomatic complaints are physical problems (e.g., palpitations, headaches, and sickness) caused or intensified by psychological issues. Including all of these dimensions in a unique instrument allows researchers and practitioners to simultaneously assess the emotional, psychological, and psychosomatic dimensions involved in the burnout experience. Furthermore, a self-report questionnaire is a valuable tool for school staff, involving periodic intervention to monitor the education sector’s well-being and prevent illness.

The BAT, developed by Schaufeli and colleagues [46], showed the best fit with a second-order model, in which the primary and secondary symptoms were separated from each other. The fit indexes were acceptable (CFI = 0.91; TLI = 0.90; RMSEA = 0.06), the Cronbach’s alphas were higher than 0.90, and convergent and discriminant validity were confirmed by assessing the correlations between the dimensions of the BAT, work engagement, workaholism, and job boredom.

Burnout is considered an occupational disease in Belgium and the Netherlands, where the BAT has been extensively investigated [55]. Whereas, in Italy, burnout isn’t on the list of occupational diseases [55,56], but is listed by the National Institute Against Accidents and Occupational Diseases (INAIL) among its “mental and behavioural disorders”. Furthermore, there is no validated tool that allows measuring, with certainty, the clinical values of a burnout syndrome. This underscores the need to develop an Italian version of the BAT, thereby assessing the onset of burnout symptoms in the Italian school context.

Over the past year there has been a growing number of studies in which BAT was used to assess the onset of burnout symptoms (e.g., [57]). De Beer and colleagues [58] offered a comprehensive overview of the BAT across seven nations (i.e., Belgium, Netherlands, Austria, Germany, Finland, Ireland, and Japan). They found that it can be used to evaluate burnout symptoms internationally. The Japanese version of the BAT has recently been validated by Sakakibara and colleagues [59], showing the same factorial structure, acceptable fit indexes, good reliability, and adequate discriminant and convergent validity as the original version. The worldwide spread of BAT underscores the need for a questionnaire that can be used as a burnout screening tool in occupational settings, given the detrimental role of burnout for both organizational effectiveness and employee health. The current study aims to validate the Italian version of the BAT by assessing its psychometric characteristics.

1.4. Aim and Hypotheses

This study aimed to validate the Italian version of the Burnout Assessment Tool (BAT), developed by Schaufeli and colleagues [46], with a sample of Italian teachers. The current study’s central aim was to confirm the BAT’s structural factors, assuming the six subscales to also have good fit indexes, in the Italian context, for both models tested (Hypothesis 1). Second, the authors expect both the BAT subscales and the global score to confirm satisfactory internal reliability of the BAT, with cut-off values of at least 0.70 [60], for both Cronbach’s alpha [61] and McDonald’s omega [62] (Hypothesis 2). Third, the study aims to assess the criterion validity of the BAT, assuming that each of its subscales will positively correlate with the emotional exhaustion subscale of the MBI (Hypothesis 3). Furthermore, the study aims to assess the discriminant validity of the BAT, assuming that each of its subscales will correlate negatively with measures of well-being (WHO-5) and engagement (UWES-3) (Hypothesis 4).

2. Materials and Methods

2.1. Participants

The study involved 846 Italian teachers, of an age range of 19 to 68 years (M age = 47.52 years, SD = 9.94), of whom 75 were men (8.9%) and 771 were women (91.1%). The criteria for inclusion in the study were that the participants be Italian teachers and voluntarily agreed to participate.

2.2. Procedure

The present cross-sectional study was conducted in Italy between March and April 2021, using convenience sampling. The data was collected using an online gathering method, as part of a teacher training-course internship. The teachers participating in the internship collected data from colleagues in their schools. Participants were informed of the research objectives and gave their informed consent for procedures for gathering and processing data before participating in the study. Teachers were assured that participation was voluntary and that their responses would remain anonymous. This study, indeed, was conducted under the requirements of privacy and informed consent laid down by current Italian law (Law Decree DL-196/2003). The research project was accepted by the Ethics Committee for Scientific Research (CERS) of LUMSA University, and the study was conducted under the Declaration of Helsinki.

2.3. Measures

Burnout Assessment Tool (BAT): the BAT [46] consists of the BAT-C and BAT-S. The BAT-C includes 23 items, measuring four core dimensions of burnout: exhaustion (eight items), mental distance (five items), emotional impairment (five items), and cognitive impairment (five items). The BAT-S includes ten items, measuring secondary symptoms, namely: psychological distress (five items) and psychosomatic complaints (five items). All items were scored on a five-point Likert scale, ranging from “never” (1) to “always” (5). Responses were summed and averaged for each subscale; scoring ranged between one and five. Total BAT scores can assess level of burnout, while independent scores on its six dimensions (core symptoms and secondary symptoms) can provide more information. In this study α = 0.924, and the Italian translation of the measure was borrowed from the official BAT website (https://burnoutassessmenttool.be/project_eng/, accessed on 15 February 2021).

Maslach Burnout Inventory—Educators Survey (MBI-ES): the MBI-ES [28] was used to confirm the BAT’s convergent and discriminant validities. MBI is a self-report questionnaire, consisting of 22 items that evaluate burnout on a seven-point rating scale. MBI subscales include emotional exhaustion (EE; nine items), depersonalization (DP; five items), and personal accomplishment (PA; eight items). All items were scored on a seven-point Likert scale ranging from “never” (0) to “every day” (6). Responses were summed and averaged for each subscale. For this study, only the emotional exhaustion (EE) scale was used. This scale describes the feeling of being exhausted by the job and is evaluated with a score ranging from 0 to 54. In this study, participants completed the Italian version of the MBI for social–educational services [63]. In this study α = 0.926.

World Health Organization Well-Being Index (WHO-5): The World Health Organization’s Five Well-Being Index (WHO-5; [64]) is a self-report questionnaire for the global assessment of well-being [65]. The WHO-5 includes five items, each rated on a six-point scale, from “all of the time” (0) to “at no time” (5). Therefore, total scores range from 0 (absence of well-being) to 25 (maximal well-being), with high scores indicating an increased sense of well-being. The authors chose the WHO-5 as a convenient tool, having already used it to investigate the well-being of workers in the Italian context [66]. In this study α = 0.890.

Utrecht Work Engagement Scale (UWES-3): The Utrecht Work Engagement Scale (UWES-3; [67]) is a short scale for assessing engagement. The UWES-3 is composed of three items, each evaluating a different factor: (1) “At my work, I feel bursting with energy” (vigor); (2): “I am enthusiastic about my job” (dedication); (3) “I am immersed in my work” (absorption). Each item is rated on a seven-point Likert scale from “never” (0) to “always” (6). Alphas were acceptable (>0.70) in five national samples of the original study. In this study α = 0.838.

Additionally, a set of questions was administered to obtain sociodemographic and occupational information. The following variables were assessed: age, gender, level of education, school level, and the years of service.

2.4. Data Analysis

First, the authors performed an item analysis using SPSS 27 [68] to investigate the items’ psychometric properties (mean, standard deviation, skewness, and kurtosis) to support the robustness of the analyses. Regarding the interpretation of asymmetry and kurtosis values, the authors considered values | 1 | (e.g., [69,70]), | 2 | (e.g., [60]) and | 3 | (e.g., [71]) as optimal. Furthermore, the scientific literature [72] indicates | 7 | as the maximum acceptable cut-off of kurtosis in a sample size greater than 300 subjects [73].

Second, in line with recommendations for measures development [71], we used confirmatory factor analysis (CFA), using Mplus 8.3 [74], to evaluate the factorial validity of the BAT. CFA is a form of psychometric assessment used in cases in which the factor structure of a measure has already been previously assessed, and researchers need to test the number of factors, their relations, and the loadings of indicators on a different sample [75]. The degree of overlap between the “real” matrix and the reproduced matrix of covariance of the a priori hypothesized structural model is compared [73], to evaluate the goodness of fit of the indexes. This strategy of analysis was based on a multistage approach. We chose to compare the two models that fitted the data significantly better in the original study [46]: Model 6, which, of all models, fitted the data significantly better, and Model 8, which was closest to the theoretical framework for which both core and secondary symptoms refer to burnout as an underlying psychological condition. Model 6 is a correlated factor model and assumes that six latent factors can be distinct. Whereas Model 8 is a second-order model and assumes that the four core factors (exhaustion, mental distance, emotional impairment, and cognitive impairment) are best represented by a single general factor (the core of burnout). In comparison, the other two factors (psychological distress and psychosomatic complaints) are best captured by a second general factor (secondary symptoms). For the Italian version of the BAT, Model 1 (M1) was a correlated six-factor CFA model with six different components (i.e., exhaustion, mental distance, emotional impairment, cognitive impairment, psychological distress, and psychosomatic complaints) were correlated, thus reproducing Model 6 in the original validation paper. Model 2 (M2) of the Italian version instead was a second-order CFA model, in which burnout is considered a syndrome comprising a core dimension and a secondary-symptoms dimension, thus reproducing Model 8 in the original validation paper. In line with recommendations regarding testing dimensionality with CFA [76,77,78], the fit indices of the two models were evaluated using systematic fit assessment procedures. Such techniques include the chi-square test of exact fit (χ²), comparative fit index (CFI; ≥0.95 for good; ≥0.90 for acceptable; [77,79,80,81]), Tucker–Lewis index (TLI; ≥0.95 for good, ≥0.90 for acceptable; [77,79,80,81]), standardized root mean square residual (SRMR; ≤ 0.05 for good, ≤ 0.10 for acceptable; [77,79,80,81]), the root mean square error of approximation (RMSEA; ≤0.06 for good, ≤0.08 for acceptable; [77,79,80,82,83,84,85] with its 90% confidence interval ([77,86], Akaike information criterion (AIC, [87]), Bayesian information criterion (BIC, [88]) and sample size-adjusted Bayesian information criterion (SAMPLE-ADJ BIC, [89]) (when comparing two or more models, the best model is the one with the lowest information criteria, [90]).

Third, the authors verified the internal consistency reliability, i.e., the degree to which the set of items in the scale covary, relative to their sum score, was calculated [91,92,93]. For this aim, Cronbach’s alpha (α, [61]) and McDonald’s omega (ω, [62]) coefficients were used. Both coefficients are interpreted according to the following cutoff values [60]: excellent, α ≥ 0.9, good, α ≥ 0.8, acceptable, α ≥ 0.7, questionable, α ≥ 0.6, poor, α ≥ 0.5, and unacceptable, α ≤ 0.5. Finally, the construct validity was evaluated via convergent and discriminant validity, confirming that the BAT was related to other measures. Figure 1 shows a graphical representation of the two models (M1 and M2).

3. Results

The participants in the study were 846 teachers (91.1% female,) aged between 19 and 68 years old (M = 47.52, SD = 9.94).

Table 1. Demographic characteristics of the study participants (N = 846).

	M ± SD	n	%
Age (years)	47.52 ± 9.94
Gender
women		771	91.1
men		75	8.9
Education
high School		227	26.8
graduation		346	40.9
post-graduate specialization		273	32.2
Working school level
kindergarten		136	16.08
primary school		408	48.22
lower secondary (middle) school		201	23.76
high school		101	11.94
Years of service	16.74 ± 11.18

Note: M, mean; SD, standard deviation.

shows the respondents’ characteristics. About 73% of participants held a university degree or postgraduate qualification, while about 27% had only a high school diploma.

Participants were employed in kindergartens (16.08%), primary schools (48.22%), lower secondary (middle) schools (23.76%), and high schools (11.94%). The years of service ranged from 0 to 50 (M = 16.74, SD = 11.18).

Table 2. Exploratory descriptive statistics of items included in the BAT (N = 846).

	Skewness		Kurtosis
	Statistic	Standard Error	Statistic	Standard Error
BAT_E_1	−0.022	0.084	−0.389	0.168
BAT_E_2	0.016	0.084	−0.467	0.168
BAT_E_3	0.047	0.084	−0.615	0.168
BAT_E_4	0.351	0.084	−0.492	0.168
BAT_E_5	0.762	0.084	0.124	0.168
BAT_E_6	0.908	0.084	0.280	0.168
BAT_E_7	0.639	0.084	−0.177	0.168
BAT_E_8	0.117	0.084	−0.480	0.168
BAT_MD_1	1.162	0.084	0.870	0.168
BAT_MD_2	1.374	0.084	0.054	0.168
BAT_MD_3	1.892	0.084	1.762	0.168
BAT_MD_4	2.919	0.084	6.745	0.168
BAT_MD_5	0.508	0.084	−0.426	0.168
BAT_CI_1	0.932	0.084	0.635	0.168
BAT_CI_2	0.281	0.084	−1.754	0.168
BAT_CI_3	0.959	0.084	0.539	0.168
BAT_CI_4	1.062	0.084	0.857	0.168
BAT_CI_5	0.986	0.084	0.341	0.168
BAT_EI_1	0.901	0.084	0.593	0.168
BAT_EI_2	0.412	0.084	−1.715	0.168
BAT_EI_3	0.742	0.084	0.233	0.168
BAT_EI_4	1.320	0.084	−0.143	0.168
BAT_EI_5	1.375	0.084	1.146	0.168
BAT_PD_1	0.899	0.084	0.021	0.168
BAT_PD_2	0.197	0.084	−0.561	0.168
BAT_PD_3	0.247	0.084	−0.444	0.168
BAT_PD_4	1.317	0.084	1.009	0.168
BAT_PD_5	0.822	0.084	−0.027	0.168
BAT_PC_1	0.895	0.084	−1.069	0.168
BAT_PC_2	0.907	0.084	−0.154	0.168
BAT_PC_3	0.461	0.084	−0.695	0.168
BAT_PC_4	0.171	0.084	−0.904	0.168
BAT_PC_5	1.016	0.084	0.910	0.168

Note. BAT_E, exhaustion; BAT_MD, mental distance; BAT_CI, cognitive impairment; BAT_EI, emotional impairment; BAT_PD, psychological distress; BAT_PC, psychosomatic complaints.

shows the descriptive findings.

The items show acceptable skewness and kurtosis values. Only one item, which reported a kurtosis value of 6.745, presented a minor violation to the most rigorous cutoff of | 3 | [71]. This value is to be considered acceptable given the sample size [72]. Therefore, it was found to have a general normal distribution for all indicators.

3.1. Factorial Validity

We tested two models: a correlated six-factor model (M1) and a second-order model (M2). Both the CFA of M1 and M2 show a good fit to the data, as reported in

Table 3. Summary of goodness-of-fit indexes for different models of BAT (N = 846).

M1								RMSEA 90% CI
	χ²	p	df	CFI	TLI	SRMR	RMSEA	Lower	Upper	AIC	BIC	SAMPLE-ADJ BIC
	1280.194	0.00	480	0.914	0.906	0.048	0.046	0.043	0.050	20,936.087	21,466.219	21,104.216
M2								RMSEA 90% CI
	χ²	p	df	CFI	TLI	SRMR	RMSEA	Lower	Upper	AIC	BIC	SAMPLE-ADJ BIC
	1376.245	0.00	488	0.905	0.897	0.054	0.049	0.046	0.052	21,038.56	21,531.489	21,194.89

Note. M1, Model 1—correlated six-factor CFA model; M2, Model 2 s-order CFA model; χ², chi-square; p, p-value; df, degree of freedom; CFI, comparative fit index; TLI, Tucker–Lewis index; SRMR, standardized root mean squared residual; RMSEA, root mean square error of approximation; CI, confidence interval; AIC, Akaike information criterion; BIC, Bayesian information criterion; SAMPLE-ADJ BIC, sample size-adjusted Bayesian information criterion.

.

The correlated six-factor confirmatory factor analysis showed good or acceptable indices of fit to our data (MLM χ2(480) = 1280.194, p < 0.001, RMSEA = 0.046, 90% CI [0.043, 0.050], CFI = 0.914, TLI = 0.906 and SRMR = 0.048, AIC was 20936.087, BIC was 21466.219 and SAMPLE-ADJ BIC was 21104.216. Likewise, the second-order model also showed good or acceptable indexes of fit (MLM χ2(488) = 1376.245, p < 0.001, RMSEA = 0.049, 90% CI [0.046, 0.052], CFI = 0.905, TLI = 0.897 and SRMR = 0.054, AIC was 21038.56, BIC was 21531.489 and SAMPLE-ADJ BIC was 21194.89.

As better models are expected to have lower AIC, BIC, and SABIC values [90], M1 shows a better fit to the data than M2. Figure 1 shows a graphical representation of the two models (M1 and M2).

3.2. Reliability

Table 4. Results of the Cronbach’s α and McDonald’s ω of the internal consistencies of questionnaire dimensions (N = 846).

	Cronbach’s α of Original Version	Cronbach’s α of Italian Version	McDonald’s ω of Italian Version
core symptoms (BAT-C)	0.95	0.900	0.913
secondary symptoms (BAT-S)	0.90	0.845	0.857
exhaustion	0.92	0.901	0.902
mental distance	0.91	0.722	0.725
cognitive impairment	0.92	0.861	0.863
emotional impairment	0.90	0.802	0.803
psychological distress	0.81	0.827	0.835
psychosomatic complaints	0.85	0.732	0.734

shows the internal consistency of each burnout dimension, as assessed by using Cronbach’s alpha (α, [61]) and McDonald’s omega (ω, [62]).

Cronbach’s α showed values between acceptable and excellent for all subscales of the BAT. Specifically, we observed α = 0.901 for the exhaustion subscale, α = 0.722 for the mental distance subscale, α = 0.861 for the cognitive impairment subscale, α = 0.802 for the emotional impairment subscale, α = 0.827 for the psychological distress subscale and α = 0.732 for the psychosomatic complaints subscale. Cronbach’s α value for the BAT-C was 0.900, while for the BAT-S was 0.845.

Cronbach’s alpha is widely used to assess the internal consistency reliability of scales [92,93]. Despite this, it is shown to be representative only when the assumptions of the essentially tau-equivalent model are satisfied ([94,95,96,97]; for a review, see [98]). However, the requirements of tau-equivalence (equal item variance) are often violated for psychological scales [94] for various reasons, particularly: (1) that the true score variance is constant across all items is an unrealistic perspective [94]; (2) unidimensionality between items’ true scores does not allow perfect intercorrelations in psychological scales, since few of them are truly unidimensional, but always tend towards a certain degree of multidimensionality [99]; and (3) moreover, multiple response formats across items may contribute to violating tau-equivalence [100]. For these reasons, Cronbach’s α is regarded by methodologists as an inappropriate measure of internal consistency reliability (see [98]). In light of this evidence, we used a second index, namely omega [62], that proved to be a more sensible index of internal consistency [96,100,101,102,103,104]. Omega, indeed, not only performs well, but even outperforms alpha, when the assumptions of the essentially tau-equivalent model are not satisfied [104]. As reported in Table 4, McDonald’s ω showed slightly better scores than Cronbach’s α, confirming the satisfactory internal reliability of the scale.

In fact, McDonald’s ω showed slightly better scores than Cronbach’s α, which were satisfactory for all subscales of the BAT. Specifically, we observed ω = 0.902 for the exhaustion subscale, ω = 0.725 for the mental distance subscale, ω = 0.863 for the cognitive impairment subscale, ω = 0.803 for the emotional impairment subscale, ω = 0.835 for the psychological distress subscale and ω = 0.734 for the psychosomatic complaints subscale. McDonald’s ω value for the BAT-C was 0.913, while for the BAT-S was 0.857.

3.3. Construct Validity

Following the item analysis, assuming that the data were normally distributed, we calculated the Pearson correlation coefficients among all scales (see

Table 5. Means. standard deviations. internal consistencies (Cronbach’s α; McDonald’s ω) and correlations of the variables used in the study (N = 846).

	Range	Mean	SD	BAT	BAT_E	BAT_MD	BAT_EI	BAT_CI	BAT_PD	BAT_PC	MBI_EE	WHO
BAT	1–5	1.9563	0.51511
BAT_E	1–5	2.4227	0.75547	0.857 **
BAT_MD	1–5	1.5362	0.50530	0.737 **	0.605 **
BAT_EI	1–5	1.5910	0.57705	0.713 **	0.518 **	0.516 **
BAT_CI	1–5	1.5797	0.55133	0.682 **	0.422 **	0.515 **	0.551 **
BAT_PD	1–5	2.2908	0.81697	0.844 **	0.634 **	0.509 **	0.475 **	0.514 **
BAT_PC	1–5	2.0376	0.70973	0.724 **	0.494 **	0.384 **	0.378 **	0.367 **	0.661 **
MBI_EE	0–6	10.7033	10.42020	0.792 **	0.709 **	0.649 **	0.493 **	0.541 **	0.679 **	0.524 **
WHO	0–5	16.2979	4.83245	−0.669 **	−0.614 **	−0.497 **	−0.422 **	−0.380 **	−0.607 **	−0.465 **	−0.634 **
UWES	0–6	14.8416	3.10464	−0.552 **	−0.499 **	−0.558 **	−0.378 **	−0.348 **	−0.427 **	−0.330 **	−0.585 **	0.640 **

Note. SD, standard deviation. ** p < 0.01.

).

Regarding the relationships between the dimensions of BAT and emotional exhaustion scale of MBI-ES [28], Pearson correlation analysis showed that the MBI emotional exhaustion scale was positively correlated with all dimensions of BAT: exhaustion (r = 0.709, p ≤ 0.01), mental distance (r = 0.649, p ≤ 0.01), emotional impairment (r = 0.493, p ≤ 0.01), cognitive impairment (r = 0.541, p ≤ 0.01), psychological distress (r = 0.679, p ≤ 0.01) and psychosomatic complaints (r = 0.524, p ≤ 0.01). Furthermore, the Pearson correlation analysis showed that the total BAT score was positively correlated with emotional exhaustion scale of Maslach Burnout Inventory (r = 0.792, p ≤ 0.01).

Regarding the World Health Organization Well-Being Index [64], the analysis highlighted significant negative correlations with the BAT scores: exhaustion (r = −0.614, p ≤ 0.01), mental distance (r = −0.497, p ≤ 0.01), emotional impairment (r = −0.422, p ≤ 0.01), cognitive impairment (r = −0.380, p ≤ 0.01), psychological distress (r = −0.607, p ≤ 0.01) and psychosomatic complaints (r = −0.465, p ≤ 0.01). Furthermore, the Pearson correlation analysis showed that the WHO was negatively correlated with the total BAT score (r = −0.669, p ≤ 0.01) and with emotional exhaustion scale of Maslach Burnout Inventory (r = −0.634, p ≤ 0.01).

Finally, with regards to the Utrecht Work Engagement Scale [51], the analysis highlighted significant negative correlations with the BAT scores: exhaustion (r = −0.499, p ≤ 0.01), mental distance (r = −0.558, p ≤ 0.01), emotional impairment (r = −0.378, p ≤ 0.01), cognitive impairment (r = −0.348, p ≤ 0.01), psychological distress (r = −0.427, p ≤ 0.01) and psychosomatic complaints (r = −0.330, p ≤ 0.01). Furthermore, Pearson-correlation analysis showed that the UWES−3 was negatively correlated, with total BAT score (r = −0.552, p ≤ 0.01) and with emotional exhaustion scale of Maslach Burnout Inventory (r = −0.585, p ≤ 0.01), while it was positively correlated with World Health Organization Well-Being Index (r = 0.640, p ≤ 0.01).

4. Discussion

The current study aimed to contribute to the validation of the Italian version of the Burnout Assessment Tool (BAT; [46]) with data collected from a sample of Italian teachers. Specifically, its psychometric properties were evaluated through item analysis, factorial validity, reliability, and construct validity. Details about the obtained results are discussed below.

Concerning the item analysis, the findings show that, except for the kurtosis value of one item (Item 4 of the mental distance factor: “I feel indifferent about my job”), the items’ psychometric properties suggested a general normal distribution for all of the indicators. Therefore, we decided not to delete the abovementioned item for the following reasons. First, according to Hair and colleagues [105] and Byrne [106], data is to be considered normal if kurtosis is between −7 and +7. In our study, Item 4 of the mental distance subscale presented a kurtosis value of 6.745, thus falling within the recommended range. Moreover, the internal consistency of the mental distance subscale (including Item 4), measured by Cronbach’s alpha [61] and McDonald’s omega [62], indicated an acceptable fit (0.722 and 0.725, respectively). Finally, when performing reliability on the mental distance subscale, excluding Item 4, omega values decrease and results are barely acceptable (from 0.725 to 0.683). Since this study aims to contribute to validate the BAT in the Italian context following as much as possible the original tool presented by Schaufeli and colleagues [46], the authors opted to maintain the original item structure.

For factorial validity, CFA analyses were conducted, focusing on the two models that fit the data better in the original study of Schaufeli and colleagues [46]. Our Model 1 (equivalent to Model 6 of the original research) was a correlated six-factors CFA model, where all the six latent burnout dimensions (exhaustion, mental distance, emotional impairment, cognitive impairment, psychological distress, and psychosomatic complaints) were correlated. Besides, our Model 2 (equivalent to Model 8 of the original study) was a second-order CFA model where burnout encompasses a core dimension and a secondary symptoms dimension. Specifically, exhaustion, mental distance, emotional impairment, and cognitive impairment were best represented by the core dimension, while psychological distress and psychosomatic complaints were best represented by the secondary symptoms dimension. Both the tested models have excellent fit indices, and results from the models’ comparison with AIC, BIC, and SAMPLE-ADJ BIC showed that Model 1 fits the data better than Model 2. This datum is similar to the one obtained by Schaufeli and colleagues [46] in their validation study. Even though the correlated six-factor model had the best fit to the data, the authors decided to opt for the second-order factor model for methodological and theoretical reasons. Firstly, and echoing Verbraak and colleagues’ findings [107], the latent correlations of their correlated six-factor model revealed that the four core dimensions were more correlated with each other than with the secondary symptoms, except for the correlation between exhaustion and psychological distress.

Furthermore, Schaufeli and colleagues [46] posited that the second-order factor model and the distinction between core and secondary symptoms had a better fit with the theoretical conceptualization of the burnout construct, as well as with the consideration of burnout as a syndrome [54]. In any case, to support this thesis and further motivate their choice, they reported several papers that showed that the second-order factor model fitted well to the data and was invariant across different national representative samples and taking into consideration age and gender [58,59,108]. Thus, based on these assumptions and considering our study’s aim and hypotheses, we decided to follow Schaufeli and colleagues’ suggestions and chose Model 2, namely the second-order factor model.

Concerning reliability, Cronbach’s alpha and McDonald’s omega, related to both the BAT subscales and the global score, confirmed satisfactory internal reliability of the scale. Indeed, all the obtained scores ranged from acceptable to excellent, according to George and Mallery’s cut-off values [60].

Moreover, to verify the construct validity, the BAT dimensions and total score were correlated with measures like the emotional exhaustion scale of the MBI, UWES, and WHO-5.

Regarding the association between BAT dimensions and MBI’s emotional exhaustion, the correlation matrix showed that they are all significantly and positively related. Specifically, the magnitude of the correlation was higher for the relationship between the emotional exhaustion dimension of the MBI and both the global score and exhaustion dimension of the BAT.

In line with previous findings (e.g., [59]), this datum is not surprising, considering that the scale aims to measure the same construct (i.e., burnout) using a self-report approach on a Likert scale, and that the exhaustion dimension is widely considered representative of the burnout phenomena (e.g., [34]). Nonetheless, the different and lower associations of MBI’s emotional exhaustion with other BAT dimensions, such as emotional impairment and psychosomatic complaints (see Table 5), suggested that MBI and BAT are convergent measures, but differ from each other when considering their respective dimensions. Therefore, this evidence further supports BAT’s multidimensionality and sheds light on the different facets of a complex phenomenon, like burnout, that could not be reduced to its mere exhaustion component.

Our findings show that BAT and its dimensions were inversely related to WHO-5 and UWES, supporting the discriminant validity of the scale. On the negative association with the well-being index (WHO-5), the more teachers in our sample were burnt-out, the less they declared feeling well in the two weeks prior to administering the test. This result is in line with previous findings (e.g., [6,109]) and highlights the correlation between excessive strain and teachers’ well-being. Moreover, the challenges related to the new digital teaching practices and obliged distance-learning might have played a pivotal role in exacerbating this negative association.

Concerning the negative link between BAT and engagement (UWES), our results corroborate the evidence that the two constructs could be considered distinct, despite sharing similarities [59,110,111]. The literature in this field has reported contrasting results on the association between burnout and engagement, usually considering them to be two sides of the same coin and overlapping constructs (e.g., [112]). Nonetheless, most of these studies have explored the relationship by measuring burnout with scales other than BAT. Considering our results and the magnitude of our correlations (the highest was r= −0.55 for BAT global score and r= −0.55 for mental distance dimension), further studies should be conducted to deepen this association and clarify the possible overlap/distinction taking into account the new burnout tool.

In sum, the present study provides preliminary evidence concerning the factorial validity, reliability, and construct validity of the BAT in the Italian context. These results track and confirm those obtained by Schaufeli and colleagues in their validation study with a representative sample of the Flemish worker population [46]. Furthermore, these findings enrich the results of De Beer and colleagues [58], who demonstrated the robustness of the BAT and its measurement invariance concerning seven representative countries (six European countries and Japan). Despite this, Italy was not among the countries in the previous study (in detail, Belgium, Austria, Finland, Germany, Ireland, Netherlands). To date, there is no available data regarding the condition of Italian workers through the use of BAT, as it has not been possible to collect it.

On the other hand, the authors themselves, within the limits of their study, suggest enriching the research on this tool by exploring the conditions of different countries in the light of existing cultural differences. Finally, congruent with the study using the Japanese version of the BAT by Sakakibara and colleagues [59], our findings further support the multi-faceted nature of the burnout construct. Overcoming the internal limitations of the gold standard MBI tool, the BAT could be considered a good alternative tool for measuring burnout in the Italian teachers’ population, and therefore offers a wide range of interrelated measures of burnout dimensions to compare with different countries.

Limitations and Future Directions

The current study has several limitations that should be addressed.

First, the data collection was carried out in the continuing challenging period, wherein teaching methods have profoundly changed, job insecurities have become more pronounced, and the relationship with students has compulsorily switched to a digital mode to which many teachers are not accustomed. Therefore, some results related to the study variables were likely conditioned by contingencies that accentuated the negative experiences related to teaching. Moreover, the sample was not gender-balanced and this could have affected the results. Although the sample shows a gender disparity that reflects reality (the percentage of female teachers is 78% in Italy, higher than that of other OECD countries [4]), in future studies, it would be interesting to examine the BAT in the Italian context with a more balanced sample of teachers and to test its invariance by considering age and gender, as De Beer and colleagues have done [58]. Furthermore, the generalizability of the data obtained in this study should be regarded cautiously. Ours was a convenience sample, with disproportionately many teachers of primary schools (64.3%) than secondary school (35.7%). New studies are hoped to overcome this limit.

Furthermore, and echoing Sakakibara and colleagues [59], there is a need to establish nation-specific cut-offs for screening burnout risk. Indeed, the cut-offs settled in the original study [54] lead to considering the burnout phenomenon as being strictly related to the specific working condition of employees. In this regard, it would be interesting to determine the cut-offs calculated for the Italian sample.

5. Conclusions

The current study represents a contribution to the validation of the Italian version of the Burnout Assessment Tool (BAT; [46]) in data collected from a sample of teachers. Our findings related to item analysis, factorial validity, reliability, and construct validity to provide evidence that the Burnout Assessment Tool is a valid and reliable scale for measuring burnout in the Italian context. Unlike the MBI and CBI, the BAT offers a more complex insight into the syndrome, bringing together different facets of burnout experience.

Specifically, the possibility of measuring various burnout symptoms in such a comprehensive way also allows for applying much more targeted interventions, primarily focused on those aspects that are particularly problematic at a given moment. Taking the secondary symptoms as an example (though this is also valid for the other BAT dimensions), they were very often subtended by other measures, and it became challenging to frame them in a more complex framework, such as burnout. Through the BAT, instead, it is possible, based on various levels of severity given by the scores to the individual dimensions (and this would become even more valid through the establishment of specific cut-offs), to act preventively and promptly to try to stem, as much as possible, the phenomenon ‘in the bud’.

Furthermore, the latest Teacher Assessment and Learning Survey (TALIS) results revealed that half of the Italian teachers who took part in the survey have not received formal training on using technology for instructional purposes. As a result, many of them do not feel ready to use it [113]. In this regard, the significant changes due to the pandemic period (e.g., the intensive use of electronic devices, the distance in the teaching-learning relationships) may have increased teachers’ stress levels. Since this teaching condition is new and utterly distinct from those previously experienced, a multi-faceted tool, such as the BAT, may further support sustainability in human resource management.