Next Article in Journal
Validation and Measurement Invariance of the Scale of Positive and Negative Experience (SPANE) in a Spanish General Sample
Next Article in Special Issue
Occupational Health and Safety Measures in German Outpatient Care Services during the COVID-19 Pandemic: A Qualitative Study
Previous Article in Journal
Quantification of the Aluminum Content Leached into Foods Baked Using Aluminum Foil
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Psychological Adjustment of Healthcare Workers in Italy during the COVID-19 Pandemic: Differences in Stress, Anxiety, Depression, Burnout, Secondary Trauma, and Compassion Satisfaction between Frontline and Non-Frontline Professionals

1
Department of Psychological, Health and Territorial Sciences, University “G. d’Annunzio” of Chieti, via dei Vestini, 66100 Chieti, Italy
2
Dipartimento di Psicologia Dinamica e Clinica, “Sapienza” Università di Roma, 00185 Roma, Italy
3
Facoltà di Psicologia, Università Telematica Internazionale Uninettuno di Roma, 00186 Roma, Italy
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(22), 8358; https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17228358
Submission received: 30 September 2020 / Revised: 3 November 2020 / Accepted: 9 November 2020 / Published: 12 November 2020
(This article belongs to the Special Issue Safety, Health and Wellbeing of Healthcare Workers)

Abstract

:
Emergency situations have been associated with negative psychological adjustment outcomes in healthcare professionals, although studies on the impact of the Coronavirus Disease 2019 (COVID-19) pandemic amongst Italian health workers are limited. The main aim of this study was to investigate the psychological adjustment of healthcare professionals during the peak of the COVID-19 pandemic, evaluating differences according to working or not with patients affected by COVID-19 and in areas with a more severe spread of this pandemic. Healthcare professionals’ attitudes toward psychological support were analyzed. The levels of anxiety, depression, psychological stress, and professional quality of life (compassion satisfaction, burnout, and compassion fatigue) and attitudes toward psychological support were measured among 627 Italian healthcare workers (mean age = 40.55 years; SD = 11.49; range: 27–72). Significantly higher levels of stress, burnout, secondary trauma, anxiety, and depression were observed among professionals working with COVID-19 patients. Higher levels of stress and burnout and lower levels of compassion satisfaction were detected in professionals working in areas with higher rates of contagion. No interaction effects were found between working (or not) with patients affected by COVID-19 and working (or not) in areas with a more severe diffusion of this pandemic. Finally, in the group of professionals who worked with COVID-19 patients, the percentage of professionals who thought to ask for psychological support was twice that of the group that did not work with COVID-19 patients. The overall findings indicate that the mental health of frontline healthcare workers requires further consideration and that targeted prevention and intervention programs are necessary.

1. Introduction

At the end of December 2019, the Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2), then renamed Coronavirus Disease 2019 (COVID-19) by the World Health Organization [1], was first identified in China. COVID-19 rapidly spread all over the world, and the WHO declared a pandemic state on 11 March 2020. At the time of the revision of this manuscript (October 2020), more than 40 million people have been infected [2].
In Italy, the first epidemic outbreak was confirmed in February 2020 [3]; since then, thousands of cases have occurred, and, during the data collection for the current research (mid-April), the most affected regions were Lombardy, Piedmont, Emilia Romagna, and Veneto, each one with more than 10,000 cases of infected individuals [2]. Facing this critical situation, the Italian government implemented measures to reduce the local transmission of the virus and promulgated a lockdown throughout the entire country. This emergency changed everyone’s lives and work habits, especially those of the healthcare workers, who play the most important role in public health [4]. They were directly involved in dealing with this critical condition, which exposed them to a major risk of becoming infected, major psychological pressure related to uncertainty about the duration of the crisis, a lack of proven therapies or a vaccine, and potential shortages of healthcare resources, including personal protective equipment [5,6]. In addition, healthcare workers have often chosen to live far from their families to protect them from the risk of contagion during this period [5].
Previous literature focused on the psychological risks of healthcare workers related to other epidemics, detecting high levels of anxiety, depression [7,8], stress, and burnout [8,9]. Symptoms such as anxiety and fear have immediately increased in the early stages of an epidemiological crisis but rapidly decreased in the later stages, whereas depression and post-traumatic stress symptoms have persisted over time [10]. Frontline healthcare workers involved in the diagnosis, treatment, and care of COVID-19 patients have reported more severe symptoms of anxiety, depression, and stress than those not on the front line [6,11].
The literature on emergency work has focused on the effects of working in critical situations, especially on burnout, as it poses a risk for healthcare workers [9,12]. As a concept, burnout refers to the emotional affect associated with feelings of frustration and powerlessness that develop in professionals’ negative attitudes at work [13,14]. Individuals who work in healthcare services have been identified as a high-risk group for burnout, and the workers who have symptoms of this syndrome are less satisfied with their work activities than others [15,16]. In a recent study in this population, more than 62% of workers showed a moderate degree of burnout [17].
With these theoretical premises in mind, we thought it important to measure the professional quality of life in the current emergency context. Figley [18,19] proposed two relevant aspects: compassion fatigue and compassion satisfaction. Compassion fatigue [12,18] is an outcome of prolonged and intense contact with patients and exposure to continuous stress; it may overlap with the concept of secondary trauma [9,20,21]. Figley [18] extended this concept to a combination of behavioral and emotional reactions following attempts to aid traumatized people. Compassion satisfaction represents the positive aspect of being a healthcare worker, related to an empathetic attitude and inclination to take care of suffering patients [9,22,23]. These factors play a fundamental role in professional quality of life [24,25,26]; stress and negative affects seem to be positively correlated with compassion fatigue, whereas positive affect and personal/social factors are positively correlated with compassion satisfaction [9]. According to Stamm [27], compassion fatigue incorporates two aspects: burnout (exhaustion, frustration, anger, and depression) and secondary trauma (negative feelings driven by fear and work-related trauma). In this paper, we refer to compassion fatigue considering its two aspects: burnout and secondary trauma.
To summarize, emergency situations, such as the current COVID-19 pandemic, have been associated with both psychological adjustment and professional quality of life outcomes.

Study Aims and Hypotheses

According to these premises, we conducted a study in a sample of healthcare workers in Italy. To the best of our knowledge, this is among the first studies in Italy that have investigated the psychological impact of the pandemic on healthcare workers. Specifically, our main aim was to analyze the psychological adjustment of Italian healthcare professionals during the peak of the COVID-19 pandemic in terms of perceived stress, anxiety, depression, and professional quality of life (i.e., burnout, secondary trauma, and compassion satisfaction). More specifically, we examined the differences in these variables according to (1) working or not working with patients affected by COVID-19, and (2) working or not working in areas with a more severe diffusion of this pandemic. According to the Ministry of Health, Italian regions with more than 10,000 infected individuals at the peak of the first wave were defined as the most critical (i.e., Lombardy, Piedmont, Emilia Romagna, and Veneto). Following previous research amongst Chinese healthcare workers [6,11,28,29], we hypothesized that healthcare professionals working with patients affected by COVID-19 and in the most affected Italian regions would report higher levels of distress than the other categories (i.e., not working with COVID-19 patients or living in other regions). Previous literature on emergency situations merged these two conditions (working with affected patients and working in areas with a more severe diffusion of the pandemic) into a single group of frontline workers. We preferred to keep the two groups separate to better observe differences between them. Finally, we aimed to explore healthcare professionals’ attitudes toward psychological support by asking them whether they have considered asking for assistance, according to working or not with COVID-19 patients and in the most affected regions.

2. Materials and Methods

2.1. Participants

The participants comprised 627 healthcare workers, with a mean age of 40.55 years (SD = 11.49; range: 27–72). The descriptive statistics of the participants’ characteristics are presented in Table 1. The participants had been working as healthcare professionals for 13.76 years on average (SD = 11.39). The majority of the sample had not been infected by COVID-19 (93.9%); only a small percentage was infected, with (1.2%) or without (4.9%) symptoms. To verify differences in psychological adjustment between healthcare workers infected by COVID-19 and not infected, the Mann–Whitney U test was carried out. Differences in the distributions of the two groups were found relating to perceived stress (p < 0.05), anxiety (p < 0.01), and depression (p < 0.05), with infected healthcare workers showing worse psychological adjustment than their colleagues. As for marital status, most of them (75.4%) had a current relationship; the remainder (24.6%) were divorced, widowers, or single, and 50.1% had at least one child.
At the time of questionnaire administration, 48.8% were working with COVID-19 patients and 33.5% of the study participants were working in the most affected Italian regions (regions with more than 10,000 infected individuals as defined by the Italian Health Ministry).

2.2. Measures

2.2.1. Sociodemographic Information

To gather sociodemographic information, an ad hoc questionnaire was prepared that consisted of questions regarding personal information (e.g., sex, age, nationality, and household income), work (e.g., years of working and the type of healthcare profession), and a question about the attitude toward psychological support (no, I do not think I need it; I thought about it but I do not think I will start it; I think I will but I have not started yet; I have already started sessions of psychological support).

2.2.2. Hospital Anxiety and Depression Scale

The Hospital Anxiety and Depression Scale (HADS) [30,31] is a reliable self-report instrument that consists of two independent subscales: anxiety (HADS-A; sample items: “I get a sort of frightened feeling as if something awful is about to happen” and “Worrying thoughts go through my mind”) and depression (HADS-D; sample items: “I have lost interest in my appearance” and “I look forward with enjoyment to things”), each containing seven items. Each item is scored on a 4-point Likert scale, with a total score between 0 and 21 for either anxiety or depression. In previous studies, this instrument showed good reliability [32], such as in our study, where the Cronbach’s α was 0.893.

2.2.3. Perceived Stress Scale

The short version of the Perceived Stress Scale (PSS-10) [33,34] was used to measure perceived stress. It was developed by Cohen et al. [33] to assess the degree to which situations in one’s life are evaluated as stressful and comprises 10 items: 6 negatively stated (sample item: “In the last month, how often have you been upset because of something that happened unexpectedly?”) and 4 positively stated (sample item: “In the last month, how often have you felt confident about your ability to handle your personal problems?”), rated on a 5-point Likert scale. The total scores range from 0 to 40, with higher scores showing higher levels of perceived stress. In this study, the Cronbach’s α was very good (α = 0.883), similar to that in previous studies that used this same tool [35,36].

2.2.4. Professional Quality of Life Scale

The perceived quality in relation to work as a helper was assessed using the Professional Quality of Life Scale-V (ProQOL-V) [37], which comprises 30 items corresponding to three subscales: compassion satisfaction (10 items), burnout (10 items), and secondary traumatic stress (10 items). The compassion satisfaction items are characterized by the positive and altruistic aspects of the helping work (sample item: “I get satisfaction from being able to help people”), unlike burnout (sample item: “I feel trapped by my job as a helper”) and secondary traumatic stress (sample item: “I think that I might have been affected by the traumatic stress of those I help”). Each item explores how often, during the previous month, the subject has experienced a series of emotional states because of the helping work. The responses are rated on a 5-point Likert scale, with higher scores on each subscale indicating higher secondary trauma, compassion satisfaction, and burnout. In our study, the value of Cronbach’s α was satisfactory (α = 0.724), in line with recent research [24].

2.3. Data Analysis

All the statistical analyses were carried out through the Statistical Package for the Social Sciences (IBM Corp; Released 2010; IBM SPSS Statistics for Windows, Version 19.0.; Armonk, NY: IBM Corp.). The significance level was set to p < 0.05. First, the skewness and kurtosis of all the variables were tested to check their distributions. To verify the existence of differences between groups in the psychological adjustment of healthcare workers, a multivariate analysis of variance (MANOVA) was carried out. The between-subject factors were working (or not) with patients affected by COVID-19 and working (or not) in areas with a more severe diffusion of this pandemic; the dependent variables were perceived stress, professional quality of life, anxiety, and depression. Finally, a chi-square analysis for categorical variables was conducted to explore healthcare professionals’ attitudes toward the need for psychological support during this emergency period caused by the COVID-19 pandemic.

2.4. Procedure

This study was a cross-sectional survey conducted on the psychological status of Italian healthcare professionals during the peak of the COVID-19 pandemic. To involve subjects from all the Italian regions, a web-based survey was sent on the Internet to several groups of healthcare workers through mainstream social media (WhatsApp, Facebook, etc.). Participants were recruited via snowball sampling and were also asked to forward the survey to others. The inclusion criteria were working as a healthcare professional, working in Italy, and being over 18 years of age. Data were collected from 11 to 16 April 2020 (six weekdays), and all 20 Italian regions were represented.
The participants were informed about the aim and the procedure of the study. They were requested to provide their informed consent before completing the questionnaires anonymously. The participation in this study was voluntary and was not compensated. The procedure and all the questionnaires used in this survey were fully compliant with the indications of the Declaration of Helsinki and with the Ethics Code of the Italian Board of Psychology (the regulatory Authority providing the national guidelines for research and clinical practice). The study protocol was reviewed by the Institutional Review Board of Psychology (IRBP; Protocol Number: 20013) of the Department of Psychological, Health and Territorial Sciences.

3. Results

The distributions of all the variables may be considered as acceptably normal since their skewness and kurtosis were between −1 and +1 [38].

3.1. Assessment of Psychological Adjustment of Healthcare Workers

To verify the differences between groups in terms of the psychological adjustment of healthcare workers, a MANOVA was carried out. The independent variables were working (or not) with patients affected by COVID-19 (“COVID Patients”) and working (or not) in areas with a more severe diffusion of this pandemic (“Most Affected Regions”); the dependent variables were perceived stress, professional quality of life, anxiety, and depression.
The results showed a significant effect of the independent variables COVID Patients (λ = 0.95; F(6618) = 5.37; p = 0.000) and Most Affected Regions (λ = 0.97; F(6618) = 2.70; p = 0.014), but not for their interaction (λ = 0.99; F(6618) = 0.41; p = 0.869). The results of the univariate effects are shown in Table 2 and Table 3 for the independent variables COVID Patients and Most Affected Regions, respectively.
With regard to COVID Patients, we observed statistically significantly higher levels of stress, burnout, secondary trauma, anxiety, and depression among professionals working with COVID-19 patients. No differences were detected in compassion satisfaction.
With regard to Most Affected Regions, our findings highlight that healthcare professionals working in the Italian regions most affected by the COVID-19 pandemic presented higher levels of perceived stress and burnout and lower levels of compassion satisfaction compared with healthcare professionals working in the other regions. No differences for other variables (anxiety, depression, and secondary trauma) were observed.

3.2. Healthcare Professionals’ Attitudes toward the Need for Psychological Support

Finally, we aimed to explore healthcare professionals’ attitudes toward the need for psychological support during this emergency period caused by the COVID-19 pandemic. Chi-square analyses were conducted. Although no significant effects were found when exploring psychological support needs according to working (or not) in the Italian regions most affected by this pandemic, significant effects were found for psychological support needs according to professionals working with COVID-19 patients and professionals not working with such patients. These results are summarized in Table 4.
The results show that most of the overall sample did not consider asking for psychological help, but this percentage significantly decreased in the group of professionals who worked with COVID-19 patients. In this latter group, the percentage of professionals who thought to ask for psychological support was double that of the group that did not work with COVID-19 patients.

4. Discussion

The literature suggests that medical staff treating patients with COVID-19 report high levels of anxiety and low self-efficacy levels [39,40], and other research on Wuhan healthcare workers reported great vulnerability to stress, anxiety, and depression, suggesting that frontline healthcare workers should be closely monitored as a high-risk group for maladjustment [29]. Excessive stress and/or anxiety in the clinical context may affect performance and can compromise patient outcomes [41].
In this study, we explored the psychological adjustment of Italian healthcare professionals during the peak of the COVID-19 pandemic in terms of perceived stress, anxiety, depression, and quality of professional life (including the three sub-variables of compassion satisfaction, secondary trauma, and burnout). In particular, our purpose was to verify whether working with patients affected by COVID-19 was associated with higher scores in the above study variables; we were also interested in verifying if these scores were affected by working or not in areas with higher reported cases and deaths. As some Chinese studies suggested, healthcare professionals working with COVID-19 patients were more at risk of higher levels of stress, anxiety, and depression [6,26,28]. In Italy, a study on the general population reported high stress levels during the COVID-19 pandemic [42], whereas studies specifically addressing healthcare workers’ adjustment are increasing but still few in number [24,35,43].
To address our first objective, we examined the existence of differences in psychological assessment results according to (1) working or not with patients affected by COVID-19, and (2) working or not in areas more severely affected by this pandemic in terms of case count. The results showed significant differences between both these groups, but not for their interaction. Specifically, the findings showed no difference in compassion satisfaction and a significant difference in the distribution of perceived stress, anxiety, depression, burnout, and secondary trauma levels between healthcare workers who worked or not with patients affected by the COVID-19 disease. In particular, consistent with our hypothesis, healthcare workers directly involved with COVID-19 patients reported higher levels in all these variables than their colleagues working with non-COVID-19 patients. This result is in line with two recent Chinese studies [6,28] that considered a sample of frontline care workers, reporting a high prevalence of mental health symptoms among these professionals. In this sense, our study supports previous research but adds further knowledge about other key variables such as burnout and secondary trauma that were not addressed in previous research on COVID-19. In the literature on other pandemics, burnout was posited to be associated with stress, anxiety, depression, and secondary trauma in frontline workers [14], and some authors found that it might contribute to the increased incidence of their stress and depression [44,45]. Zhang et al. [9] demonstrated that burnout levels in medical staff and other healthcare workers were typically higher than in other populations, and Bansal et al. [46] stated that clinicians were more likely to show burnout symptoms than the general workforce, which decreased performance and quality of life. In general, the rates of burnout among healthcare workers are posited to be high [47]. Although several authors have evaluated burnout in frontline workers, our study is, to the best of our knowledge, the first to address it in the context of the COVID-19 pandemic in Italy.
The concept of secondary trauma is strictly associated with burnout, as both form compassion fatigue; more specifically, it refers to secondary exposure to people who have experienced extremely or traumatically stressful events [27]. As expected, healthcare professionals working with COVID-19 patients showed higher levels of secondary trauma than their colleagues, but this result conflicts with the findings of Li et al. [48], who reported higher levels of this variable among non-frontline nurses than among frontline ones. Following the authors’ suggestion, their data could be explained by the possible higher psychological endurance of frontline nurses [48]. In this sense, frontline medical workers could be more accustomed to potentially distressing experiences than non-frontline workers, therefore showing a lower negative response to challenging situations. However, this was not the case in our frontline sample, which showed higher levels of secondary trauma compared with non-frontline colleagues.
With regards to the differences between healthcare workers that carry out their work in the regions most heavily affected by COVID-19 and those working in less impacted areas, our findings highlight that healthcare professionals working in the Italian regions most affected by the COVID-19 pandemic presented higher levels of perceived stress and burnout and lower levels of compassion satisfaction than the other group. This result is consistent with previous studies analyzing interregional differences. Lai et al. [6], for instance, found higher levels of distress in medical staff in the most affected Chinese region (Hubei) and, specifically, in Wuhan [6]. Our study is amongst the first to address this finding in Italy, which suffered high rates of contagion throughout the country. On one of the last days of our data collection (15 April 2020), Italy was the first country in the world in terms of the number of deaths (21,647 in total from the beginning of the COVID-19 pandemic) and the second country (Spain was the first) for the number of infections (165,155 total cases) [1]. However, the rates of contagion showed relevant differences according to different Italian geographical areas. On the last day of our data collection, the most affected Italian region was Lombardy, with more than 63,000 cases, while the least affected regions were Molise and Basilicata, with about 300 cases each (www.protezionecivile.gov.it).
If we observe the differences between a) working or not with COVID-19 patients and b) working or not in the Italian regions with the higher infection rates and number of deaths, we notice some interesting elements. Overall, the former condition was associated with a worse psychological adjustment, as it presented higher levels of all the study variables (namely, stress, burnout, secondary trauma, anxiety and depression), with the exception of compassion satisfaction. The latter condition (working in the Italian regions with the higher rates of infected individuals and deaths) was only associated with higher stress and burnout.
Another interesting result was that contrary to our expectations, we found no significant interactions between these two variables (working or not with COVID-19 patients and working or not in the Italian regions with the higher infection rates and number of deaths). We suppose that other variables (such as the relationships that develop within the medical staff or the perceived social support) should be considered to explain why healthcare professionals working in the most critical situations (i.e., with COVID-19 patients and in the most affected Italian regions) had no worse psychological adjustment than the other categories of colleagues. However, as we did not address these variables, no conclusions can be drawn in this direction. Other research is needed in order to further explore these issues with larger and numerically homogeneous groups. Overall, our findings underline that, differently from previous literature on emergency situations that merged these two conditions (working with affected patients and working in areas with a more severe diffusion of the pandemic) into a single group of frontline workers [11,28,29], future studies should separate those two groups to more clearly observe their differences.
A further result from our study concerns compassion satisfaction, which refers to an empathetic attitude and inclination to take care of suffering patients [9,22]. In some studies, it was associated with positive affect, empathy, and high levels of professional quality of life [9,22,49]. Compassion satisfaction occurs in the form of the helper’s altruistic behaviors that are aimed at alleviating the patients’ suffering [50]; healthcare workers experiencing high levels of compassion satisfaction were able to provide competent and compassionate care [50]. This construct was identified as a factor protective against burnout and compassion fatigue in previous studies [51]. In our results, no differences were detected between professionals working or not with COVID-19 patients, whereas it was significantly lower in professionals working in the Italian regions most affected by the current pandemic. We may explain this apparently contradictory result as a consequence of the particularly difficult conditions in which healthcare professionals worked in the most affected regions during the peak of pandemic: a fundamental basis of compassion satisfaction is constituted of an empathetic relationship with the patient’s family and support from colleagues [49]. All these aspects were likely compromised by working in the Italian regions most affected by the COVID-19 pandemic but not in the other regions when also working with COVID-19 patients. However, this result has to be considered cautiously; future research is needed to explore this finding.
Finally, we investigated healthcare professionals’ attitudes toward the need for psychological support during this emergency period. We observed that most of the total sample did not consider asking for psychological help, but this percentage was significantly lower in the group of professionals who worked with COVID-19 patients. This finding could be explained by the very difficult conditions in which they lived, from both personal and employment viewpoints. They had to deal with overwork, a high risk of infection, and, sometimes, inadequate protection from contamination, frustration, high-risk patients with negative feelings, discrimination, isolation, a lack of contact with their families, and exhaustion [11]. It was not surprising that they thought of asking for psychological help. However, caution is needed in drawing conclusions, as this question does not highlight if individuals currently receiving psychological support had actually started it before the COVID-19 pandemic, and if those who were considering support were doing so because of their work or because of other personal factors. Therefore, these issues should be analyzed in future studies.
The limitations of this study include the fact that the sample size was unbalanced for sex, with women representing the majority, although this distribution is representative of the sex differences among Italian healthcare workers [52]; the number of participants at the intersection of the two between-subject factors we considered (working or not with COVID-19 patients and working or not in the Italian regions with the higher infection rates and number of deaths) was not balanced; our measurements were exclusively based on self-report questionnaires; and the cross-sectional design did not allow drawing cause–effect relationships. All these aspects limit the generalizability of the current research to other professional groups and situations different from the current COVID-19 pandemic.
Notwithstanding these limitations, our study offers some suggestions about health professionals’ psychological adjustment during the climax of the COVID-19 pandemic. This category of workers has been working on the frontline during the outbreak, facing extreme conditions that have never occurred before, at least in Italy, with isolation from family and friends, social distancing, and shifts requiring intense work. No proven therapies or vaccines were available during the first wave of the pandemic. We hypothesize that these conditions found healthcare professionals unprepared to deal with them; despite this, all the healthcare workers faced this exceptional situation, which in some cases led them to become infected with COVID-19. In this group, we observed higher levels of stress, anxiety and depression when compared with their non-infected colleagues, but, as the percentage of infected workers in our sample was very low, further research is necessary to more deeply explore this issue.
As future pandemics may be more frequent than in the past [53], it is important to implement support programs specifically dedicated to medical staff. Protecting the mental health of healthcare workers is an important issue for public health and policymakers. If pandemics are disastrous events affecting the entire population, they especially impact healthcare systems (e.g., inadequate resources for medical responses, healthcare worker illness, fewer services for the population, etc.), having important social and economic repercussions [54]. In this difficult situation, policymakers could increase hiring in this job sector, encourage more flexible work shifts, and implement psycho-social support for this category of workers. Therefore, health authorities should not only improve working conditions but also plan specific intervention and support services to reduce the psychological burden on health workers. Among these specific interventions, psychological support should be delivered remotely (by the Internet or phone) and scheduled to adapt to the work shifts of the professionals.

5. Conclusions

In our study, to the best of our knowledge, we were amongst the first to examine the psychological adjustment of Italian healthcare professionals during the peak of the current COVID-19 pandemic, considering a number of variables referring to personal and professional adjustment. This category of professionals was the one most directly involved in managing the emergency caused by the very rapid spread of the SARS-CoV-2 pandemic in Italy, which produced a high number of deaths and individuals infected that were in serious medical conditions (on 15 April 2020 in Italy, there were 27,643 individuals hospitalized with symptoms and 3079 hospitalized in intensive care units [55]). Overall, our findings highlight that professionals working with COVID-19 patients are at higher risk of stress, burnout, secondary trauma, anxiety, and depression, and health professionals working in the most affected areas are at a major risk of stress, burnout, and low compassion satisfaction. We also found that in the group of professionals who worked with COVID-19 patients, the percentage of professionals who thought to ask for psychological support was double that of the group that did not work with such patients. This last topic has never been addressed before during the current COVID-19 pandemic. Taken together, these results indicate that the mental health of frontline health workers requires further consideration and that targeted prevention and intervention programs are necessary. If, as it seems, we will have to acclimate to living with the risk of pandemics in our future, it is certainly important to be prepared, especially by supporting the professionals who will work on the front line.

Author Contributions

Conceptualization, C.T., C.C. and A.B.; methodology, C.T., S.M.B., G.B., C.C., L.C., S.C., M.C., L.L., S.P., M.L.V. and A.B.; software, C.T., S.M.B., G.B., M.C., L.L., S.P., M.L.V. and A.B.; validation, C.T., S.M.B., G.B., M.C., L.L., S.P., M.L.V. and A.B.; formal analysis, C.T., S.M.B., G.B., M.C., L.L., S.P., M.L.V. and A.B.; investigation, C.T., S.M.B., C.C., L.C., S.C., M.C., L.L., S.P., M.L.V. and A.B.; resources, C.T., C.C., L.C., S.C. and A.B.; data curation, C.T., S.M.B., G.B., M.C., L.L., S.P., M.L.V. and A.B.; writing—original draft preparation, C.T., S.M.B., C.C., L.C., S.C., M.C., L.L., S.P., M.L.V. and A.B.; writing—review and editing, C.T., S.M.B., G.B., C.C., L.C., S.C., M.C., L.L., S.P., M.L.V. and A.B.; visualization, C.T., S.M.B., C.C., L.C., S.C., M.C., L.L., S.P., M.L.V. and A.B.; supervision, C.T. and A.B.; project administration, C.T. and A.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

The authors want to thank all the health professionals who agreed to participate and dedicated a part of their time to this study.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. WHO. Coronavirus Disease 2019 (COVID-19) Situation Report-87. Available online: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200416-sitrep-87-covid-19.pdf?sfvrsn=9523115a_2 (accessed on 27 April 2020).
  2. John Hopkins University. Available online: https://0-coronavirus-jhu-edu.brum.beds.ac.uk/map.html (accessed on 27 April 2020).
  3. Istituto Superiore di Sanità. Available online: https://www.epicentro.iss.it/coronavirus/aggiornamenti (accessed on 27 April 2020).
  4. Tian, F.; Li, H.; Tian, S.; Yang, J.; Shao, J.; Tian, C. Psychological symptoms of ordinary Chinese citizens based on SCL-90 during the level I emergency response to COVID-19. Psychiatry Res. 2020, 288, 112992. [Google Scholar] [CrossRef] [PubMed]
  5. Wu, P.E.; Styra, R.; Gold, W.L. Mitigating the psychological effects of COVID-19 on health care workers. Can. Med. Assoc. J. 2020, 192, E459–E460. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  6. Lai, J.; Ma, S.; Wang, Y.; Cai, Z.; Hu, J.; Wei, N.; Wu, J.; Du, H.; Chen, T.; Li, R.; et al. Factors Associated with Mental Health Outcomes Among Health Care Workers Exposed to Coronavirus Disease 2019. JAMA Netw. Open 2020, 3, e203976. [Google Scholar] [CrossRef] [PubMed]
  7. Cocco, E.; Gatti, M.; Lima, C.A.D.M.; Camus, V. A comparative study of stress and burnout among staff caregivers in nursing homes and acute geriatric wards. Int. J. Geriatr. Psychiatry 2002, 18, 78–85. [Google Scholar] [CrossRef]
  8. Wong, T.W.; Yau, J.K.; Chan, C.L.; Kwong, R.S.; Ho, S.M.; Lau, C.; Lau, F.L.; Lit, C.H. The psychological impact of severe acute respiratory syndrome outbreak on healthcare workers in emergency departments and how they cope. Eur. J. Emerg. Med. 2005, 12, 13–18. [Google Scholar] [CrossRef]
  9. Zhang, Y.-Y.; Han, W.-L.; Qin, W.; Yin, H.-X.; Zhang, C.-F.; Kong, C.; Wang, Y. Extent of compassion satisfaction, compassion fatigue and burnout in nursing: A meta-analysis. J. Nurs. Manag. 2018, 26, 810–819. [Google Scholar] [CrossRef]
  10. Wu, P.; Fang, Y.; Guan, Z.; Fan, B.; Kong, J.; Yao, Z.; Liu, X.; Fuller, C.J.; Susser, E.; Lu, J.; et al. The Psychological Impact of the SARS Epidemic on Hospital Employees in China: Exposure, Risk Perception, and Altruistic Acceptance of Risk. Can. J. Psychiatry 2009, 54, 302–311. [Google Scholar] [CrossRef]
  11. Kang, L.; Ma, S.; Chen, M.; Yang, J.; Wang, Y.; Li, R.; Yao, L.; Bai, H.; Cai, Z.; Yang, B.X.; et al. Impact on mental health and perceptions of psychological care among medical and nursing staff in Wuhan during the 2019 novel coronavirus disease outbreak: A cross-sectional study. Brain Behav. Immun. 2020, 87, 11–17. [Google Scholar] [CrossRef]
  12. Ruíz-Fernández, M.D.; Ramos-Pichardo, J.; Ibáñez-Masero, O.; Cabrera-Troya, J.; Carmona-Rega, M.I.; Ortega-Galán, Á.M. Compassion fatigue, burnout, compassion satisfaction, and perceived stress in healthcare professionals during the COVID-19 health crisis in Spain. J. Clin. Nurs. 2020. [Google Scholar] [CrossRef]
  13. Algunmeeyn, A.; El-Dahiyat, F.; Altakhineh, M.M.; Azab, M.; Babar, Z.-U.-D. Understanding the factors influencing healthcare providers’ burnout during the outbreak of COVID-19 in Jordanian hospitals. J. Pharm. Policy Pract. 2020, 13, 1–8. [Google Scholar] [CrossRef]
  14. Leiter, M.P.; Harvie, P.; Frizzell, C. The correspondence of patient satisfaction and nurse burnout. Soc. Sci. Med. 1998, 47, 1611–1617. [Google Scholar] [CrossRef]
  15. Heath, C.; Sommerfield, A.; Von Ungern-Sternberg, B.S. Resilience strategies to manage psychological distress among healthcare workers during the COVID-19 pandemic: A narrative review. Anaesthesia 2020, 75, 1364–1371. [Google Scholar] [CrossRef]
  16. Shanafelt, T.D.; Balch, C.M.; Bechamps, G.J.; Russell, T.; Dyrbye, L.; Satele, D.; Collicott, P.; Novotny, P.J.; Sloan, J.; Freischlag, J.A. Burnout and career satisfaction among American surgeons. Ann. Surg. 2009, 3, 463–471. [Google Scholar] [CrossRef]
  17. Liu, Y.; Lu, L.; Wang, W.-X.; Liu, S.; Chen, H.-R.; Gao, X.; Huang, M.-Y.; Liu, Y.-N.; Ren, Y.-M.; Wang, C.-C. Job Burnout and Occupational Stressors among Chinese Healthcare Professionals at County-Level Health Alliances. Int. J. Environ. Res. Public Health 2020, 17, 1848. [Google Scholar] [CrossRef] [Green Version]
  18. Figley, C.R. Compassion Fatigue: Coping with Secondary Traumatic Stress Disorder in those who Treat the Traumatized; Brunner-Mazel: New York, NY, USA, 1995. [Google Scholar]
  19. Figley, C.R. Treating Compassion Fatigue; Brunner-Routledge: New York, NY, USA, 2002. [Google Scholar]
  20. Stewart, D.W. Review Casualties of war: Compassion fatigue and health care providers. Med. Surg. Nurs. 2009, 18, 91–94. [Google Scholar]
  21. Bride, B.E.; Radey, M.; Figley, C.R. Measuring Compassion Fatigue. Clin. Soc. Work. J. 2007, 35, 155–163. [Google Scholar] [CrossRef]
  22. Hart, P.L.; Brannan, J.D.; De Chesnay, M. Resilience in nurses: An integrative review. J. Nurs. Manag. 2012, 22, 720–734. [Google Scholar] [CrossRef] [Green Version]
  23. Okoli, C.T.; Seng, S.; Otachi, J.K.; Higgins, J.T.; Lawrence, J.; Lykins, A.; Bryant, E. A cross-sectional examination of factors associated with compassion satisfaction and compassion fatigue across healthcare workers in an academic medical centre. Int. J. Ment. Health Nurs. 2019, 29, 476–487. [Google Scholar] [CrossRef]
  24. Buselli, R.; Corsi, M.; Baldanzi, S.; Chiumiento, M.; Del Lupo, E.; Dell’Oste, V.; Bertelloni, C.A.; Massimetti, G.; Dell’Osso, L.; Cristaudo, A.; et al. Professional Quality of Life and Mental Health Outcomes among Health Care Workers Exposed to Sars-Cov-2 (Covid-19). Int. J. Environ. Res. Public Health 2020, 17, 6180. [Google Scholar] [CrossRef]
  25. Stamm, B.H. Measuring Compassion Satisfaction as well as fatigue: Developmental history of the Compassion Satisfaction and fatigue test. In Treating Compassion Fatigue; Figley, C.R., Ed.; Brunner-Routledge: New York, NY, USA, 2002; pp. 107–119. [Google Scholar]
  26. Stamm, B.H. The Professional Quality of Life Scale: Compassion Satisfaction, Burnout, and Compassion Fatigue/Secondary Trauma Scales; Sidran Press: Latherville, MD, USA, 2005. [Google Scholar]
  27. Stamm, B.H. The Concise ProQOL Manual, 2nd ed.; Eastwoods LLCC: Denver, CO, USA, 2010. [Google Scholar]
  28. Cai, H.; Tu, B.; Ma, J.; Chen, L.; Fu, L.; Jiang, Y.; Zhuang, Q. Psychological Impact and Coping Strategies of Frontline Medical Staff in Hunan Between January and March 2020 During the Outbreak of Coronavirus Disease 2019 (COVID-19) in Hubei, China. Med. Sci. Monit. 2020, 26, e924171-1. [Google Scholar] [CrossRef]
  29. Du, J.; Dong, L.; Wang, T.; Yuan, C.; Fu, R.; Zhang, L.; Liu, B.; Zhang, M.; Yin, Y.; Qin, J.; et al. Psychological symptoms among frontline healthcare workers during COVID-19 outbreak in Wuhan. Gen. Hosp. Psychiatry 2020, in press. [Google Scholar] [CrossRef] [PubMed]
  30. Zigmond, A.S.; Snaith, R.P. The Hospital Anxiety and Depression Scale. Acta Psychiatr. Scand. 1983, 67, 361–370. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  31. Costantini, M.; Musso, M.; Viterbori, P.; Bonci, F.; Del Mastro, L.; Garrone, O.; Venturini, M.; Morasso, G. Detecting psychological distress in cancer patients: Validity of the Italian version of the Hospital Anxiety and Depression Scale. Support. Care Cancer 1999, 7, 121–127. [Google Scholar] [CrossRef] [PubMed]
  32. Scoppetta, O.; Cassiani-Miranda, C.A.; Arocha-Díaz, K.N.; Cabanzo-Arenas, D.F.; Campo-Arias, A. Validity of the Patient Health Questionnaire-2 (PHQ-2) for the Detection of Depression in Primary Care in Colombia. J. Affect. Disord. 2020. [Google Scholar] [CrossRef]
  33. Cohen, S.; Kamarck, T.; Mermelstein, R. A Global Measure of Perceived Stress. J. Health Soc. Behav. 1983, 24, 385. [Google Scholar] [CrossRef]
  34. Cohen, S.; Williamson, G. Perceived stress in a probability sample of the United States. In The Social Psychology of Health; Spacapan, S., Oskamp, S., Eds.; SAGE: Newbury Park, CA, USA, 1988; pp. 31–68. [Google Scholar]
  35. Babore, A.; Lombardi, L.; Viceconti, M.L.; Pignataro, S.; Marino, V.; Crudele, M.; Candelori, C.; Bramanti, S.M.; Trumello, C. Psychological effects of the COVID-2019 pandemic: Perceived stress and coping strategies among healthcare professionals. Psychiatry Res. 2020, 293, 113366. [Google Scholar] [CrossRef]
  36. Swami, V.; Horne, G.; Furnham, A. COVID-19-related stress and anxiety are associated with negative body image in adults from the United Kingdom. Personal. Individ. Differ. 2020, 170, 110426. [Google Scholar] [CrossRef]
  37. Stamm, B.H. Professional Quality of Life: Compassion Satisfaction and Fatigue Version 5 (ProQOL). 2009. Available online: https://proqol.org/ (accessed on 27 April 2020).
  38. Marcoulides, G.A.; Hershberger, S.L. Multivariate statistical methods. In A First Course; Erlbaum: Mahwah, NJ, USA, 1997. [Google Scholar]
  39. Shechter, A.; Diaz, F.; Moise, N.; Anstey, D.E.; Ye, S.; Agarwal, S.; Birk, J.L.; Brodie, D.; Cannone, D.E.; Chang, B.; et al. Psychological distress, coping behaviors, and preferences for support among New York healthcare workers during the COVID-19 pandemic. Gen. Hosp. Psychiatry 2020, 66, 1–8. [Google Scholar] [CrossRef]
  40. Xiao, H.; Zhang, Y.; Kong, D.; Li, S.; Yang, N. The Effects of Social Support on Sleep Quality of Medical Staff Treating Patients with Coronavirus Disease 2019 (COVID-19) in January and February 2020 in China. Med. Sci. Monit. 2020, 26. [Google Scholar] [CrossRef]
  41. Ignacio, J.; Dolmans, D.; Scherpbier, A.; Rethans, J.-J.; Chan, S.; Liaw, S.Y. Stress and anxiety management strategies in health professions’ simulation training: A review of the literature. BMJ Simul. Technol. Enhanc. Learn. 2016, 2, 42–46. [Google Scholar] [CrossRef]
  42. Flesia, L.; Monaro, M.; Mazza, C.; Fietta, V.; Colicino, E.; Segatto, B.; Roma, P. Predicting Perceived Stress Related to the Covid-19 Outbreak through Stable Psychological Traits and Machine Learning Models. J. Clin. Med. 2020, 9, 3350. [Google Scholar] [CrossRef] [PubMed]
  43. Berlanda, S.; De Cordova, F.; Fraizzoli, M.; Pedrazza, M. Risk and Protective Factors of Well-Being among Healthcare Staff. A Thematic Analysis. Int. J. Environ. Res. Public Health 2020, 17, 6651. [Google Scholar] [CrossRef] [PubMed]
  44. Asai, M.; Morita, T.; Akechi, T.; Sugawara, Y.; Fujimori, M.; Akizuki, N.; Nakano, T.; Uchitomi, Y. Burnout and psychiatric morbidity among physicians engaged in end-of-life care for cancer patients: A cross-sectional nationwide survey in Japan. Psycho Oncol. 2007, 16, 421–428. [Google Scholar] [CrossRef] [PubMed]
  45. Bianchi, R.; Schonfeld, I.S.; Laurent, E. Burnout–depression overlap: A review. Clin. Psychol. Rev. 2015, 36, 28–41. [Google Scholar] [CrossRef] [Green Version]
  46. Bansal, P.; Bingemann, T.A.; Greenhawt, M.; Mosnaim, G.; Nanda, A.; Oppenheimer, J.; Sharma, H.; Stukus, D.; Shaker, M. Clinician Wellness During the COVID-19 Pandemic: Extraordinary Times and Unusual Challenges for the Allergist/Immunologist. J. Allergy Clin. Immunol. Pract. 2020, 8, 1781–1790.e3. [Google Scholar] [CrossRef]
  47. Shanafelt, T.D.; Hasan, O.; Dyrbye, L.N.; Sinsky, C.; Satele, D.; Sloan, J.A.; West, C.P. Changes in Burnout and Satisfaction with Work-Life Balance in Physicians and the General US Working Population Between 2011 and 2014. Mayo Clin. Proc. 2015, 90, 1600–1613. [Google Scholar] [CrossRef]
  48. Li, Z.; Ge, J.; Yang, M.; Feng, J.; Qiao, M.; Jiang, R.; Bi, J.; Zhan, G.; Xu, X.; Wang, L.; et al. Vicarious traumatization in the general public, members, and non-members of medical teams aiding in COVID-19 control. Brain Behav. Immun. 2020, 88, 916–919. [Google Scholar] [CrossRef]
  49. Sacco, T.L.; Copel, L.C. Compassion satisfaction: A concept analysis in nursing. Nurs. Forum 2017, 53, 76–83. [Google Scholar] [CrossRef]
  50. Radey, M.; Figley, C.R. The Social Psychology of Compassion. Clin. Soc. Work. J. 2007, 35, 207–214. [Google Scholar] [CrossRef]
  51. Collins, S.; Long, A. Working with the psychological effects of trauma: Consequences for mental health-care workers--a literature review. J. Psychiatr. Ment. Health Nurs. 2003, 10, 417–424. [Google Scholar] [CrossRef]
  52. Buzzi, N. Rapporto Sanità 2019—Il Personale Dipendente Delle Aziende Sanitarie Pubbliche; Nebo Ricerche PA: Rome, Italy, 2019. [Google Scholar] [CrossRef]
  53. WHO. Managing Epidemics: Key Facts about Major Deadly Diseases; World Health Organization: Geneva, Switzerland, 2018; ISBN 978-92-4-1565530. [Google Scholar]
  54. Weiner, J.A.; Swiatek, P.R.; Johnson, D.J.; Louie, P.K.; Harada, G.K.; McCarthy, M.H.; Germscheid, N.; Cheung, J.P.Y.; Neva, M.H.; El-Sharkawi, M.; et al. Learning from the past: Did experience with previous epidemics help mitigate the impact of COVID-19 among spine surgeons worldwide? Eur. Spine J. 2020, 29, 1789–1805. [Google Scholar] [CrossRef] [PubMed]
  55. Department of Civil Protection, Press Releases Section. Available online: www.protezionecivile.gov.it (accessed on 27 April 2020).
Table 1. Demographic characteristics of the study sample (N = 627).
Table 1. Demographic characteristics of the study sample (N = 627).
Sample CharacteristicsN (%)
SexMale
Female
125 (19.9)
502 (80.1)
NationalityItalian
Foreign
608 (97)
19 (3)
Household income (EUR/year)0–15,000
15,001–28,000
28,001–55,000
55,001–75,000
>75,000
56 (8.9)
235 (37.5)
220 (35.1)
65 (10.4)
51 (8.1)
Working with COVID-19 patients
N = 306 (48.8%)
Working in regions with higher rates of contagion and deaths
N = 210 (33.5%)
131 (42.8)
Working in other regions
N = 417 (66.5%)
175 (57.2)
Working without COVID-19 patients
N = 321 (51.2%)
Working in regions with higher rates of contagion and deaths N = 210 (33.5%)79 (24.6)
Working in other regions
N = 417 (66.5%)
242 (75.4)
Table 2. Mean differences between healthcare professionals working or not working with COVID-19 patients.
Table 2. Mean differences between healthcare professionals working or not working with COVID-19 patients.
Not Working with COVID-19 Patients
Mean (SD)
Working with COVID-19 Patients
Mean (SD)
F(1,623)Partial Eta Squared
Perceived Stress17.82 (7.27)19.78 (6.88)8.47 **0.013
Burnout26.38 (6.76)29.70 (7.35)24.01 ***0.037
Secondary Trauma23.95 (7.23)26.96 (8.58)18.74 ***0.029
Compassion Satisfaction45.36 (8.79)44.63 (8.15)0.170.000
Anxiety7.89 (4.28)8.95 (4.14)8.59 **0.014
Depression6.24 (4.02)7.37 (4.06)8.51 **0.013
Note: ** p ≤ 0.01; *** p ≤ 0.001.
Table 3. Mean differences between healthcare professionals working or not working in the most affected Italian regions.
Table 3. Mean differences between healthcare professionals working or not working in the most affected Italian regions.
Not Working in the Most Affected Regions
Mean (SD)
Working in the Most Affected Regions
Mean (SD)
F(1623)Partial Eta Squared
Perceived Stress18.07 (7.12)20.18 (6.99)7.93 **0.013
Burnout27.33 (7.05)29.33 (7.46)5.30 *0.008
Secondary Trauma25.13 (8.20)25.99 (7.76)0.120.000
Compassion Satisfaction45.59 (8.28)43.85 (8.79)5.28 *0.008
Anxiety8.22 (4.21)8.80 (4.29)0.910.001
Depression6.57 (3.97)7.22 (4.25)1.600.003
Note: * p ≤ 0.05; ** p ≤ 0.01.
Table 4. Distributions of the variables levels between healthcare professionals working or not working with COVID-19 patients.
Table 4. Distributions of the variables levels between healthcare professionals working or not working with COVID-19 patients.
Variables Not Working with COVID-19 Patients
(N = 321)
N (%)
Working with COVID-19 Patients
(N = 306)
N (%)
Total Sample N (%)Sig.
About the possibility of seeking psychological supportI do not need it N216 (67.3%)135 (44.1%)351 (56.0%)χ2(2 df) = 36.041 ***
Expected values179.7171.3
I considered it but I do not think I will start N67 (20.9%)95 (31.0%)162 (25.8%)
Expected values82.979.1
I think I will ask for it but I have not started yet N27 (8.4%)57 (18.6%)84 (13.4%)
Expected values4341
I have already started itN11 (3.4%)19 (6.2%)30 (4.8%)
Expected values15.414.6
N321306627
Note: *** p ≤ 0.001. adjusted standardized residual ≥±2.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Trumello, C.; Bramanti, S.M.; Ballarotto, G.; Candelori, C.; Cerniglia, L.; Cimino, S.; Crudele, M.; Lombardi, L.; Pignataro, S.; Viceconti, M.L.; et al. Psychological Adjustment of Healthcare Workers in Italy during the COVID-19 Pandemic: Differences in Stress, Anxiety, Depression, Burnout, Secondary Trauma, and Compassion Satisfaction between Frontline and Non-Frontline Professionals. Int. J. Environ. Res. Public Health 2020, 17, 8358. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17228358

AMA Style

Trumello C, Bramanti SM, Ballarotto G, Candelori C, Cerniglia L, Cimino S, Crudele M, Lombardi L, Pignataro S, Viceconti ML, et al. Psychological Adjustment of Healthcare Workers in Italy during the COVID-19 Pandemic: Differences in Stress, Anxiety, Depression, Burnout, Secondary Trauma, and Compassion Satisfaction between Frontline and Non-Frontline Professionals. International Journal of Environmental Research and Public Health. 2020; 17(22):8358. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17228358

Chicago/Turabian Style

Trumello, Carmen, Sonia Monique Bramanti, Giulia Ballarotto, Carla Candelori, Luca Cerniglia, Silvia Cimino, Monia Crudele, Lucia Lombardi, Silvia Pignataro, Maria Luisa Viceconti, and et al. 2020. "Psychological Adjustment of Healthcare Workers in Italy during the COVID-19 Pandemic: Differences in Stress, Anxiety, Depression, Burnout, Secondary Trauma, and Compassion Satisfaction between Frontline and Non-Frontline Professionals" International Journal of Environmental Research and Public Health 17, no. 22: 8358. https://0-doi-org.brum.beds.ac.uk/10.3390/ijerph17228358

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop