http://orcid.org/0000-0003-1504-1553
Figures
Abstract
Background
The Stigma Scale of the Explanatory Model Interview Catalogue (EMIC-SS) is a useful option to investigate leprosy-related stigma, but its psychometric qualities are unknown in Brazil. This study investigated the factor structure, the convergent and known-groups validity, and the reliability of the EMIC-SS for Brazilians affected by leprosy.
Methodology
The Brazilian Portuguese version of the EMIC-SS was validated in 180 persons affected by leprosy at a Reference Center in Rio de Janeiro. Confirmatory factorial analysis (CFA) and Cronbach alpha were used to assess the EMIC-SS internal consistency. The Construct validity was tested using Spearman Correlation, Kruskal-Wallis, and Mann-Whitney tests comparing with the Participation Scale, Rosenberg Self-esteem Scale, Beck Depression Inventory, and a Sociodemographic Questionnaire. Test-retest reliability was evaluated with intra-class correlation (ICC).
Main findings
CFA confirmed the one- and two-dimensional models of the scale after retaining 12 of the 15 EMIC-SS items. The 12—item EMIC-SS was consistent (α = 0.78) and reproducible (ICC = 0.751, 95% Confidence Interval = 0.657–0.822, p < 0.0001). A significant correlation was observed between the EMIC-SS and the other scales confirming convergent validity. The EMIC-SS and its factors were able to differentiate several hypothesized groups (age, change of occupation, monthly family income, communicating others about the disease, and perception of difficulty to follow treatment) confirming the scale known-groups validity, both in its one and two-dimensional models.
Conclusions/Significance
Our study found support for the construct validity and reliability of the EMIC-SS as a measure of stigma experienced by people affected by leprosy in Brazil. However, future studies are necessary in other samples and populations with stigmatizing conditions to determine the optimal factor structure and to strengthen the indications of the validated scale.
Citation: Frota da Rocha Morgado F, Kopp Xavier da Silveira EM, Pinheiro Rodrigues do Nascimento L, Sales AM, da Costa Nery JA, Nunes Sarno E, et al. (2020) Psychometric assessment of the EMIC Stigma Scale for Brazilians affected by leprosy. PLoS ONE 15(9): e0239186. https://doi.org/10.1371/journal.pone.0239186
Editor: Lucy Busija, Monash University, AUSTRALIA
Received: June 28, 2019; Accepted: September 1, 2020; Published: September 17, 2020
Copyright: © 2020 Frota da Rocha Morgado et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Most relevant data are within the manuscript. Further data are available from the Leprosy Laboratory, Oswaldo Cruz Institute (contact via asa.lahan@ioc.fiocruz.br) for researchers who meet the criteria for access to confidential data.
Funding: The author(s) received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The past decade has been important for developing the new science of stigma, a fundamental social cause of health inequalities. The paradigm of stigma has changed and today it is viewed as a cultural disease. It affects processes such as resource availability, social relationships, psychological, and behavioural responses, all of which mediate adverse population health outcomes [1]. At the individual level, the exclusion, rejection, and guilt or devaluation sensation resulting from the experience, perception, or anticipation of an adverse social judgment [2], might affect the health outcome of the stigmatized person. This judgment is based on a long-life characteristic of the identity conferred to a specific health problem, a condition, or social aspects [3].
Usually, stigma frameworks focus on specific diseases in isolation [4]. But leprosy, unlike other stigmatized diseases, is a composite of different conditions. It is a chronic infectious disease that mainly affects the skin and the peripheral nerves, which may be aggravated during the immunoinflammatory episodes known as leprosy reactions. The presence of overt skin lesions, physical disability, and deformities, all of which may remain even after release from treatment [5], can all lead to discrimination. Moreover, leprosy affects the most deprived individuals in society [6]. Leprosy has diverse social, economic, cultural, and political dimensions amassed into a deeply stigmatized social category that, throughout history, as well as in contemporary societies, has been used mainly to exclude the affected people [7]. Thus, the stigma experience of persons affected by leprosy might be amplified due to intersecting stigmas, which need to be addressed when studying and understanding the stigma framework in leprosy.
Leprosy-related stigma has been the most persistent and immanent form of prejudice, social injustice, and discrimination that societies have inflicted on human beings throughout history. It adversely impacts the psychological and emotional wellbeing of the affected people, who are denied their most basic human rights. Being unjustly blamed for their disease, or feeling excluded or punished for supposed wrongdoing, the people affected by leprosy suffer from low self-esteem, fear, shame, guilt, anxiety, hopelessness, anger. As expected, this has damaging effects in various life domains, leading to self-isolation and thus, withdrawal from work and social life. At the societal level, it undermines social relationships, enabling rejection, forced isolation, and restricted social participation [8]. Finally, leprosy-related stigma is a major contributor to the burden of illness and may influence effective case finding and treatment, which are key elements for disease control. In Brazil, the long history of involuntary isolation of individuals affected by leprosy, added to years of beliefs, myths, and misconceptions regarding its cause, transmission, treatment, and consequences [9–11], have left a profound scar in our society. The Brazilian government has strived to reduce leprosy-related stigma and to improve social integration. An initial attempt was by changing the term leprosy to hanseniasis, initiated during the 1970s, and officially recognized by federal law in 1995 [12].
Leprosy control measures to reduce transmission and stigma have been mainly directed towards early diagnosis and adequate treatment to prevent disabilities, as well as health education, and assessment of activity limitation and social participation [13]. However, to only look at visible sequelae and the physical attributes as the main causes of stigma has been rendered obsolete [2].
The Explanatory Model Interview Catalog (EMIC) is an adaptable semi-structured interview developed by Weiss and colleagues in response to the challenge of integrating clinical, epidemiological, and social-science frameworks. It was created from open-ended pilot interviews with psychiatric and medical patients in India [14]. Based on Kleinman’s explanatory models, the EMIC incorporates the advances in cultural psychiatry to analyse the relationships between illness-related beliefs and practices, and clinical outcomes [15].
The EMIC interviews have been used to assess stigma in various conditions [16–18]. Questions considered indicators of stigma have been combined into a scale, the EMIC Stigma Scale (EMIC-SS), which is variable in the number of items. According to the condition evaluated and the context of the application, the EMIC-SS can have between 12 [19, 20] up to 17 items [21]. This adaptable scale has been validated in several languages to study stigmas produced by various neglected diseases and conditions, such as Buruli ulcer [22], onchocerchal skin disease [17, 23], leishmaniasis [19], leprosy [21], tuberculosis [2], depression [20], schizophrenia [24], HIV/AIDS [25] and physical disabilities [21].
The EMIC-SS evaluates both the experienced (ES) and the perceived stigma (PS) in one-dimensional [16, 26] as well as two-dimensional models [27]. The ES is the direct stigma suffered by an individual, such as discrimination, rejection, abuse, job loss, or divorce. It occurs when a member of society behaves negatively towards the person affected by a condition. PS refers to the perception, expectation, or fear of experiencing stigma and the awareness of negative attitudes or practices in society [28, 29].
Stigma is the main element that leads to the distress associated with medical conditions and illness behavior [15], so it needs to be systematically assessed and quantified to enable consistent public health policies and to measure interventions against discrimination and prejudice. Generic instruments such as the EMIC-SS are important to measure the extent and severity of stigma, and its change over time [30]. Well validated instruments are essential to compare data from different countries and settings [31]. Several studies have shown satisfactory values of the EMIC-SS construct validity, internal consistency, and test-retest reliability. Rensen et al [21] studying 806 people affected by leprosy in India, found an internal consistency of 0.88, a test–retest reproducibility of 0.70, no floor and ceiling effects, and strong positive correlations between the 17-Likert items EMIC-SS and the Internalised Stigma of Mental Illness (0.70) and the Participation Scales (0.68). Vlassoff et al [17] studied 468 patients with onchocercal skin disease in Cameroon, Ghana, Nigeria, and Uganda, found a Cronbach's alpha of 0.80 in a 13-item EMIC-SS. The 15-item EMIC-SS was validated in 260 adults with non-leprosy physical disabilities in Hong Kong [32], demonstrating good internal consistency (0.897), a moderate correlation (r = 0.48, p = 0.001) with the Participation Scale, and good reliability (the person and item reliabilities were 0.74 and 0.90, respectively).
At present, Brazilian researchers do not have a validated tool to quantify leprosy-related stigma. Considering that Brazil is one of the countries with the highest number of leprosy cases, with a high prevalence of disabilities [33], and that the current Global Leprosy Strategy 2016–2020: Accelerating towards a leprosy-free world [34] calls all national leprosy programmes to stop discrimination, it is imperative to quantify stigma due to leprosy.
The EMIC-SS recently underwent a rigorous process of cross-cultural adaptation for the Brazilian population [35], but its psychometric qualities are still unknown. We expect to confirm the internal consistency of its one- and/or two-dimensional structures that has been shown in previous studies [27] using confirmatory factor analysis (CFA) and Chronbach alpha. We also planned to evaluate the construct validity and the reproducibility of the Brazilian Portuguese version of the EMIC-SS.
The risk factors described to promote stigma are similar across cultures, regardless of the prevalence, severity, or type of stigma. Among them, visible deformities and disabilities lead to high levels of both felt stigma and self-stigmatization in persons affected by leprosy [21, 29]. Stigma has also been found associated with misconceptions or poor knowledge about the illness, low educational level, and monthly family income [25, 26]. A high level of stigma has been reported in the unemployed or individuals unable to secure employment and, thus that generate no income [26]. Stereotypes and biases are also common regarding obese individuals [36]. In general, age, educational level, income, physical activity, and BMI have shown a correlation in previous studies of stigma related to various conditions [20, 29, 36].
The consequences of stigma are variable in the person affected by leprosy, ranging from psychosocial dysfunction to isolation, depression [37], and low self-esteem [21]. The prevalence of psychiatric disorders among people affected by leprosy is higher than among the general population. In particular, depression is the most prevalent and may lead to decreased social participation and social exclusion [37]. Social participation restrictions are recognized as the result of the stigmatizing attitudes suffered by the persons affected by leprosy [38]. Previous researches have shown a reciprocal relationship between participation and stigma, where a high percentage of patients report experiencing restriction due to stigma [21, 25]. The opposite is observed in individuals who practice physical activity, that are expected to be more resilient to the negative attitudes suffered [39].
Therefore, we formulated the hypothesis that the EMIC-SS score would have significant moderate to high correlations with theoretically related constructs and measures, namely participation restriction, depression, and low self-esteem to assess the construct validation. Besides, the EMIC-SS scores would have significant low correlations with factors that have been associated with stigma, including the social, demographic, and cultural context, psychosocial, and various characteristics of leprosy [21, 26, 29, 36], such as income, educational level, BMI, and frequency of physical activity, were selected for the discriminant validation of the EMIC-SS and its factors.
Materials and methods
Participants
Participants were recruited among the patients treated or under surveillance at the Souza Araújo Outpatient Clinic (ASA), Fiocruz, Rio de Janeiro, a leprosy reference center certified by the Joint Commission International. Persons affected by leprosy aged 18 years or older were included during or after multidrug therapy (MDT). People affected by other stigmatizing diseases (e.g. HIV/AIDS); those with previous MDT (were on leprosy relapse or retreatment) or had been treated for less than 2 months, and people with cognitive impairment were excluded.
The sample size to perform the CFA was calculated according to established criteria that recommend at least 10 participants for each item of the scale for an adequate analysis of the model fit and validation [31, 40, 41]. Thus, considering that the scale is composed of 15 items, a minimum of 150 participants needed to be included. The same sample used in the CFA analysis was used to assess the construct validity. The sample size for the reproducibility analysis depended on the participant’s compliance with the appointment.
Instruments
Inventory Explanatory Model Stigma Scale (EMIC-SS).
The cross-culturally adapted EMIC-SS for persons affected by leprosy in Brazil [35] has 15 questions with four Likert-type options: no (0), not sure (1), possibly yes (2), and yes (3). The answer "yes" indicates a strong signal of stigma therefore, it has assigned the highest value (3 points), except for item 2 that has a reversed score. Thus, the higher the total score obtained by the sum of responses, the more severe the stigma perceived by the person affected by leprosy.
We divided the EMIC-SS into the two factors suggested by Weiss [27] by theoretically separating the items to structure each factor according to the definitions of perceived and experienced stigmas.
Participation Scale (P Scale).
The scale has good evidence of validity and reliability [15, 26, 38]. It is composed of 18 items that evaluate social participation and the impact of stigma [38]. The individual ranks his/her participation comparing with "peers", ie. people similar to him/herself in all senses but the disease. The response options are scored as not relevant (0), no problem (0), small problem (1), medium problem (2), large problem (4). High P-Scale scores indicate restrictions on social participation.
Rosenberg Self-esteem Scale (RSS).
It evaluates a set of self-esteem and self-acceptance emotions. It consists of 10 statements measuring both positive and negative feelings about the self and a Likert-type scale that ranges from strongly agree (1), to strongly disagree (4). The scale is one-dimensional with the highest scores (maximum of 40) indicating a good self-esteem level. The minimum score possible is 10. The scale was adapted and validated in Brazil [42] and had its factorial structure revised by Hutz and Zanon [43].
Beck Depression Inventory (BDI).
Composed of 21 statements and a Likert-type scale that ranges from 0 to 3, the BDI is used to detect depressive symptoms. The total score results from adding each response. A high score is suggestive of depression. The instrument was adapted and validated in Brazil and presented good signs of validity and reliability [44].
Socio-demographic questionnaire.
Developed specifically for this study, it included age, gender, ethnicity, marital status, education level, occupation, monthly family income, physical activity, and general knowledge about the disease, among other items. Clinical data were obtained from the patient’s file.
Procedure
Four experienced and trained female researchers interviewed the participants and collected data between September 2016 and April 2018. The persons affected by leprosy, previously selected from the ASA database based on the inclusion criteria, were invited to participate while waiting for his/her consultation at the clinic. After free and informed consent each participant was interviewed in a private room for an average of 40–60 minutes when the socio-demographic questionnaire and the scales were applied. The BDI and RSS were offered to be self-applied, except when the participant was illiterate.
To evaluate the test-retest reliability, the same or different interviewer applied the EMIC-SS at the follow-up visit for patients that were under treatment, or 15–30 days after the first interview to participants that came for their annual visit. Non-compliant participants were reinvited up to 2 times.
Data analysis
The EMIC-SS dimensionality was evaluated by CFA using the LISREL 7 program [45]. The CFA is a theory-driven technique that allows confirming or refuting theoretical and hypothesized factor structures by comparing the statistical fit of the hypothesized model with the fit of theoretically alternative models [46]. If the hypothesized factor structure fits the data, then the factor structure replicates. If it does not fit the data, modification indices are used to inform the localization of the factor pattern constraints that caused the misfit [47, 48]. Standard indices values were used to evaluate the model fit: Goodness-of-fit (GFI) > 0.90, Adjusted Goodness-of-Fit (AGFI) > 0.95, Non-normal Fit (NNFI) > 0.95, Comparative Fit (CFI) ≥ 0.95, Standardized Chi-square (χ2/df) < 3.0, and Root Mean Square Error of Approximation (RMSEA) <0.08. Items were dropped if they had factor loadings (λ) ≤ 0.3 [49]. Missing data were deleted listwise. One and two-dimensional models were tested in all subsequent analyses.
The SPSS 19.0 software was used for descriptive and inferential analysis, considering a 95% confidence level. Internal consistency was also assessed with the use of Cronbach's alpha and was considered acceptable with alpha values greater than or equal to 0.7. The presence of floor and ceiling effects was identified when 15% of the participants or more had the lowest (0) and the highest possible scores for the EMIC-SS and its factors [31].
Construct validity was evaluated by testing convergent and known-groups validity. It was considered sufficient if at least 75% of the results were in correspondence with the hypotheses [31]. Convergent validity was assessed using the Spearman correlation. Considering the non-parametric characteristics of the EMIC-SS and the two factors scores, Kruskal-Wallis and Mann-Whitney tests, for non-binomial and binomial variables, respectively, were used for the known-groups validity and exploratory analyses. Post-hoc analysis was used to define the pairs presenting differences of medians.
Regarding convergent validity, we hypothesized that the EMIC-SS score would be related to restriction on social participation, depressive symptoms, and low self-esteem. Thus, we expected a moderate positive correlation (a Spearman correlation coefficient between 0.4 and 0.8) between EMIC-SS, BDI, and P-Scale total scores, and that a moderate but negative correlation would exist between EMIC-SS and RSS. Moderate to high correlations were also hypothesized for the PS and ES factors of the EMIC-SS. Besides, we hypothesized that the EMIC-SS scores would show a significant low correlation with some related socio-demographic and clinical characteristics consistent with discriminant validity testing: the EMIC-SS would have a positive correlation with age, educational level, and BMI (calculated from self-reported height and weight), while it would have a negative correlation with monthly family income, and frequency of physical activity.
Considering known-groups validity, we expected the EMIC-SS to allow differentiating groups related to socio-demographic aspects, knowledge about leprosy, and clinical variables. For these analyses, we categorized schooling and income into binomial variables. Because of the low correlation between the EMIC-SS and PS factor with age, we collapsed it into 20-year groups to assess the group with the highest EMIC-SS scores. Also, BMI was categorized to evaluate the association of obesity in leprosy patients as a stigma enhancer.
According to previous studies on stigma, we hypothesized that females, the elderly, the participants self-declared as black, the unemployed, those who changed occupation when diagnosed with leprosy, that had low schooling level or low income, and those who practiced little or no physical activity would have higher scores than the other categories of each variable. On the other hand, participants who communicated to others about their disease or had adequate knowledge about leprosy would have EMIC-SS scores in the low range. Regarding the clinical characteristics, we expected participants with a disability or visible deformities, complications, or that had BMI within the obesity range, would have higher scores than other groups in the variable.
Finally, regarding reproducibility, we assessed the test-retest reliability using the type C intra-class correlation (ICC) for single measures and the two-way mixed-effects model where people effects are random and measures effects are fixed. Reliability was considered acceptable with an ICC agreement of at least 0.70 within the 95% confidence interval [50].
Ethical issues
The Oswaldo Cruz Foundation/Fiocruz Institutional Review Board approved the study (Study Registry—CAAE 50625615.9.0000.5248) and Mitchell G. Weiss, the first author of the original scale [3], granted us authorization for the cultural adaptation and validation of the EMIC-SS in Brazil. It is important to note that, following the Brazilian regulation for human research, the participants voluntarily accepted to participate and received no payment for their participation.
Results
General characteristics of the sample
A total of 208 individuals were invited to participate, 13 declined the invitation mainly due to lack of time, and 15 people were excluded (Fig 1).
MDT = multidrug therapy.
The final sample consisted of 180 individuals, between 18 and 89 years of age, mean age of 49.3 years (SD = 15.14). The participants lived in Rio de Janeiro and surrounding cities but came from States of the five regions of Brazil (Table 1). A high proportion of cases (55.7%) had less than 10 years of schooling (median = 9, min = 0, max = 16). Mean Body Mass Index (BMI) was 26.89 ± 4.82 Kg/m2, and 21% of the participants were classified as obese (BMI ≥ 30).
Internal consistency
Confirmatory factor analysis.
Due to the absence of multivariate normality, the factor model was estimated by the unweighted least squares method. Based on previous studies [16–18], we performed the CFA on the 15 items in two hypothesized factorial models: (1) one-dimensional model and (2) two-dimensional model composed of the factor that evaluates PS (items 1, 2, 3, 4, 5 and 14) and the factor that studies ES (items 6, 7, 8, 9, 10, 11, 12, 13 and 15).
The one-dimensional model did not provide an adequate fit to the data (χ2/df = 2.18, RMSEA = 0.082, GFI = 0.94, AGFI = 0.92, NNFI = 0.94, CFI = 0.95). Only the CFI was at cutoff for acceptable fit. Items 1 (λ = 0.291), 2 (λ = -0.075) and 12 (λ = 0.244) had low factor loading and were dropped. The new model retained 12 items and provided a good fit to the data (χ2/df = 2.02, RMSEA = 0.076, GFI = 0.96, AGFI = 0.95, NNFI = 0.99, CFI = 0.99). All of the items included had their factorial loads ranging from λ = 0.368 to 0.651 (Table 2). Thus, we confirmed the factor structure of the 12-item EMIC-SS one-dimensional model.
The two-dimensional model also did not provide an adequate fit to the data in the analysis of the whole scale (χ2/df = 2.18, RMSEA = 0.081, GFI = 0.94, AGFI = 0.92, NNFI = 0.94, CFI = 0.95). Like the one-dimensional scale, the RMSEA values were above cutoff and GFI, AGFI and NNFI were below cut-off for acceptance. Items 1, 2 and 12 had low factor loading (λ = 0.299, -0.082 and 0.251, respectively), and thus, they were dropped. The new two-dimensional model provided a good fit to the data (χ2/df = 2.06, RMSEA = 0.077, GFI = 0.97, AGFI = 0.95, NNFI = 0.99, CFI = 0.99) and its factor structure was also co-signed. All of the 12 items (PS factor = 4 items; ES factor = 8 items) had their factorial loads ranging from λ = 0.374 to λ = 0.681 (Table 2). After the exclusion of the three problematic items, the one and two-dimensional models had their factorial structures confirmed.
Regarding these problematic items, item 2, had only yes (0) or no (3) answers by the 180 participants (Fig 2). Items 2 and 12 had the highest proportion of 0 responses, 93.3% and 85%, respectively, although most of the items had a no (0) answer, except for items 1 and 4.
0 = no, 1 = not sure, 2 = possibly yes and 3 = yes. NB. Question 2 has a reversed score.
Cronbach alpha
The one and two-dimensional 12-item models showed variable internal consistency indices (Table 2). The EMIC-SS total score and the ES factor had good internal consistency, Chronbachs’s alphas of 0.78 and 0.71, respectively, the PS factor, on the other hand, had a moderate internal consistency (α = 0.63).
Floor and ceiling effect
No floor or ceiling effects were identified for the whole scale: One (0.6%) of the participants had the highest possible score of 36 in the EMIC-SS while 10.6% of the participants scored 0. No ceiling effect was observed for either of the factors, 2.2% of the participants scored 12, the maximum score in the PS, while 1.1% of the participants scored 24, the maximum score in the ES. However, in both the factors a floor effect was identified, since 38% and 20.1% of the participants had the lowest possible score of 0 in the PS and the ES, respectively.
The next steps in the data analysis were to evaluate the reliability and the convergent and known-groups validities. All the analyses included both the total score of the scale (one-dimensional model) and the scores by each one of the factors (two-dimensional model).
Construct validity
Two measures of construct validity were evaluated: convergent and known-groups validity. Three scales that evaluate constructs related to stigma manifestations, such as social participation, depression, and self-esteem were used for convergent validity (Table 3). RSS and BDI were self-applied in a few cases, 17 (9.7%) and 24 (13.9%) participants, respectively.
Convergent and discriminant validities
The correlations observed between the scales compared confirmed the a priori hypothesis (Tables 4 and 5). Moderate positive correlations were observed between the total scores of the 12-item EMIC-SS when compared with the BDI and P Scale, and a moderate negative correlation was found with the RSS scores (Table 4). The PS and ES factors had moderate positive correlations with BDI scores, but low correlations with P Scale and RSS scores. The total score and the PS factor showed a low and negative correlation with age (rs = -0.224 and -0.240, respectively). ES factor also showed a very low and negative correlation with monthly family income (rs = - 0.165). The other variables evaluated, educational level, BMI, and the frequency of physical activity showed no correlation with the EMIC-SS or its factors (Table 5).
Given that all of the correlations of the EMIC-SS with the instruments were in correspondence with the hypotheses we confirmed its convergence validity.
Known-groups validity
For known-groups validity, we verified if the 12-item EMIC-SS and its two factors could differentiate groups concerning socio-demographic aspects, knowledge about leprosy, and clinical variables (Tables 6–8). The EMIC-SS and the ES factor allowed discriminating groups related to age, change of occupation after leprosy diagnosis, monthly family income, and communicating about leprosy diagnosis to others. The PS factor only discriminated according to age categories (Table 6).
The median scores of the 12-item EMIC-SS and the factors were significantly different between the three age groups: χ2 = 12.558 (p value = 0.002) for the EMIC-SS, χ2 = 10.603 (p value = 0.005) for the PS factor, and χ2 = 10.290 (p value = 0.006) for the ES factor. Participants 60 years old or more had a significantly lower scores (medians of 3.5, 3.0 and 0 for the EMIC-SS, ES and PS, respectively) than participants between 18–40 (medians of 7.0, 4.0, and 3.0) and 41–59 years old (medians of 7.5, 5.0 and 3.0).
As expected, participants that changed occupation when diagnosed with leprosy had a significantly higher score (median EMIC-SS score = 13) than those who remained at their jobs (median EMIC-SS score = 6). Likewise, participants with a low income had higher scores (median EMIC-SS score = 6.5) when compared to those with higher family income (median EMIC-SS score = 5.5).
All of the participants, but two, mentioned having told about their diagnosis to someone. Nevertheless, 12 participants denied discussing the problem with the person closest to them at the EMIC-SS interview (answered “no” to item 2, Fig 2). Those who told everyone about their disease (median EMIC-SS score = 5) also had significantly lower scores than those who only told to family members (median EMIC-SS score = 7), or to family and friends or people at work (median EMIC-SS score = 6.5).
Out of the 76 participants that referred to knowing what caused leprosy, 88.2% told the exact cause: bacteria, bacilli, or microorganism. Asked about how they get the disease, 62.8% said they knew, but in fact, only 4.9% knew the correct form of the transmission. For 68% of the cases, the transmission was via the air or wind, which we considered a wrong answer. The scales did not allow discriminating groups about this knowledge. Nevertheless, EMIC-SS and the ES factor discriminated between knowing or not about signs and symptoms (Table 7). Among the symptoms more frequently cited by the participants were skin patches (37.8%) and numbness (32.9%). Interestingly, six participants referred to fear, sadness, shame, isolation, or depression. Participants who indicated several symptoms of the disease had higher scores (median = 6 and 4, for EMIC-SS and ES factor, respectively) than participants who were unable to cite any symptoms or only one (median = 3 and 2, for EMIC-SS and EF factor, respectively). In addition, the PS factor distinguished between participants that searched for information about the disease and those who did not. Participants referred to looking for information only at the media (40.4%), only at the health center (28%) and only from acquaintances (4.3%), or in all of them (27.2%). Surprisingly, those who referred to not searching for information had significantly lower scores (median = 1) than the participants who referred to looking for information (median = 3).
Regarding the clinical data, the scale and its factors discriminated only between participants with difficulties in handling his/her treatment (Table 8). People who referred finding the treatment difficult had significantly higher (p value < 0.001) EMIC-SS scores (median = 14), PS factor scores (median = 6) and ES factor scores (median = 9) than those who did not (EMIC-SS median = 6 and PS and ES factors median = 3).
Reproducibility
To determine the reproducibility of the EMIC-SS, we evaluated the stability measure of the test applied on two different occasions in the same person. A total of 112 (62%) participants returned to the clinic less than three months after the day of inclusion. The retest was applied at a mean of 36.5 ± 18.2 days after the first interview. We expected this period to be long enough to prevent recall, and short enough to ensure that the participants have had no significant clinical change.
Only 21.4% of the participants maintained the same score on the 12-item scale. However, a good reliability was observed indicated by the test-retest ICC of 0.751 (95% Confidence Interval = 0.657–0.822, p < 0.0001) for the 12-item scale. For the ES factor, a just above cut-off ICC value of 0.702 (95% Confidence Interval = 0.595–0.785, p < 0.0001) was observed, and an insufficient ICC for the PS factor, 0.636 (95% Confidence Interval = 0.511–0.735, p < 0.0001).
Discussion
The Global leprosy strategy 2016–2020: accelerating towards a leprosy-free world [24] calls for action to reduce stigma and discrimination and to promote the inclusion of persons affected by leprosy. This study evaluates the factor structure, validity, and reliability of EMIC-SS applied to persons affected by leprosy in Brazil, making available an important tool for researchers to allow the production of information about the prevalence of social stigma in the country. It can be an aid to obtain data for monitoring stigma in Brazil, which is an essential indicator of the third pillar of the strategy: “Stop discrimination and promote inclusion”.
Our results demonstrated that the scale is a valid and reliable measure of stigma in the sample studied. The factorial structures of the two models fit the observed data after reducing the number of 15 items to 12. It is worth remembering that the elimination of items presenting problems in their structure is one of the basic assumptions in the instrument validation process [40, 41]. This is an accepted procedure in scale validation studies [52]. Other authors have decided to retain or drop items from the EMIC only guided by the Cronbach alpha statistic [16–18]. The selection of items in the scale based on statistical criteria should take into consideration how it affects the measurement properties of the instrument. However, the Stigma Scale obtained from the semistructured EMIC inter-view is not a fixed scale, so its component items can differ in both number and type in the different studies according to the population and condition under evaluation.
It is interesting to note that the three problematic items were the same in both models. Several hypotheses may warrant such findings. In the first place, the content of these three items may not have been sensitive to the stigma assessment in the sample and, therefore, did not present good adjustments in the models tested. Another hypothesis would be related particularly to the inability of items 1 and 2 to discriminate groups. These two items were also considered problematic in a validation study for people with non-leprosy disabilities [32]. Although frequently patients omit that they are affected by leprosy, as reported in Northeast of Brazil [53] where 74.8% of 107 participants had not disclosed their condition to others, our study pointed out that only 2 of the 180 participants concealed this information. This fact may have influenced the dropping of item 1, which seeks to know the potential for the omission of informing others about the disease, and item 2, which evaluates the interviewee's willingness to talk about the disease with close contacts. The third hypothesis is related to the cross-cultural adaptation. Some of the items translated into Portuguese may have used words or phrasal structures that are not sufficiently clear for the reality of the population. This may have been particularly the case of item 12, which raised doubts regarding the term "to have a love relationship with another person" that was used instead of the original “to marry”. Many couples in Brazil live together as partners without a formal marriage, but it was common for participants to understand the question as loving oneself or to ask for an explanation. Specifically regarding item 2, the question structure only allows a yes or no answer. Thus, this item could be separated from the scoring calculation, but still can be maintained as part of the instrument.
The internal consistency of the scale and its factors was moderate. The PS factor had the lowest alpha coefficient, probably due to the small number of items included in this factor. The alpha coefficient is proportional to the number of items in a scale or factor. Scales with many items are more reliable and have high alpha values. Thus, the one-dimensional model of the EMIC-SS with the highest number of items (12 items) tends to be more reliable (with greater internal consistency) than the two-dimensional model in the evaluation of stigma. The alpha coefficient for the one-dimensional model approached values reported in previous studies [16, 21].
We hypothesized that the EMIC-SS could be associated with theoretically related constructs such as depression, restriction of social participation, and low self-esteem. Thus, the BDI, P Scale, and RSS, three well-known validated instruments, were used for convergent validity analysis. As hypothesized, the total scores of the scale and its factors had moderate positive monotonic correlations with depression and restriction in social participation, and negative monotonic correlations with self-esteem, confirming the convergent validity for the one and two-dimensional models.
Other hypothesized constructs such as age, educational level, income, physical activity, and BMI have shown a correlation in previous studies of stigma related to various conditions [20, 29, 38, 53–55]. We observed a weak negative monotonic correlation of the EMIC-SS and the PS factor scores and age. However, no association was established with educational level, income, frequency of physical activity, nor with BMI. The Quetelet Index (kg/m2) was calculated from self-reported height and weight, which may be inaccurate. But in a study about teen obesity, the authors recommended the use of the self-reported parameters to study obesity [56]. However, they reported that weight perception in teens and their parents was not related to BMI. Discrepancies between image perception regarding weight perception and BMI are common. In obese individuals recovering normal body mass, the effects of stigma persist regardless of the body mass [39].
By looking at different factors previously reported to be associated with leprosy-related stigma such as the socio-demographic and clinical aspects, and the participants’ knowledge about leprosy, we showed that the EMIC-SS and its factors were able to discriminate several groups. Age groups presented significant differences in the three scores. The scores of participants from 18–39 years old were significantly higher than those of elderly participants, as shown by Shi-Jie et al. [20]. However, other authors have reported the association of stigma with older-age [57] or no significant differences with age [58]. Even though the contradictions found in the literature, the results suggest the validity of the scale and its factors discriminating age groups.
Leprosy commonly affects less favored social classes, reduces the quality of life and work capacity of the affected person. These factors may contribute to perpetuating leprosy-associated stigma [24]. Accordingly, low socioeconomic status has been associated with stigma [29, 58]. Similarly, in our sample, the EMIC-SS and the ES factor discriminated groups regarding the monthly family income.
The EMIC-SS did not discriminate groups related to employment status. Although unemployed participants had higher scores on the scale and its factors, they were not significantly higher than employed participants or other groups. Previous studies report that unemployment is an important risk factor for negative attitudes towards people with leprosy [29, 58]. Besides, a recent study in China [20] showed significant differences in stigma scores between retired people affected by leprosy and full-employed participants.
Consistent with Adhikari et al. [59], which found high stigma among participants in Nepal who changed their occupation after diagnosis, our findings also revealed that participants who reported having changed their occupation after receiving the diagnosis of the disease presented higher scores on the scale and the ES factor. It is another indication of the known-groups validity of the scale.
A clear sign of stigma is hiding the diagnosis of leprosy [53]. Only ES factor discriminated group concerning telling others about leprosy. The omission is a cultural pattern of confrontation or even a defence mechanism against negative conceptions of society. The ES factor is especially sensitive to the experiences of discrimination suffered, which would lead to avoiding telling others about the disease.
Traditional beliefs and lack of knowledge may lead to the persistence of stigma. Among the several variables related to knowledge about leprosy, we observed that the EMIC-SS or its factors discriminated groups only regarding the search for information about the disease and being able to name signs and symptoms of leprosy. The participants mostly named their symptoms which are related to their own experience of illness. In this way, their self-perception of the disease could be reflected in the higher ES factor scores observed in this group of participants in comparison to those who were not able to identify signs or symptoms. It may also explain why only ES factor that reflects the direct stigma suffered by the individual, and not the PS factor, was able to discriminate between those who knew from those who did not know the signs and symptoms. Interventions to address stigma in leprosy endemic countries such as Thailand and Nepal have proven effective when including self-care practice, tailored-made leprosy information, and promotion of income-generating activities in people affected by leprosy, the community, and health care workers [60, 61]. Most of the people we interviewed had some sort of income, had schooling, and were encouraged to practice self-care. These characteristics may reduce the capacity of EMIC-SS and its factors to discriminate groups in our sample according to their knowledge of leprosy.
Only the PS factor discriminated participants who searched for information from those who did not. At the outpatient clinic, the patients receive information about the disease, the treatment, and possible complications in a program of health education aimed to increase patients’ knowledge regarding the disease, its complications, and sequelae. Most participants referred to searching for information about the disease, mainly in media and with health professionals. PS reflects the fear of the people affected by leprosy of experiencing stigma, which could drive them to look for information about the disease [4].
Regarding the clinical characteristics evaluated, the EMIC-SS and its factors only discriminated between patients that had difficulty in managing their treatment from those who easily handled their treatment. Similarly, Adhikari et al. [59] found that persons affected by leprosy who considered it difficult to handle treatment had higher scores in the EMIC as compared to those who did not consider it difficult. It is widely known that delayed treatment and poor adherence to treatment are common in people who experience, perceive, or internalize health-related stigma [4].
EMIC-SS and its factors did not discriminate groups concerning the grade of physical disability and visible deformities. Studies in India [21] and Nepal [59] have found higher stigma perception in individuals with deformity when compared to those without. These contradictory findings could be related to the diverse cultural context of the three countries (Brazil, India, and Nepal). Cross and Choudhary [61] investigated the effect of the Stigma Elimination Program (STEP) on the social participation of persons affected by leprosy aged between 15–65 years, in Nepal using the P Scale as a proxy scale for measuring stigma. The authors found that only 10.5% of the 76 participants with a visible disability that received the intervention had restricted participation, while among the 81 with disability from the non-STEP villages, 38% reported participation restriction. At the outpatient clinic, the patients are followed closely by a multidisciplinary team, with support from a physiotherapist and social worker, as part of the program of prevention of disabilities. Although not focused on stigma, multidisciplinary support may have contributed to the low scores observed in patients with deformities. Thus, interventions with a focus on stigma reduction, such as continuous health education and support groups, accessibility to physical rehabilitation centers, may be useful in making people with deformities more responsive to stigma.
The ability of the EMIC-SS and its factors to differentiate several hypothesized groups (age, change of occupation, monthly family income, communicating others about the disease, search for information about the disease, knowledge about leprosy signs and symptoms, and perception of difficulty to follow treatment) confirms the scale known-groups validity, both in its one and two-dimensional models. It is important to note that most of the groups were related to socio-demographic variables. When comparing the scores of EMIC-SS and PS and ES factors, it was possible to observe that EMIC-SS and ES factor were able to discriminate the same six groups, while the PS factor, only discriminated three groups, one of which was not discriminated by the whole scale nor the ES factor. Again, as for the reliability, the limited PS factor of discriminating groups may be associated with the few numbers of items included in this factor, which was below the recommended value of 5 for a strong and stable factor [37].
The Brazilian Portuguese version of the EMIC-SS was reliable. The strong ICC found between the test and retest ensures the stability of the instrument. Another validation study in India [21] also reported a strong association between test and retest, corroborating the results of this research.
Regarding the total scores of the EMIC-SS, some aspects need consideration. The values observed in the present study were below previous studies that have used the scale. Rensen et al [21], in a sample of 806 Indians affected by leprosy, found a mean EMIC score of 13.8 ± 12 (ranging from 0–54); Rafael et al. [16], with a sample of 80 people with epilepsy in Benin, found a mean of 21.9 ± 11.8 (ranging from 0–48). The apparent difference in the total scores is related to the different versions of the EMIC-SS that have been used in various cultural contexts. While in our study the models used to analyze the scores resulted from the present validation process, with 12 items in the one and two-dimensional versions, the studies by Rensen et al. [21] and Rafael et al. [16] used the EMIC-SS with 17 and 15 items respectively, some of which are a different in content. It is worth remembering that the versions of the scales validated in different countries with different number of items make it difficult to compare the scores obtained in cross-cultural studies. Future studies could continue to explore the factorial structure of EMIC-SS in different cultural contexts to obtain more homogeneous versions of the number and type of items of the scale.
Because of the differences in the total scoring and item selection to compose the EMIC-SS in the various settings, the interpretability of the scale scoring is essential. The transformation of the quantitative value is therefore necessary to convey a qualitative meaning to the scores. Since the score is a non-continuous variable, it would be appropriate to use the 25th and 75th percentiles of the 12-item scale score to define the cut-off values to establish the level of stigma. Further studies using the Brazilian cross-culturally adapted EMIC-SS are needed to set up the levels to identify a meaningful change in individual and group evaluations or to compare relevant sub-group of patients. Classifying stigma perception into low, moderate, or high levels can facilitate comparison between groups and populations and assessing the effect of stigma elimination interventions.
Possible limitations of the study should be mentioned regarding the sample and interview methodology. A heterogeneous and representative sample of the target population is recommended for instrument validation [49]. In a continental multicultural country such as Brazil, it is difficult to achieve this ideal in a single-center study. Nevertheless, we recruited participants from the five regions of the country. Although most of them were born in Rio de Janeiro, the State has a history of being attractive to migrations from other Brazilian States and countries. The metropolitan area of Rio de Janeiro concentrated most of the State population between 1900 and 1970. First, as the country’s capital then because of industrialization, it has always attracted a great diversity of interstate migrations [62], which renders the population heterogeneous. Therefore, we understand that there is little sampling bias in this exploratory study.
It is usually recommended that data collection procedures be homogeneous and free of bias. Although we had a mixture of interviewer-administrated and self-administrated forms, a small proportion of the RSS and BDI was self-administered. Having the interviewer apply the instruments was necessary since we received people with low education levels and functional illiterates. Our adaptation looked to include these types of participants, despite cognitive limitation due to the high level of illiteracy in the country IBGE and among people affected by leprosy [63]. Besides, the researchers who collected the data were experienced in this type of procedure. Another limitation is inherent in any instrument validation process. Measures intended to be indicators of particular constructs may not be retained [64, 65]. Any scale resulting from a validation study is imperfect since it presents several points that need improvement.
Conclusions
Stigma measurement is essential to understand the burden stigma and to assess the effectiveness of interventions. Quantitative and qualitative instruments are available, but the EMIC-SS serves as a generic tool to study leprosy-related perceived and self-stigma [51]. We studied the factor structure and the psychometric qualities of an instrument capable of evaluating perceived and experienced stigma in both one-dimensional and two-dimensional models. Our findings indicate that the one-dimensional model presents satisfactory evidence of factorial validity, internal consistency, convergent and known-groups validity, no floor or ceiling effects, and test-retest reliability to evaluate leprosy-related stigma in Brazil. We recommend that further exploration of the factorial structure of EMIC-SS in other samples and populations with stigmatizing conditions to strengthen the indications of the validated scale and to determine the optimal factor structure. We encourage researchers to continue to explore the factorial structure, validity, and reliability of EMIC-SS in different samples and contexts. These types of studies are rare in the field of leprosy, and evaluating a construct as complex as stigma with a single instrument is particularly challenging.
Acknowledgments
We are grateful to the patients at the Souza Araújo outpatient clinic who willingly accepted to participate in the study, and to our colleagues at Leprosy laboratory.
References
- 1. Hatzenbuehler ML, Phelan JC, Link BG. Stigma as a Fundamental Cause of Population Health Inequalities. Am J Public Health. 2013;103(5):813–21. pmid:23488505
- 2. Weiss MG, Auer C, Somma D, Abouihia A, Jawahar MS, Karim F, et al.Gender and tuberculosis: Cross-site analysis and implications of a multi-country study in Bangladesh, India, Malawi, and Colombia. WHO/TDR SEB Report Series, 2006, Number 4. p. 100. Available from https://www.who.int/tdr/publications/tdr-research-publications/gender-tb-multicountry-study/en/
- 3. Weiss MG, Ramakrishna J, Somma D. Health-related stigma: rethinking concepts and interventions. Psychol Health Med. 2006;11(3):277–87. pmid:17130065
- 4. Stangl AL, Earnshaw VA, Logie CH, van Brakel W, C Simbayi L, Barré I, et al. The Health Stigma and Discrimination Framework: a global, crosscutting framework to inform research, intervention development, and policy on health-related stigmas. BMC Med. 2019;17(1):31 pmid:30764826
- 5. Sales AM, Campos DP, Hacker MA, da Costa Nery JA, Düppre NC, Rangel E, et al. Progression of leprosy disability after discharge: is multidrug therapy enough? Trop Med Int Health. 2013;18(9):1145–1153 pmid:23937704
- 6. Nery JS, Ramond A, Pescarini JM, Alves A, Strina A, Ichihara MY, et al.Socioeconomic determinants of leprosy new case detection in the 100 Million Brazilian Cohort: a population-based linkage study. Lancet Glob Health. 2019;7(9):e1226–e1236. pmid:31331811
- 7.
Human Rights Council. Stigmatization as dehumanization: wrongful stereotyping and structural violence against women and children affected by leprosy—Report of the Special Rapporteur on the elimination of discrimination against persons affected by leprosy and their family members. Report to the Human Rights Council, United Nations General Assembly, 2019. A/HRC/41/47. GE.19-06395(E). 19pp.
- 8. Lusli M, Zweekhorst MBM, Miranda-Galarza B, Peters RMH, Cummings S, Seda FSSE, et al. Dealing with stigma: Experiences of people affected by disabilities and leprosy. Biomed Res Int. 2015; 1–09.
- 9. Femina LL, Soler ACP, Nardi SMT, Paschoal VDA. Lepra para hanseníase: a visão do portador sobre a mudança de terminologia. (Leprosy to Hanseniasis: the effect of the terminology Change According to the the persons affected) Hansenol. int. (Online). 2007; 32(1): 37–48.
- 10. Faria L, Santos LAC. A hanseníase e sua história no Brasil: a história de um “flagelo nacional”. (Hanseniasis and its history in Brazil: the history of a “national scourge) Hist. Cienc. Saude-Manguinhos. 2015, 22(4): 1491–95.
- 11. Sillo S., Lomax C, e Wildt G, Fonsea MDS, Galan NGDA, Prado RBR. A Temporal and sociocultural exploration of the stigma experiences of leprosy patients in Brazil. Lepr. Rev., 2016; 87, 378–395
- 12.
Casa Civil. Lei 9.010/1995. Subchefia para assuntos jurídicos. Presidency, Brazil, 1995. Available from: http://www.presidencia.gov.br/ccivil_03/Leis/L9010.htm
- 13.
Ministry of Health. Diretrizes para vigilância, atenção e eliminação da hanseníase como problema de saúde pública: manual técnico. Secretaria de Vigilância em Saúde, Mistério da Saúde, 2016, 1 ed. Brasília.
- 14. Weiss MG, Doongaji DR, Siddhartha S, Wypij D, Pathare S, Bhatawdekar M, et al. The Explanatory Model Interview Catalogue (EMIC). Contribution to cross-cultural research methods from a study of leprosy and mental health. British Journal of Psychiatry: the Journal of Mental Science. 1992; 160:819–30.
- 15. Weiss MG. Explanatory Model Interview Catalogue: Framework for Comparative Study of Illness Experience. Transcultural Psychiatry. 1997, 34: 235–63.
- 16. Rafael F, Houinato D, Nubukpo P, Dubreuil CM, Tran DS, Odermatt P, et al. Sociocultural and psychological features of perceived stigma reported by people with epilepsy in Benin. Epilepsia. 2010; 51(6):1061–8. pmid:20132293
- 17. Vlassoff C, Weiss M, Ovuga EBL, Eneanya C, Nwel PT, Babalola SS, et al. Gender and the stigma of onchocercal skin disease in Africa. Social Science and Med, 2000; 50:1353–1368.
- 18. Weiss MG. Prominence of symptoms and level of stigma among depressed patients in Calcutta. Journal of the Indian Medical Association, 2001; 99, 20–23. pmid:11480952
- 19. Garapati P, Pal B, Siddiqui NA, Bimal S, Das P, Murti K, et al. Knowledge, stigma, health seeking behaviour and its determinants among patients with post kalaazar dermal leishmaniasis, Bihar, India. PLoS ONE, 2018; 13(9): e0203407. pmid:30192805
- 20. Shi-Jie F, Hong-Mei G, Li W, Bin-Hong W, Yi-Ru F, Gang W, et al. Perceptions of stigma and its correlates among patients with major depressive disorder: A multicenter survey from China. Asia-Pacific Psychiatry. 2017;9:1–6. https://doi.org/10.1111/appy.12260
- 21. Rensen C, Bandyopadhyay S, Gopal PK, Van Brakel WH. Measuring leprosy-related stigma—a pilot study to validate a toolkit of instruments. Disabil Rehabil. 2010;33(9):711–9. pmid:20690861
- 22. Stienstra Y, van der Graaf WT, Asamoa K, van der Werf TS. Beliefs and attitudes toward Buruli ulcer in Ghana. Am. J. Trop. Med. Hyg. 2002; 67(2): 207–213. pmid:12389949
- 23. Brieger WR, Oshiname FO, & Ososanya OO. (1998). Stigma associated with onchocercal skin disease among those affected near the Ofiki and Oyan Rivers in western Nigeria. Social Science & Med, 47, 841–852.
- 24. Singh A, Mattoo SK, Grover S. Stigma and its correlates in patients with schizophrenia attending a general hospital psychiatric unit. Indian J Psychiatry. 2016; 58(3): 291–300 pmid:28066007
- 25. Stevelink SA, Van Brakel WH, & Augustine V. Stigma and social participation in Southern India: Differences and commonalities among persons affected by leprosy and persons living with HIV/AIDS. Psychol, Health & Med, 2011, 16:695–707.
- 26. Voorend CG, Van Brakel WH, Cross H, Augustine V, Ebenso B Report of the Stigma Research Workshop for the development of scientific consensus papers and field guidelines on health-related stigma, held in Amsterdam, the Netherlands from 11–14 October 2010. Lepr Rev. 2011; 82(2): 188–201 pmid:21888143
- 27. Weiss MG, Somma D, Obrist B. Training Manual for Cultural Epidemiology: Examples from a Study of Tuberculosis and Gender: Swiss Tropical Institute. Department of Public Health and Epidemiology; 2008.
- 28. Van Brakel WH. Measuring Leprosy Stigma—A Preliminary Review of the Leprosy Literature. Int J Lepr Other Mycobacterial Diseases. 2003;71(3):190–197
- 29. Adhikari B, Kaehler N, Marahatta SB, Ggyanwali K, Chapman RS. Risk factors of stigma related to leprosy—A systematic review. JMMIHS. 2013;1(2):3–11.
- 30. Van Brakel WH. Measuring health-related stigma—a literature review. Psychol Health Med. 2006;11(3):307–34. pmid:17130068
- 31. Terwee CB, Bot SD, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007; 60: 34–42. pmid:17161752
- 32. Chung EY, Lam G. Validation of two scales for measuring participation and perceived stigma in Chinese community-based rehabilitation programs. Health Qual Life Outcomes. 2018;16(1):105. pmid:29843809
- 33. World Health Organization. Global leprosy update, 2018: accelerating reduction of disease burden. Wkly Epidemiol Rec. 2019;94(36):445–56
- 34.
World Health Organization. Global leprosy strategy 2016–2020: accelerating towards a leprosy-free world– 2016 operational manual. World Health Organization, Regional Office for South-East Asia; 2016
- 35. Morgado FFR, Silveira EMKXD, Sales AM, Nascimento LPRD, Sarno EN, Nery JADC, et al. Cross-cultural adaptation of the EMIC Stigma Scale for people with leprosy in Brazil. Rev Saude Publica. 2017;4:51:80.
- 36. Prado GD, Prado RBR, Marciano LHSC, Nardi SMT, Cordeiro JA, Monteiro HL. WHO disability grade does not influence physical activity in Brazilian leprosy patients. Lepr Rev. 2011;82(3):270–8. pmid:22125935
- 37. Fife BL & Wright ER. The dimensionality of stigma: A comparison of its impact on the self of persons with HIV/AIDS and cancer. J Health Soc Behav, 2000; 41:50–67. pmid:10750322
- 38. Van Brakel WH, Anderson AM, Mutatkar RK, Bakirtzief Z, Nicholls PG, Raju MS, et al. The participation scale: measuring a key concept in public health. Disabil Rehabil. 2006;28(4):193–203 pmid:16467054
- 39. Mustillo SA, Hendrix KL, Schafer MH. Trajectories of Body Mass and Self-Concept in Black and White Girls: The Lingering Effects of Stigma. J Health Soc Behav, 2012, 53(1), 2–16. pmid:22382717
- 40.
DeVellis RF. Scale development: theory and applications. Newbury Park CA: Sage Publications; 2003.
- 41.
Nunnally JC. Psychometric theory. New York: McGraw Hill; 1967.
- 42.
Hutz CS. Adaptação brasileira da escala de autoestima de Rosenberg [monografia]. Porto Alegre: Universidade Federal do Rio Grande do Sul; 2000.
- 43. Hutz CS, Zanon C. Revisão da adaptação, validação e normatização da Escala de Autoestima de Rosenberg. Avaliação Psicológica. 2011;10(1),41–49.
- 44. Gorenstein C, Andrade L. Validation of a Portuguese version of Beck Depression Inventory and the State Trait Anxiety Inventory in Brazilian subjects. Braz J Med Biol Res. 1998; 29(4):453–457
- 45.
Jöreskog K, Sörbom D. LISREL 7: A guide to the program and applications. Chicago. SPSS Inc.; 1988.
- 46. Reise SP, Widaman KF, Pugh RH. Confirmatory factor analysis and item response theory: two approaches for exploring measurement invariance. Psychol Bull. 1993;114(3):552. pmid:8272470
- 47. Morgado FFR, Meireles JFF, Neves CM, Amaral ACS, Ferreira MEC. Scale development: ten main limitations and recommendations to improve future research practices. Psicol. Reflex. Crit. 2017; 30(3):1–20.
- 48.
Jöreskog K, Sörbom D. Prelis 2: User’s reference guide. Chicago: Scientific Software International; 1999.
- 49. Schreiber JB, Nora A, Stage FK, Barlow EA and King J. Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review. J Educ Res, 2006; 99:6, 323–338,
- 50. Schuck P. Assessing reproducibility for interval data in health-related quality of life questionnaires: which coefficient should be used? Qual Life Res. 2004;13(3):571–586. pmid:15130022
- 51.
ILEP. Guidelines to reduce stigma. How to assess healthrelated stigma The International Federation of Anti-Leprosy Associations (ILEP) and the Netherlands Leprosy Relief (NLR), London / Amsterdam, 2011.48p.
- 52. Dadun, Peters RMH, Van Brakel WH, Lusli M, Damayanti R, Bunders JFGet al. Cultural validation of a new instrument to measure leprosy-related stigma: the SARI Stigma Scale. Lepr Rev. 2017;88:23–42. pmid:30188086
- 53. Lustosa AA, Nogueira LT, Pedrosa JIS, Teles JBM, Campelo V. The impact of leprosy on health-related quality of life. Rev Soc Bras Med Trop. 2011;44(5):621–6. pmid:22031079
- 54. Groot R, Van Brakel WH, De Vries HJC. Social implications of leprosy in the Netherlands—stigma among ex-leprosy patients in a non-endemic country. Lepr Rev. 2011; 82:168–177 pmid:21888141
- 55. Decker KM, Thurston IB, Kamody RC. The mediating role of internalized weight stigma on weight perception and depression among emerging adults: Exploring moderation by weight and race. Body Image. 2018; 27:202–210 pmid:30384173
- 56. Goodman E, Hinden BR, Khandelwal S. Accuracy of Teen and Parental Reports of Obesity and Body Mass Index. Pediat 2000, 106 (1) 52–58.
- 57. Rao PSS, Raju MS, Barkataki A, Nanda NK, Kumar S. Extent and correlates of leprosy stigma in Rural India. Indian J Lepr. 2008; 80(2):167–74. pmid:19425512
- 58. Van Brakel WH, Sihombing B, Djarir H, Beise K, Kusumawardhani L, Yulihane R, et al. Disability in people affected by leprosy: the role of impairment, activity, social participation, stigma and discrimination. Glob Heart Action. 2012;5:1–11
- 59. Adhikari B, Kaehler N, Chapman RS, Raut S, Roche P. Factors Affecting Perceived Stigma in Leprosy Affected Persons in Western Nepal. PLOS Neg Trop Dis. 2014;8(6): e2940
- 60. Sermrittirong S. Stigma and stigma interventions related to leprosy and tuberculosis in Thailand. Ridderprint, Ridderkerk, The Netherlands, 2014. 216 p.
- 61. Cross H, Choudhary R. STEP: an intervention to address the issue of stigma related to leprosy in Southern Nepal. Lepr Rev. 2005;76(4):316–24 pmid:16411511
- 62. Souza J de, & Frutuozo J.V de P. Rio de Janeiro: considerações sobre os processos de expansão urbana e interiorização do crescimento (1980–2010). Urbe. Rev Bras Gestão Urbana, 2018; 10(1), 124–139.
- 63.
IBGE. Educação 2017. Pesquisa Nacional por Amostra de Domicílios Contínua. Diretoria de Pesquisas, Coordenação de Trabalho e Rendimento.Brasil, 2018. p.12.
- 64. Costello AB, Jason O. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res & Eval. 2005;10(7):1–9.
- 65. Anderson JC, Gerbing DW. Predicting the Performance of Measures in a Confirmatory Factor Analysis With a Pretest Assessment of Their Substantive Validities. J Appl Psy. 1991;76(5):732–740.