Next Article in Journal
Treasure Hunt—Roman Inquisition and Magical Practices Ad Inveniendos Thesauros in Southern Tuscany
Previous Article in Journal
Antisemitism in the Muslim Intellectual Discourse in South Asia
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Educational Environments with Cultural and Religious Diversity: Psychometric Analysis of the Cyberbullying Scale

by
María Tomé-Fernández
1,
José Manuel Ortiz-Marcos
2,* and
Eva María Olmedo-Moreno
2
1
Faculty of Education and Sports Sciences, Department of Research Methods and Diagnosis in Education, University of Granada, 52071 Melilla, Spain
2
Faculty of Education Sciences, Department of Research Methods and Diagnosis in Education, University of Granada, 18071 Granada, Spain
*
Author to whom correspondence should be addressed.
Submission received: 24 June 2019 / Revised: 15 July 2019 / Accepted: 18 July 2019 / Published: 21 July 2019

Abstract

:
The objective of this research is to adapt and validate a useful instrument to diagnose cyberbullying, provoked by intolerance towards cultural and religious diversity, identifying the profile of the aggressor and the victim. The study was carried out using the Delphi technique, exploratory factor analysis (EFA), and confirmatory factor analysis (CFA). The selected sample was composed of 1478 adolescents, all students from Compulsory Secondary Education of Spain. The instrument items were extracted from relevant scales on the topic. The initial questionnaire was composed of 52 items and three underlying constructs. After validation with EFA (n = 723), the structure was checked, and the model was later corroborated with CFA (n = 755) through structural equations (RMSEA = 0.05, CFI = 0.826, TLI = 0.805). The reliability and internal consistency of the instrument were also tested, with values for all dimensions being higher than 0.8. It is concluded that this new questionnaire has 38 items and three dimensions. It has an acceptable validity and reliability, and can be used to diagnose cyberbullying caused by the non-acceptance of cultural and religious diversity in Compulsory Secondary Education students.

1. Introduction

In today’s society, the misuse of information and communications technology and its notoriety amongst youth has given rise to a considerable increase in cyberbullying, or electronic bullying, among these students (Olweus and Limber 2018). This educational problem is characterized, according to Estévez et al. (2018), as a type of abuse exercised through electronic means, with a negative intention of the aggressor towards the victim, durability over time, and a power imbalance between both parties given the greater technological control of the aggressors.
With regard to this situation, different research has shown the existing correlation that exists between the dependency behavior of the victims and the antisocial behavior of the aggressors who perform the cyberbullying (Muñoz et al. 2016). Cyberbullies do not harass casually; their violence is intentional, aimed at strengthening their social position or marginalizing opponents of a specific group (Navarro et al. 2015), leading to violent situations with bigger audiences and lasting longer than traditional bullying, with the guarantee of anonymity (Cavezza and McEwan 2014; Matos et al. 2016).
Cyberbullying is currently present in Spanish schools; proof of this are the interventions against this type of violence that have taken place in the country (Della et al. 2015; Garaigordobil and Martínez-Valderrey 2015), where the influence of relevant preventive and training programs in the world is reflected (Cross et al. 2015). The main reason for this phenomenon is caused by technological revolution. Currently, communication through digital devices and social networks establishes a world of fluid and almost permanent exchange, in which social interactions easily shift from personal to virtual relationships, with a series of advantages in terms of time and spatial availability that do not exist with traditional relationships (Romera et al. 2016). These aspects are beneficial for the interactions of Spanish students, even though, as Fernández-Montalvo et al. (2015) indicate, they are not exempt from certain risks, such as cyberbullying.
With the incorporation of technology into the educational world, there has been a quick surge of studies related to this topic (Zych et al. 2016). Nevertheless, the criteria used to define the phenomenon and assessment strategies are still being debated, primarily in terms of aspects related to the harassment suffered by students from other countries (Allison and Bussey 2017). With regard to this educational problem, Pottie et al. (2015) found that the consequences of discrimination are more negative for members of disadvantaged groups. Additionally, the majority of harassment experiences are inflicted on ethnic minorities (Rhee et al. 2017) or those people with a different religion to the majority (Francis and McKenna 2018).
According to Vitoroulis and Georgiades (2017), students from other cultures, races, or religions have problems adjusting to intolerant contexts. Sometimes in these situations, these students suffer from intimidation and verbal and physical aggression by other racist students. These problems increase in the Compulsory Secondary Education stage (Fernández-Lasarte et al. 2019).
Due to these important risk factors for electronic harassment (Hinduja and Patchin 2017; Patchin and Hinduja 2015), it is fundamental that an instrument with solid psychometric properties in diagnosing the problem and that is able to intervene efficiently is developed and validated, above all at a time when there exist few investigations linking cyberbullying with intercultural students (Rodríguez and Guzmán 2019). This is proposed with the purpose of improving the intercultural performance of the current teacher, since, as indicated by Buendía et al. (2015), faculty in Spain are limited to responding to educational policies based on compensatory measures focused on diversity, obviating important aspects of intercultural teaching.
The majority of existing questionnaires for diagnosing cyberbullying ignore religious, cultural, or ethnic variables (Mérida et al. 2015). This is seen in instruments like the European Bullying Intervention Project Questionnaire (Elipe et al. 2017), the European Cyberbullying Intervention Project Questionnaire (Del Rey et al. 2015; Romera et al. 2016), and the Cyberbullying questionnaire (Hall 2016). For this reason, the instrument used in this study establishes three dimensions that analyze cyberbullying, prompted by intolerance to cultural and religious diversity of the environment in which it occurs.
The first dimension is called intercultural cyberbullying and relates to what Bueno and Domingo (2016) have established as coercion of the dominant culture over minorities. According to Llorent et al. (2016), social networks reflect increasingly dominant attitudes, encouraged by the homogenization of society, which devalue immigrants and people from other minority cultures and religions.
The second dimension is called digital racist threats and concerns violent situations provoked by fear of the unknown or that which is different, causing behavioral problems such as insults or harassment through digital means (Rey et al. 2018).
Finally, the third dimension is the designated usurpation of identity to people of different ethnicities, cultures, and religions. This dimension connects identity impersonation on social networks to the denigration, exclusion, and exposure of the victim’s privacy (Yudes-Gómez et al. 2018). These types of occurrences cause, especially among youth from minority cultures and religions, a socio-emotional imbalance, where personality, self-esteem, and social habits are affected when confronted with a digital environment of rejection (Zych et al. 2015).
The objective of this research is to adapt and validate a useful instrument to diagnose cyberbullying promoted by intolerance towards cultural and religious diversity, identifying the profiles of the aggressor and the victim.

2. Method

2.1. Population and Sample

The sample of 1478 adolescents was selected through non-probability sampling (convenience) in which the scale was given to students from centers who wished to participate. The selection was carried out in two stages, differentiated by the validity analysis to be performed for each.
The objective of the first stage was to find the structure and dimensions in which the questionnaire items are organized according to validation using exploratory factor analysis (EFA). In this stage, 723 students were selected. The age of the students was between 12 and 16 years (M = 13.97 years; SD = 1.461), of which 362 (50.1%) were male and 361 (49.9%) were female. In addition, 493 (68.2%) students were White, 148 (19.8%) Latinos, 29 (4%) African, and 10 (1.4%) Asian, and 43 (6.6%) did not know how to answer this question. There were also students from minority ethnic groups: 20 (2.8%) were Gypsy, 3 (0.41%) Celtic, 7 (1%) Armenian, and 98 (14%) Mongolian. Finally, regarding the religion variable, 502 (69.4%) students were Christian, 39 (5.4%) Jewish, 25 (3.5%) Islamic, 7 (1%) Taoist, and 2 (0.3%) Buddhists. The remaining 148 (20.4%) students did not practice any religion.
During the second stage, the purpose of the validity analysis was of confirmatory and had the aim of finding a model that explained the structure of the instrument regarding the applied sample. For this, 755 students were selected. The age of the students was between 12 and 16 years (M = 13.96; SD = 1.452), of which 376 (49.8%) were male and 379 (50.2%) were female. As in the previous stage of the study, the sample was characterized by diversity in race, ethnicity, and religion. In this case, 506 (67%) were White students, 155 (21%) Latinos, 37 (5%) African, and 11 (1.5%) Asian, and 46 (5.5%) did not know how to answer this question. The students from minority ethnic groups were 27 (3.6%) Gypsy, 4 (0.53%) Celtic, 10 (1.32%) Armenian, and 99 (13.1%) Mongolian. Lastly, regarding religion, the sample was composed of 518 (68.60%) Christian, 40 (5.3%) Jewish, 35 (4.63%) Islamic, 9 (1.2%) Taoist, and 3 (0.4%) Buddhists. The remaining 150 (19.9%) students did not practice any religion.
All educational centers that participated in the study belong to cities of Andalucía, Ceuta, and Melilla, characterized by having high percentages of immigrants (INE 2019) that are reflected in the cultural and religious diversity of the classrooms (Pena-Diaz 2019). Table 1 shows the number of participants in each period, in the different provinces.

2.2. Instrument

The Cyberbullying Scale for Students with Cultural and Religious Diversity (CSCRD) was an adaptation of the Spanish version of the European Cyberbullying Intervention Project Questionnaire (ECIPQ) (Del Rey et al. 2015; Romera et al. 2016), of the European Bullying Intervention Project Questionnaire (Elipe et al. 2017), and of the Cyberbullying questionnaire of Hall (2016). The adapted instrument was designed for adolescents between 12 and 16 years of age, and is structured with 52 items (Appendix A) that evaluate three dimensions (intercultural cyberbullying, digital racist threats, and usurpation of identity to people of different ethnicities, cultures, and religions). Following theoretical indications (Humphrey-Murto et al. 2017), the creation of the questionnaire was carried out in three phases: preliminary, exploratory, and final.
In the preliminary phase, the coordinator for the expert group presented the problem and selected the work team. As established in other research using the Delphi method (Rodríguez-Gómez et al. 2018), two human groups were formed for the validation process for content and appearance of the adapted instrument. The coordinator group was composed of participating research members characterized as having a good understanding of the method, being academic researchers of the study topic, and having strong intercommunication abilities (Varho et al. 2016). The expert group was composed of 22 professionals from different Spanish universities, of which 12 were researchers related to intercultural education, 5 were associated with the use of technology in the educational field, and 5 had lines of work in bullying and school violence. The validity of the initial adapted instrument was obtained by following this method, with it being one of the most effective in social science research (Rikkonen et al. 2019). This is mainly due to the democratization of the process to construct meaning among experts; by including participants with knowledge of and experience with the study material; and by the anonymity of the expert responses, which avoids, in a group debate process, having people who might manipulate the opinions of others (Martínez-García et al. 2019).
In the exploratory phase, the final interpretation of the adapted instrument was developed. To do this, three rounds of analysis were executed through discussions between members of the coordinator group, taking into account the adjustments and corrections presented by the expert group. The percentage of concordance of the coordinator group in the first round of discussion oscillated between K ≥ 64 and K ≥ 80; in the second, between K ≥ 76 and K ≥ 89; and in the third, between K ≥ 87 and K ≥ 92.
The expert group was asked to rate, from 1–4 on a Likert scale (nothing–a lot) and in different phases, the adequacy of the instrument’s information, the efficiency of the questionnaire measurement, and the comprehension and writing of the items (Mérida et al. 2015). This was done by email, in which the initial instrument was sent, together with the research plan and a record sheet, to collect demographic data and the quantitative and qualitative responses (López 2018).
The items whose percentage of concordance between the judges of the coordinator group was K ≥ 70 and whose scores given by the expert group on the Likert scale were mostly less than 3 were modified, eliminated, or regrouped. After applying the method, various items were not significantly modified, 6 were completely eliminated, and some were grouped into 4, with 12 initial questions. This meant that the questionnaire was finally composed of 38 questions divided into three dimensions.
Lastly, in the final phase, the definitive instrument was obtained. For this, quantitative data was analyzed, acquired through the application of the scale to the participants in two periods. In the first period, EFA was performed and in the second period, confirmatory factor analysis (CFA) was conducted. Both analyses kept the 38 items of the scale grouped in three dimensions: intercultural cyberbullying, digital racist threats, and usurpation of identity to people of different ethnicities, cultures, and religions (see Appendix B).

2.3. Data Collection and Analysis Process

To carry out the research, the educational schools’ directors were informed of the objectives, and were asked for their corresponding permission to participate. In addition, when working with minors, authorization was requested from the students’ legal guardians, insisting on the voluntary nature of their participation and that their answers would be confidential. The application of the research instrument was done in a collective manner in the participants’ classrooms in the presence of the teachers, in sessions lasting between fifteen and twenty minutes. The analysis of the data in the EFA was made using the latest version of SPSS statistical software and for the analysis of the data (CFA), the AMOS statistical program was used.
Throughout the scale application process, the ethical guidelines established in the Declaration of Helsinki and the protocol approved by the Ethics Committee of the University of Granada were followed.

3. Results

3.1. Exploratory Factor Analysis

Before this analysis, the data was subjected to the Kaiser, Meyer and Olkin (KMO) test and the Bartlett’s test of sphericity, to confirm whether the values allowed the use of EFA as a technique to interpret the information of the matrix. The KMO index, the Bartlett chi-square approximation, and the level of significance obtained accepted the null hypothesis (H0) (Table 2), so the model was considered optimum for applying EFA (Glick et al. 2018). In addition, Duarte et al. (2019) indicated that a KMO value close to 1 reveals that the correlation pattern is sufficiently compact to produce different and reliable factors.
Then, the scale’s reliability was calculated using a sample of N = 723. The calculation was done through Cronbach’s alpha coefficient, whose global value is α = 0.966, showing that the instrument has an acceptable internal consistency and suitable factor distribution for the measurement (Asare and Manoj 2014).
Furthermore, it was observed that the analysis of commonalities after extraction resulted in elements above 0.40 (Table 3), which indicates that all items could be included in the EFA, and the number of participants used for this analysis was sufficient to perform it correctly (Love et al. 2019).
According to Bernstein and Calamia (2019), EFA is an excellent technique for exploring the set of latent variables or common factors that explain the responses to items on a questionnaire. For this, in the study, the Varimax rotation method, rotated factor loadings, and the variance percentage were calculated.
The rotated factor matrix indices regrouped the 38 items into three factors that represent 69.42% of the total explained variance, with saturation charges that oscillate between 0.312 and 0.904 (Table 3), distributed as follows: the dimension named intercultural cyberbullying was composed of 19 variables; the dimension named digital racist threats was composed of 12 variables; and the last dimension referred to as usurpation of identity to people of different ethnicities, cultures, and religions was composed of 7 items. Additionally, the explained variance of the first factor was 34.28%, of the second factor was 54.17%, and of the third factor was 65.34%.

3.2. Confirmatory Factor Analysis

Given the existence of a questionnaire model and with evidence of the discriminatory power of its items, the CFA was performed to corroborate the suitability of the indicators to evaluate the latent variables (Anderson-Butcher et al. 2016). This statistical test was done with N = 755 and analyzed with AMOS software, version 24. The CFA is presented in path diagrams, where the circles represent latent variables and squares represent observed variables. The single-headed arrows are used to imply a direction of assumed influence, and two-headed arrows represent the covariance between the three latent variables (Figure 1).
To carry out the CFA, CMIN was observed (Cb, minimum value of the discrepancy) with a χ2 distribution (Table 4).
The recommendations of Bentler (2007) and Byrne (2010) were followed, which indicate that since the chi-square goodness of fit test (χ2) tends to be insufficient for the size of the sample used, it is recommended that the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the Root Mean Square Error of Approximation (RMSEA) are observed, in order to know the degree of adjustment between the covariance matrix of the observed data and the covariance matrix predicted by the model, established by the goodness of fit index.
The CFI, TLI, and RMSEA obtained show a good internal consistency of the latent factors (Tsaur and Tu 2019). In addition, according to Lecerf and Canivez (2018), the CFI and TLI are considered acceptable adjustment the closer to 1 that they are, which is a characteristic that is fulfilled in the results of this research, whose CFI = 0.83 and TLI = 0.81 (Table 5).
In terms of the RMSEA value, excellent adjustment is considered below 0.06 (Maydeu-Olivares et al. 2017; Jöreskog and Sörbom 1984), which occurred in the study (Table 6).
Taking these results into account, it can be deduced that the proposed model exhibits reasonable contiguity of the data, and ratifies the hypothesis of the multidimensionality of the construct.
The latent variables for the CSCRD were presented in 12, 7, and 19 items (Figure 1). In the first factor, the items are adequately explained by the hypothesized construct, showing a minimum saturation of 0.80 and a maximum of 1.16. The second factor presents saturations that oscillate between 0.95 and 1.34. Lastly, the third factor shows a minimum saturation of 1 and maximum of 1.33, which indicates that its items show a satisfactory correlation with the total of the test (Runions et al. 2017).
Once the CFA was completed and having confirmed the instrument structure of 38 items, the internal consistency test (Cronbach’s alpha) was redone, obtaining a reliability factor of α = 0.90 for the first dimension, α = 0.82 for the second dimension, and α = 0.76 for the third dimension, which indicates a good scale reliability (Appleton et al. 2016).

4. Discussion and Conclusions

The main objective of this research was the adaptation and validation of a measuring instrument for cyberbullying provoked by the intolerance of cultural and religious diversity in educational environments. To do this, the psychometric properties of the questionnaire were analyzed in three phases.
In the initial phase, the Delphi method was used, which analyzed the validity of the scale content. In this method, 22 experts used existing literature (Del Rey et al. 2015) to consolidate and establish the denomination of the dimensions. It should be mentioned that the reference studies focus on analyzing bullying, or cyber-bulling; nevertheless, they do not delve into the intercultural characteristics of the student to find out the possible xenophobic causes of this type of violence. Therefore, an innovative tool has been created with high scientific application in the globalized world in which we live, in which it is not uncommon to see physical or verbal violence in youth of different cultures, ethnicities, or religions (Domínguez et al. 2017). Offering a reliable and empirically proven instrument is of great importance at a time when digital media is full of violence hidden behind masked identities and aliases, making diagnosis and educational intervention focused on cyber-aggressors difficult (Palladino et al. 2015).
In the second analysis phase, the reliability of the scale was obtained with EFA, and the items that showed positive psychometric behavior were visible. In respect to this, it was found that after the extraction of the commonalities, all the elements were suitable to keep, which conserved the 38 items established by the experts after applying the Delphi method.
In the third phase of the analysis, after performing the CFA, the model was confirmed by obtaining acceptable indices (CFI and TLI) (Morin et al. 2015), improving even those obtained in the original scales from which the instrument was adapted (Del Rey et al. 2015; Elipe et al. 2017; Hall 2016; Romera et al. 2016). In addition, the construct validity analysis shows corroboration with the scale used in most proportions for adaptation (Del Rey et al. 2015), also obtaining three dimensions: the first latent variable comprises the items 26, 32, 31, 29, 27, 28, 30, 33, 38, 34, 36, 25, 24, 21, 20, 23, 22, 35, and 37 of the questionnaire; the second latent variable is made up of items 3, 13, 10, 12, 4, 2, 14, 1, 11, 8, 7, and 9 of the questionnaire; and the third latent variable includes the items 16, 18, 17, 15, 6, 19, and 5 of the questionnaire.
The decision to develop new instruments should be based on a careful consideration of the advantages and disadvantages of those already existing (Van et al. 2018). In this sense, the results provide not only empirical evidence that shows advantages over the referenced tools, but also present an instrument that allows the prevention and intervention of cyberbullying to be favored, taking into account the racist and xenophobic attitudes of the aggressors and the victims that suffer (Thomas et al. 2019).
In light of this situation, educational centers that are far from eliminating the inappropriate use of technology should propose preventive measures (Selwyn and Bulfin 2015), in order to mitigate hate towards those who are different (Giménez-Gualdo et al. 2015; Rodríguez-Gómez et al. 2018), making this questionnaire (see Appendix A) a starting point to do this, if used to diagnose the situation. This could even improve teaching strategies (Bevilacqua et al. 2017) in contexts where students have misguided value judgments of diverse cultures and religions (Ramos-Soler et al. 2018), and in some cases, even religious extremism prevails (Prud’homme 2019), which makes peaceful coexistence difficult, which can only be countered with education on values (Niemi et al. 2019).

Author Contributions

J.M.O.-M., M.T.-F. and E.M.O.-M. conceived the hypothesis of this study. J.M.O.-M. participated in data collection. J.M.O.-M. and E.M.O.-M. analyzed the data. All authors contributed to data interpretation of the statistical analysis and wrote the paper with significant input. All authors read and approved the final manuscript.

Funding

Part of this work has been funded by the Research Project Competitive “Values for intercultural coexistence in the students of the Autonomous City of Melilla. An intervention proposal”. Main Research: Tomé-Fernández, María.

Conflicts of Interest

The authors declare no conflicts of interest.

Appendix A

  • Other persons say bad word or insulted me for having different color skin using email, WhatsApp or another social network.
  • Someone say bad word or insulted me for being of a different ethnicity or religion using email, WhatsApp or another social network.
  • Racial comments have been made about me on social networks by dint of my race.
  • Racial comments have been made about me on social networks by dint of my ethnicity.
  • Racial comments have been made about me on social networks by dint of my religion.
  • Lies have been told to other classmates about my race, ethnicity or religion using the Internet, WhatsApp or social networks.
  • Lies have been told to other classmates about me for being different.
  • I have been threatened through messages on Messenger, WhatsApp or other social networks because of my religious or ethnic traditions.
  • Someone has hacked into my social network account and taken information to ignite hatred against my racial group.
  • Someone has hacked into my social network account and taken information to ignite hatred against my ethnic group.
  • Someone has hacked into my social network account and taken information to ignite hatred against my religion group.
  • Someone has hacked into my account and pretended to be me on Facebook or Twitter to ridicule my religious or ethical traditions.
  • A false account has been created by students pretending to be me in order to spark fear about my country’s customs or my ethnicity or religion.
  • Someone has posted personal information about my family online to make fun of our traditions and customs.
  • Someone has posted videos or photos online about my group’s religious or ethnic traditions to humiliate me.
  • Someone has posted videos or photos online about my race to humiliate me.
  • Fake photos have been posted online to ridicule my group’s religious or ethnic activity.
  • Fake photos have been posted online to ridicule my race.
  • I have been excluded or ignored on social networks or chat because of being of another race or belonging to a religion or ethnicity that is different from my classmates’.
  • I have been recorded getting beaten up and the video was posted online because I am different from them (color of skin hair, clothing, traditions or religion).
  • I have been recorded getting beaten up and the video was posted online because I am different race and ethnic.
  • Someone has called me on my cell and imitated my speech to make fun of my language.
  • I have been sent threatening WhatsApp audios telling me that my race, religion or ethnicity should be exterminated.
  • They have mocked me for fun for dressing or wearing different clothes than the rest of my classmates.
  • I have been mocked for fun for wearing veil on my head.
  • I have told someone bad words or insulted other classmates for having different colored skin using email, WhatsApp or another social network.
  • I have told someone bad words or insulted other classmates for being of a different ethnicity or religion using email, WhatsApp or another social network.
  • I have made racist comments on social networks about other races, ethnicities or religions.
  • I have told lies to classmates about other races, ethnicities or religions using the Internet, WhatsApp or social networks.
  • I have threatened other religious or ethnic traditions using Messenger, WhatsApp or other social networks.
  • I have hacked a social network account and taken information to ignite hatred against another racial group.
  • I have hacked a social network account and taken information to ignite hatred against another ethnic group.
  • I have hacked a social network account and taken information to ignite hatred against another religion group.
  • I have hacked an account and posed as someone else on Facebook or Twitter to ridicule religious or ethnic traditions that are different from mine.
  • I have hacked an account and posed as someone else on Facebook or Twitter to ridicule cultural traditions that are different from mine.
  • Someone has posed as me on a forum or social network, insulting and threatening others to create fear towards people of my race, ethnicity or religion.
  • I have created a false account posing as someone else in order to promote fear about customs of other countries or different ethnicities or religions.
  • I have posted personal information about the families of others online to make fun of their traditions and customs.
  • I have posted videos or photos online about religious or ethnic traditions of other groups to humiliate them.
  • I have posted videos or photos online about the race of others to humiliate them and make them feel bad.
  • I have posted fake photos online to ridicule the religious or ethnic activities of another group.
  • I have posted faked photos online to ridicule the race of another group.
  • I have posted faked photos online to ridicule those races whose skin colour differs from mine.
  • I have excluded or ignored others on social networks or chat for being from a different race, religion or ethnicity from the rest of my classmates.
  • I have beaten up classmates, recorded and posted the videos online because they are different than me (color of skin, hair, clothing, tradition, religion).
  • I have called by cell and imitated the speech of my classmate to make fun of their language.
  • I have sent threatening WhatsApp audios saying that the race, religion or ethnicity of other classmates should be extinguished.
  • I have sent threatening WhatsApp audios saying that the race, religion or ethnicity of other classmates are inferior.
  • I have made fun of other classmates for dressing or wearing different clothing from the rest of my classmates.
  • I have put myself through another on a forum or social network, insulting and threatening to create fear towards people of other races.
  • I have pretended to be someone else on a forum or social network, insulting and threatening to spark fear towards people of other ethnicities.
  • I have pretended to be someone else on a forum or social network, insulting and threatening to spark fear towards people of other religions.

Appendix B

Cyberbullying Scale for Students with Cultural and Religious Diversity (CSCRD)
Sex: ________________ Age: _____________ City: _____________________________
Institution: ________________________________________ Course: ___________________
Nationality: _______________________________________
Circle your answer:
Religion: Judaism—Christianity—Islamic—Buddhism—Taoism—I don’t belong to any
Ethnicity: Gypsy—Celtic—Armenian—Mongolian—Don’t know/no answer
Race: Asian— African—Latinos—White—Don’t know/no answer
Mark an X in the following boxes where:
Level of cyber victim or cyberbullying in adolescents
1: None    2: Little    3: Sufficient    4: Quite a bit    5: A lot
Thank you for your cooperation
Cyber victim12345
1. Someone say bad word or insulted me for having different color skin using email, WhatsApp or another social network.
2. Someone say bad word or insulted me for being of a different ethnicity or religion using email, WhatsApp or another social network.
3. Racial comments have been made about me on social networks for my race, ethnicity or religion.
4. Lies have been told to other classmates about my race, ethnicity or religion using the Internet, WhatsApp or social networks.
5. I have been threatened through messages on Messenger, WhatsApp or other social networks because of my religious or ethnic traditions.
6. Someone has hacked into my social network account and taken information to create hate against my racial, ethnic or religious group.
7. Someone has hacked into my account and pretended to be me on Facebook or Twitter to ridicule my religious or ethical traditions.
8. A false account has been created by someone pretending to be me in order to promote fear about my country’s customs or my ethnicity or religion.
9. Someone has posted personal information about my family online to make fun of our traditions and customs.
10. Someone has posted videos or photos online about my group’s religious or ethnic traditions to humiliate me.
11. Someone has posted videos or photos online about my race to humiliate me.
12. Fake photos have been posted online to ridicule my group’s religious or ethnic activity.
13. Fake photos have been posted online to ridicule my race.
14. I have been excluded or ignored on social networks or chat because of being of another race or belonging to a religion or ethnicity that is different from my classmates’.
15. I have been recorded getting beaten up and the video was posted online because I am different from them (color of skin hair, clothing, traditions or religion).
16. Someone has called me on my cell and imitated my speech to make fun of my language.
17. I have been sent threatening WhatsApp audios telling me that my race, religion or ethnicity should be exterminated.
18. They have mocked me for fun for dressing or wearing different clothes than the rest of my classmates.
19. Someone has posed as me on a forum or social network, insulting and threatening others to create fear towards people of my race, ethnicity or religion.
Cyber bully12345
20. I have told someone bad words or insulted other classmates for having different colored skin using email, WhatsApp or another social network.
21. I have told someone bad words or insulted other classmates for being of a different ethnicity or religion using email, WhatsApp or another social network.
22. I have made racist comments on social networks about other races, ethnicities or religions.
23. I have told lies to classmates about other races, ethnicities or religions using the Internet, WhatsApp or social networks.
24. I have threatened other religious or ethnic traditions using Messenger, WhatsApp or other social networks.
25. I have hacked a social network account and taken information to create hate against other racial, ethnic or religious groups.
26. I have hacked an account and posed as someone else on Facebook or Twitter to ridicule religious or ethnic traditions that are different from mine.
27. I have created a false account posing as someone else in order to promote fear about customs of other countries or different ethnicities or religions.
28. I have posted personal information about the families of other classmates online to make fun of their traditions and customs.
29. I have posted videos or photos online about religious or ethnic traditions of other groups to humiliate them.
30. I have posted videos or photos online about the race of others to humiliate them.
31. I have posted fake photos online to ridicule the religious or ethnic activities of another group.
32. I have posted faked photos online to ridicule the race of another group.
33. I have excluded or ignored others on social networks or chat for being from a different race, religion or ethnicity from the rest of my classmates.
34. I have beaten up classmates, recorded and posted the videos online because they are different than me (color of skin, hair, clothing, tradition, religion).
35. I have called by cell and imitated the speech of my classmate to make fun of their language.
36. I have sent threatening WhatsApp audios saying that the race, religion or ethnicity of other classmates should be extinguished.
37. I have made fun of other classmates for dressing or wearing different clothing from the rest of my classmates.
38. I have put myself through another on a forum or social network, insulting and threatening to create fear towards people of other races, ethnicities or religions.

References

  1. Allison, Kimberley, and Kay Bussey. 2017. Individual and collective moral influences on intervention in cyberbullying. Computers in Human Behavior 74: 7–15. [Google Scholar] [CrossRef]
  2. Anderson-Butcher, Dawn, Anthony J. Amorose, Leeann M. Lower, Allison Riley, Allison Gibson, and Donna Ruch. 2016. The case for the perceived social competence scale II. Research on Social Work Practice 26: 419–28. [Google Scholar] [CrossRef]
  3. Appleton, Paul R., Nikos Ntoumanis, Eleanor Quested, Carme Viladrich, and Joan L. Duda. 2016. Initial validation of the coach-created Empowering and Disempowering Motivational Climate Questionnaire (EDMCQ-C). Psychology of Sport and Exercise 22: 53–65. [Google Scholar] [CrossRef] [Green Version]
  4. Asare, Matthew, and Sarma Manoj. 2014. Establishing validity and reliability of a health belief model and acculturation scale for measuring safe-sex and sexual communication behaviors among African immigrants for protecting against HIV/AIDS. Journal of Immigrant & Refugee Studies 12: 191–209. [Google Scholar] [CrossRef]
  5. Bentler, Peter M. 2007. Can scientifically useful hypotheses be tested with correlations? American Psychologist 62: 772–82. [Google Scholar] [CrossRef] [PubMed]
  6. Bernstein, John P. K., and Matthew Calamia. 2019. Dimensions of driving-related emotions and behaviors: An exploratory factor analysis of common self-report measures. Accident Analysis and Prevention 124: 85–91. [Google Scholar] [CrossRef] [PubMed]
  7. Bevilacqua, Leonardo, Nichola Shackleton, Daniel Hale, Elizabeth Allen, Lyndal Bond, Deborah Christie, Diana Elbourne, Natasha Fitzgerald-Yau, Adam Fletcher, Rebecca Jones, and et al. 2017. The role of family and school-level factors in bullying and cyberbullying: A cross-sectional study. BMC Pediatrics 17: 1–10. [Google Scholar] [CrossRef]
  8. Buendía, Leonor, Jorge Expósito, Eva Aguadez, and Christian Sánchez. 2015. Analysis of Coexistence in Multicultural Secondary Education Classrooms. Revista de Investigación Educativa 33: 303–19. [Google Scholar] [CrossRef]
  9. Bueno, Xiana, and Andreu Domingo. 2016. The management of iterculturality in times of crisis: The discourse of technicians in local administration in Catalonia. Migraciones 39: 39–65. [Google Scholar] [CrossRef]
  10. Byrne, Barbara M. 2010. Structural Equation Modeling with AMOS. Basic Concepts, Applications and Programming. Mahwah: Lawrence Erlbaum Associates. [Google Scholar]
  11. Cavezza, Cristina, and Troy E. McEwan. 2014. Cyberstalking versus off-line stalking in a forensic simple. Psychology, Crime and Law 20: 955–70. [Google Scholar] [CrossRef]
  12. Cross, Donna, Amy Barnes, Alana Papageorgiou, Kate Hadwen, Lydia Hearn, and Leanne Lester. 2015. A social ecological framework for understanding and reducing cyberbullying behaviors. Aggression and Violent Behavior 23: 109–17. [Google Scholar] [CrossRef]
  13. Del Rey, Rosario, José A. Casas, Rosario Ortega-Ruiz, Anja Schultze-Krumbholz, Herbert Scheithauer, Peter Smith, Fran Thompson, Vassilis Barkoukis, Haralambos Tsorbatzoudis, Antonella Brighi, and et al. 2015. Structural validation and cross-cultural robustness of the European Cyberbullying Intervention Project Questionnaire. Computers in Human Behavior 50: 141–47. [Google Scholar] [CrossRef]
  14. Della, Victoria, Amy O’Neil, and Wendy Craig. 2015. Learning from traditional bullying interventions: A review of research on cyberbullying and best practice. Aggression and Violent Behavior 23: 61–68. [Google Scholar] [CrossRef]
  15. Domínguez, José, Enrique Álvarez, and Elia Vázquez. 2017. Predictors of violence in compulsory secondary education. Revista de Investigación Educativa 35: 337–51. [Google Scholar] [CrossRef]
  16. Duarte, Eva, María Gouveia-Pereira, Hugo Gomes, and Daniel Sampaio. 2019. Social representations about the functions of deliberate self-harm: Construction and validation of a questionnaire for portuguese adults. Behaviour Change 36: 12–28. [Google Scholar] [CrossRef]
  17. Elipe, Paz, María De la Oliva, and Rosario Del Rey. 2017. Homophobicbullying and cyberbullying: Study of a silenced problem. Journal of Homosexuality 65: 672–86. [Google Scholar] [CrossRef]
  18. Estévez, Estefanía, Teresa Jiménez, and David Moreno. 2018. Aggressive behavior in adolescence as a predictor of personal, family, and school adjustment problems. Psicothema 30: 66–73. [Google Scholar] [CrossRef]
  19. Fernández-Lasarte, Oihane, Eider Goñi, Igor Camino, and Miren Zubeldia. 2019. School Adjustment and Academic Self-concept in Secondary Education. Revista de Investigación Educativa 37: 163–79. [Google Scholar] [CrossRef]
  20. Fernández-Montalvo, Javier, Alicia Peñalva, and Itziar Irazábal. 2015. Internet Use Habits and Risk Behaviours in Preadolescence. Comunicar 44: 113–20. [Google Scholar] [CrossRef]
  21. Francis, Leslie, and Ursula McKenna. 2018. The Experience of Victimization among Muslim Adolescents in the UK: The Effect of Psychological and Religious Factors. Religions 9: 243. [Google Scholar] [CrossRef]
  22. Garaigordobil, Maite, and Vanesa Martínez-Valderrey. 2015. Effects of Cyberprogram 2.0 on “face-to-face” bullying, cyberbullying, and empathy. Psicothema 27: 45–51. [Google Scholar] [CrossRef]
  23. Giménez-Gualdo, Ana, Simón Hunter, Kevin Durkin, Pilar Arnaiz, and Javier Maquilón. 2015. The emotional impact of cyberbullying: Differences in perceptions and experiences as a function of role. Computers & Education 82: 228–35. [Google Scholar] [CrossRef]
  24. Glick, Peter, Jennifer Berdahl, and Natalya Alonso. 2018. Development and validation of the masculinity contest culture scale. Journal of Social Issues 74: 449–76. [Google Scholar] [CrossRef]
  25. Hall, William. 2016. Initial development and validation of the Bullyharm: The bullying, harassment, and aggression receipt measure. Psychology in the Schools 53: 984–1000. [Google Scholar] [CrossRef]
  26. Hinduja, Sameer, and Justin W. Patchin. 2017. Cultivating youth resilience to prevent bullying and cyberbullying victimization. Child Abuse & Neglect 73: 51–62. [Google Scholar] [CrossRef]
  27. Humphrey-Murto, Susan, Timothy J. Wood, and Lara Varpio. 2017. Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical Education 51: 994–95. [Google Scholar] [CrossRef]
  28. INE (Instituto Nacional de Estadística, Statistics National Institute). 2019. Demografía y Población. Available online: http://www.ine.es/inebmenu/mnu_migrac.htm (accessed on 7 September 2018).
  29. Jöreskog, Karl G., and Dag Sörbom. 1984. LISREL VI. Analysis of Linear Structural Relationships by Maximum Likelihood, Instrumental Variables, and Least Squares Methods. Mooresville: Scientific Software. [Google Scholar]
  30. Lecerf, Thierry, and Gary L. Canivez. 2018. Complementary exploratory and confirmatory factor analyses of the French WISC-V: Analyses based on the standardization sample: Correction to Lecerf and Canivez. Psychological Assessment 30: 793–808. [Google Scholar] [CrossRef]
  31. Llorent, Vicente J., Rosario Ortega-Ruiz, and Izabela Zych. 2016. Bullying and cyberbullying in minorities: Are they more vulnerable than the majority group? Frontiers Psychology 7: 1507. [Google Scholar] [CrossRef]
  32. López, Ernesto. 2018. The Delphi method in current educational research: A theoretical and methodological review. Educación XX1 21: 17–40. [Google Scholar] [CrossRef]
  33. Love, Steven, Lee Kannis-Dymand, and Geoff P. Lovell. 2019. Development and validation of the metacognitive processes during performances questionnaire. Psychology of Sport & Exercise 41: 91–98. [Google Scholar] [CrossRef]
  34. Martínez-García, Inmaculada, María T. Padilla-Carmona, and Magdalena Suárez-Ortega. 2019. Applying Delphi method to identification of success factors in entrepreneurship. Revista de Investigación Educativa 37: 129–46. [Google Scholar] [CrossRef]
  35. Matos, Armanda P. M., Cristina C. Vieira, Joao Amado, Teresa Pessoa, and María J. Martins. 2016. Cyberbullying in Portuguese Schools: Prevalence and Characteristics. Journal of School Violence 17: 123–37. [Google Scholar] [CrossRef]
  36. Maydeu-Olivares, Alberto, Dexin Shi, and Yves Rosseel. 2017. Assessing Fit in Structural Equation Models: A Monte-Carlo Evaluation of RMSEA Versus SRMR Confidence Intervals and Tests of Close Fit. Structural Equation Modeling: A Multidisciplinary Journal 25: 389–402. [Google Scholar] [CrossRef]
  37. Mérida, Rosario, Antonio Serrano, and Carmen Tabernero. 2015. Design and validation of a questionnaire for children’s self-esteem assessment. Revista de Investigación Educativa 33: 149–62. [Google Scholar] [CrossRef]
  38. Morin, Alexandre J. S., Katrin Arens, and Herbert W. Marsh. 2015. A Bifactor Exploratory Structural Equation Modeling Framework for the Identification of Distinct Sources of Construct-Relevant Psychometric Multidimensionality. Structural Equation Modeling: A Multidisciplinary Journal 23: 116–39. [Google Scholar] [CrossRef]
  39. Muñoz, Raquel, Raquel Ortega, María R. López, Carmen Batalla, Josep Manresa, Nuria Montellá, Adrés Chamarro, and Pepe Torán. 2016. The problematic use of Information and Communication Technologies (ICT), in adolescents by the cross sectional JOITIC study. BMC Pediatrics 16: 1–11. [Google Scholar] [CrossRef]
  40. Navarro, Raúl, Santiago Yubero, and Elisa Larrañaga. 2015. Psychosocial Risk Factors for Involvement in Bullying Behaviors: Empirical Comparison between Cyberbullying and Social Bullying Victims and Bullies. School Mental Health 7: 235–48. [Google Scholar] [CrossRef]
  41. Niemi, Pia M., Saija Benjamin, Arniika Kuusisto, and Liam Gearon. 2019. How and Why Education Counters Ideological Extremism in Finland. Religions 9: 420. [Google Scholar] [CrossRef]
  42. Olweus, Dan, and Susan Limber. 2018. Some problems with cyberbullying research. Current Opinion in Psychology 19: 139–43. [Google Scholar] [CrossRef]
  43. Palladino, Benedetta, Annalaura Nocentini, and Ersilia Menesini. 2015. Psychometric Properties of theFlorence CyberBullying-CyberVictimization Scales. Cyberpsychology, Behavior, and Social Networking 18: 112–19. [Google Scholar] [CrossRef]
  44. Patchin, Jastin W., and Sameer Hinduja. 2015. Measuring cyberbullying: Implications for research. Aggression and Violent Behaviour 23: 69–74. [Google Scholar] [CrossRef]
  45. Pena-Diaz, Carmen. 2019. Child language brokering: Challenges in Spanish intercultural education. Intercultural Education 30: 368–82. [Google Scholar] [CrossRef]
  46. Pottie, Kevin, Govinda Dahal, Katholiki Georgiades, Kamila Premji, and Ghayda Hassan. 2015. Do first generation immigrant adolescents face higher rates of bullying, violence and suicidal behaviours than do third generation and native born? Journal of Immigrant and Minority Health 17: 1557–66. [Google Scholar] [CrossRef]
  47. Prud’homme, Joseph. 2019. Security, Religion, and Political Culture: A Defense of Weak Disestablishment. Religions 10: 88. [Google Scholar] [CrossRef]
  48. Ramos-Soler, Irene, Carmen López-Sánchez, and Teresa Torrecillas-Lacave. 2018. Online risk perception in young people and its effects on digital behaviour. Comunicar 26: 71–79. [Google Scholar] [CrossRef] [Green Version]
  49. Rey, Lourdes, Cirenia Quintana-Orts, Sergio Mérida-López, and Natalio Extremera. 2018. Emotional intelligence and peer cybervictimisation in adolescents: Gender as moderator. Comunicar 26: 9–18. [Google Scholar] [CrossRef] [Green Version]
  50. Rhee, Siyon, Sei-Young Lee, and Sook-Hee Jung. 2017. Ethnic differences in bullying victimization and psychological distress: A test of an ecological model. Journal of Adolescence 60: 155–60. [Google Scholar] [CrossRef]
  51. Rikkonen, Pasi, Petri Tapio, and Heidi Rintamäki. 2019. Visions for small-scale renewable energy production on finnish farms—A delphi study on the opportunities for new business. Energy Policy 129: 939–48. [Google Scholar] [CrossRef]
  52. Rodríguez, Daniel, and Remedios Guzmán. 2019. Academic achievement of adolescents declared at risk. Revista de Investigación Educativa 37: 147–62. [Google Scholar] [CrossRef]
  53. Rodríguez-Gómez, David, Diego Castro, and Julio Meneses. 2018. Problematic uses of ICTs among young people in their personal and school life. Comunicar 26: 91–100. [Google Scholar] [CrossRef] [Green Version]
  54. Romera, Eva M., Juan J. Cano, Cristina García, and Rosario Ortega. 2016. Cyberbullying: Social Competence, Motivation and Peer Relationships. Comunicar 48: 71–79. [Google Scholar] [CrossRef]
  55. Runions, Kevin C., Michal Bak, and Therese Shaw. 2017. Disentangling functions of online aggression: The Cyber-Aggression typology questionnaire (CATQ). Aggressive Behavior 43: 74–84. [Google Scholar] [CrossRef]
  56. Selwyn, Neil, and Scott Bulfin. 2015. Exploring school regulation of students’ technology use—Rules that are made to be broken? Educational Review 68: 274–90. [Google Scholar] [CrossRef]
  57. Thomas, Hannah J., James G. Scott, Jason M. Coates, and Jason P. Connor. 2019. Development and validation of the bullying and cyberbullying scale for adolescents: A multi-dimensional measurement model. British Journal of Educational Psychology 89: 75–94. [Google Scholar] [CrossRef]
  58. Tsaur, Sheng-Hshiung, and Jin-Hua Tu. 2019. Cultural competence for tour leaders: Scale development and validation. Tourism Management 71: 9–17. [Google Scholar] [CrossRef]
  59. Van, Paul V., Jurjen Jansen, Joseph Onibokun, Jean Camp, and Petko Kusev. 2018. Security and privacy in online social networking: Risk perceptions and precautionary behaviour. Computers in Human Behavior 78: 283–97. [Google Scholar] [CrossRef]
  60. Varho, Vilja, Pasi Rikkonen, and Saija Rasi. 2016. Futures of distributed small-scale energy in Finland—A Delphi study of the opportunities and obstacles up to 2025. Technological Forecasting & Social Change 104: 30–37. [Google Scholar] [CrossRef]
  61. Vitoroulis, Irene, and Katholiki Georgiades. 2017. Bullying among immigrant and non-immigrant early adolescents: School- and student-level effects. Journal of Adolescence 61: 141–51. [Google Scholar] [CrossRef]
  62. Yudes-Gómez, Carolina, Daniela Baridon-Chauvie, and Joaquin M. González-Cabrera. 2018. Cyberbullying and problematic Internet use in Colombia, Uruguay and Spain: Cross-cultural study. Comunicar 26: 49–58. [Google Scholar] [CrossRef]
  63. Zych, Izabela, Rosario Ortega-Ruiz, and Rosario Del Rey. 2015. Scientific research on bullying and cyberbullying: Where have we been and where are we going. Aggression and Violent Behavior 24: 188–98. [Google Scholar] [CrossRef]
  64. Zych, Izabela, Rosario Ortega-Ruiz, and Rosario Marín-López. 2016. Cyberbullying: A systematic review of research, its prevalence and assessment issues in Spanish studies. Psicología Educativa 22: 5–18. [Google Scholar] [CrossRef]
Figure 1. Confirmatory Factor Analysis Diagram.
Figure 1. Confirmatory Factor Analysis Diagram.
Religions 10 00443 g001
Table 1. Distribution of sample by province.
Table 1. Distribution of sample by province.
ProvincesEFA N (%)CFA N (%)
Granada62 (8.6%)59 (7.8%)
Málaga61 (8.5%)70 (9.3%)
Almería57 (7.8%)54 (7.2%)
Jaén85 (11.7%)76 (10.1%)
Córdoba91(12.6%)102(13.5%)
Sevilla106 (14.7%)99 (13.1%)
Cádiz65 (9.0%)62 (8.2%)
Huelva42 (5.8%)47 (6.2%)
Melilla64 (8.8%)108 (14.3%)
Ceuta90 (12.5%)78 (10.3%)
TOTAL723 (100%)755 (100%)
Table 2. Kaiser, Meyer and Olkin (KMO) and Bartlett tests.
Table 2. Kaiser, Meyer and Olkin (KMO) and Bartlett tests.
Exploratory Factor Analysis
Kaiser-Meyer-Olkin measure of sampling adequacy0.926
Bartlett’s test of sphericity (Aprox. Chi-squared)31,491.842
gl 703
p 0.000
Table 3. Rotated Factor Matrix.
Table 3. Rotated Factor Matrix.
VariablesRotated Factor Matrix
F1F2F3
I260.9040.1970.134
I320.8940.1090.077
I310.8850.1670.072
I290.8740.2640.015
I270.8720.1250.156
I280.8650.2580.013
I300.8630.1710.147
I330.8450.1420.171
I380.8180.1840.188
I340.8090.0760.215
I360.7960.2150.257
I250.7750.1940.130
I240.7720.2940.075
I210.7720.2080.090
I200.6720.0770.262
I230.6710.2180.040
I220.6680.1910.160
I350.6670.0620.326
I370.6570.1110.256
I30.0590.8000.151
I130.2760.7870.123
I100.2710.7790.097
I120.2460.7760.106
I40.1450.7180.267
I20.1550.7130.267
I140.2370.6590.384
I10.1680.6350.312
I110.2700.6140.332
I80.2340.6000.428
I70.1840.5940.537
I90.2070.5680.340
I160.1880.2650.678
I180.0880.2420.662
I170.2390.4880.631
I150.3210.3550.617
I60.2560.4690.569
I190.3120.4750.525
I50.0970.4870.525
Table 4. NPAR, CMIN, DF, P and CMIN/DF.
Table 4. NPAR, CMIN, DF, P and CMIN/DF.
ModelNPARCMINDFpCMIN/DF
Default model1172414.5366620.0003.647
Saturated model7790.0000
Independence model3810,789.8227410.00014.561
Table 5. Comparative fit index and Tucker-Lewis index tests.
Table 5. Comparative fit index and Tucker-Lewis index tests.
ModelNFIIFITLICFI
Default Model0.7760.7500.8050.826
Saturated Model1.0001.000 1.000
Independence Model0.0000.0000.0000.000
Table 6. Root Mean Square Error of Approximation Test.
Table 6. Root Mean Square Error of Approximation Test.
ModelRMSEA
Default Model 0.059
Independence Model0.134

Share and Cite

MDPI and ACS Style

Tomé-Fernández, M.; Ortiz-Marcos, J.M.; Olmedo-Moreno, E.M. Educational Environments with Cultural and Religious Diversity: Psychometric Analysis of the Cyberbullying Scale. Religions 2019, 10, 443. https://0-doi-org.brum.beds.ac.uk/10.3390/rel10070443

AMA Style

Tomé-Fernández M, Ortiz-Marcos JM, Olmedo-Moreno EM. Educational Environments with Cultural and Religious Diversity: Psychometric Analysis of the Cyberbullying Scale. Religions. 2019; 10(7):443. https://0-doi-org.brum.beds.ac.uk/10.3390/rel10070443

Chicago/Turabian Style

Tomé-Fernández, María, José Manuel Ortiz-Marcos, and Eva María Olmedo-Moreno. 2019. "Educational Environments with Cultural and Religious Diversity: Psychometric Analysis of the Cyberbullying Scale" Religions 10, no. 7: 443. https://0-doi-org.brum.beds.ac.uk/10.3390/rel10070443

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop