What’s the Matter with Kansas?: The Development and Confirmation of the Evolutionary Attitudes and Literacy Survey (EALS)

Abstract

The present survey was designed to assess predominant regional belief systems and the roles these beliefs play in science understanding and attitudes, and curricular effectiveness in colleges and universities. To this end, we created a wide variety of theory-driven subscales (lower order factors) reflecting, for example, exposure to evolutionary material, young earth creationist beliefs, moral and social objections, political ideology, endorsement of intelligent design fallacies, knowledge (and distrust) of the scientific enterprise, and attitudes of evolutionary theory’s relevance in several domains (e.g., sciences and humanities). We also included potentially important demographic variables (e.g., rural upbringing, family size). Finally, we assessed openness to experience, a key facet of personality. Hierarchical Structural Equation Modeling (SEM) analysis showed the 16 constructs to have a theoretically meaningful and quantitatively coherent higher order factor structure. In this large Kansas sample of university students, creationist reasoning and conservative orientation (political and religious) were negatively associated with exposure to evolutionary theory, knowledge about it, and positive attitudes toward its relevance. At the same time, exposure to the theory was positively associated with knowledge and positive attitudes. Importantly, though most Kansas-specific demographic variables (e.g., rural origins) were largely unrelated to outcomes of interest in this university-based sample, the personality factor openness to experience appears to be highly relevant for several higher order factors (e.g., exposure, knowledge and relevance, and creationist reasoning). We close with implications for educators and the next steps in survey development.

Evolution is a theory. God is the true creator and He created animals to change w/their environment.

–KS teacher

Thank you for trying to teach what they’re not in Kansas public schools. I was educated mostly by Kansas public schools. Even in high school, we barely touched evolution, and my biology teacher was too afraid to talk about human evolution! No wonder my relatives outside of the state don’t take Kansas seriously. ¹

Introduction

The state of Kansas has had lengthy and very public problems with the theory of evolution, the lion’s share of which were centered on high school science curriculum standards and culminated in the “Kansas Evolution Hearings” in May of 2005, an event widely held by the scientific community to be a “kangaroo court.” Rather than staying local, the science debates in Kansas have been noted in international news outlets and have led the state to be the brunt of jokes on the web (The Onion). Television (Jon Stewart’s The Daily Show), and print magazines (The New Yorker).

What’s the matter with Kansas?²

In fact, the problem is not limited to Kansas. Anti-evolution sentiment permeates not only public perception but also science education nationwide. The United States, however, may be unique in its politicization of evolutionary theory (Miller et al. 2006). During recent presidential administrations, the Republican Party explicitly advocated teaching “creation science” as part of high school curricula (and the Discovery Institute was founded to this end). In 1985, 45% of the nation’s population accepted that humans had evolved, with 7% being unsure. By 2005, only 40% expressed acceptance of human evolution (Miller et al. 2006; see also Pew 2009; Gallup Poll 2009), while 21% were unsure (31% of the population believes that humans have always existed in their present form). In a widely distributed, multinational survey, only Turkey scored lower on acceptance of evolution than the United States (Miller et al. 2006). In terms of widely held stereotypes, pro-evolutionists tend to characterize evolution objectors as backwards, biblical literalists who fear the shaking of their worldview that promises redemption and salvation. Anti-evolutionists portray pro-evolutionists as immoral, Godless materialists who undermine the fabric of society.

Is there any truth to these stereotypes and, if there is, what are the implications for science education, especially as it pertains to teaching the theory of evolution? To explore these and other related questions, we created a multi-item survey to assess not only political and spiritual leanings but also knowledge of evolution, distrust of the scientific enterprise, knowledge about the scientific enterprise, and attitudes toward and objections against evolutionary theory. Our goal ultimately is to create a standard tool to assess the curricular effectiveness of courses on evolution and/or biology in colleges and universities (e.g., O’Brien et al. 2009) and to assess the effectiveness of the ID propaganda machine on a regional basis by way of assessing the endorsement of intelligent design fallacies.

The Present Study

The present survey represents an important step in the process of designing a useable tool for university and college educators to assess (a) predominant regional belief systems and their roles in science understanding and attitudes, (b) curricular effectiveness and attainment of specified learning goals, and (c) changes in attitudes about course material. Though several interesting measurement tools exist (e.g., Rutledge and Warden 1999; Rutledge and Sadler 2007; Aikenhead and Ryan 1992; Siegel and Ranney 2003; Lombrozo et al. 2008; Johnson and Peeples 1987; Miller et al. 2006), to date, the present survey represents the most comprehensive attempt to incorporate a broad array of explanatory variables of theoretical importance.

Accordingly, the present work has three primary goals. First, we seek to model the structure of beliefs about and attitudes toward evolutionary theory and its application. To this end, we will apply confirmatory factor analysis to our multi-item survey to derive a reduced array of meaningful constructs. Doing so will enable us to pursue our second goal, namely, to explore and model the complex interrelationships among the reduced number of constructs for maximal communicative and explanatory utility. Finally, we will employ hierarchical regression analysis to explore the degree to which key demographic and personality variables predict said constructs through which we will be able to address questions particularly concerning our regional sample.

Creating the Survey

Many learned authors have laid important groundwork for our present measures. Accordingly, we may accurately claim that our intended dimensions are theoretically grounded. We will take each of these subscales in turn.

Exposure to Evolutionary Theory

Research has shown that those who seek information about evolutionary theory may in fact know more about it and accordingly hold more positive attitudes (Lombrozo et al. 2008). Natural history museums, for example, function in part to teach the public about evolution (Diamond and Evans 2007). Therefore, museum visitors appear to (a) know more about evolution than non-visitors (McFadden et al. 2007) and (b) hold more positive views. Accordingly, 28% of museum visitors engage in creationist reasoning compared to 45% of the general public (Evans et al. 2006). For these reasons, it is important to assess respondents’ exposure to evolution during youth (presumably due to high school education and/or parental influence) and self-selected exposure in adulthood.

Religious Identity and Activity

One’s religious/spiritual orientation appears to play a primary role in one’s standing on evolutionary theory. For example, conservative Christian denominations are less accepting than more liberal denominations (Scott 2004). In addition to assessing participants’ qualitative religious affiliation (identity), we also queried them about how active they are in their spiritual behavior (e.g., frequency of prayer and service attendance), previously documented to be negatively associated with knowledge of and attitudes toward evolution (e.g., Lombrozo et al. 2008).

Young Earth Creationist Beliefs

In order to assess the degree to which biblical literalism (i.e., the earth was made by God in its present form over a span of six days less than 10,000 years ago, etc.) is represented on college campuses, and its role in attitudes toward and knowledge about evolution, we assessed young earth creationist beliefs as inspired by our reading of, for example, Dinosaurs by Design (Gish et al. 1992; see also Whitcomb and Morris 1961).

Moral Objections

Arguably, objections to evolutionary theory based on moral grounds (e.g., evolutionary theory stripping meaning from our lives) are associated with religious orientation (Dennett 1995; Scott 2004). Indeed, 55% of Americans believe that science and religion are in conflict (Pew 2009). At the same time, moral objections may contribute explanatory power over and above religiosity.

Political Ideology and Activity

When construed as motivated social cognition with deep psychological roots (i.e., a belief system that reduces uncertainty and threat; Carney et al. 2008), political conservatism has been found to be associated with support for a traditional authority, religious forms of morality, perceptions of the world as dangerous, an intolerance for ambiguity and unpredictability, and more moral condemnation (Jost 2006). As such, conservatism has been described as having a palliative function for the stress associated with uncertainty and change (Jost and Hunyady 2002). In contrast, liberalism is associated with more open-minded thinking, a predilection for novelty and creativity, and a willingness to engage in prolonged, effortful cognition (Jost 2006; Kruglanski et al. 2006; Jost et al. 2003). Because the theory of evolution does not ascribe special meaning to human life, nor offer prescriptions on how a life can be well lived, its acceptance might be expected to positively associate with liberalism and negatively associate with conservatism (Miller et al. 2006 supporting material; Paterson and Rossow 1999). In addition to assessing political self-identity, we also asked participants to report the degree to which they are politically active or to what degree they perceive that their political selves influence their daily lives.

Attitudes Toward Life

Relevant to religious and political ideology are one’s beliefs about the timing of life’s beginning (Miller et al. 2006). If one holds an ideological commitment to human exemptionalism (e.g., humans standing apart from the natural world; Richards 2008), then one also might view human life as sacred at the moment of conception. Accordingly, we assessed such beliefs.

ID Fallacies

The intelligent design proponents are technologically sophisticated and well organized. These facts make it easy to identify key intelligent design (ID) talking points and slogans (e.g., talkorigins.org; discovery.org). To our way of thinking, the degree to which respondents endorsed such items (e.g., “Evolution is a theory in crisis”; Denton 1986; Perloff 1999) reflects the degree to which the ID movement has been successful in its dissemination enterprise.

Scientific, Genetic, and Evolutionary Literacy

In terms of literacy, we assessed both knowledge of evolutionary processes and genetic literacy (Miller et al. 2006), fully anticipating that both of these constructs would be positively correlated. In addition, we assessed a basic knowledge about the scientific enterprise (i.e., scientific epistemology: e.g., understanding the distinction between “theory” and “belief”; see Liang et al. 2006; Johnson and Peeples 1987; Lombrozo et al. 2008). Regarding knowledge about evolution, we included two types of questions: those that can be easily answered with some basic exposure (e.g., tree thinking; Baum et al. 2005; the role of mutation; Garvin-Doxas and Klymkowsky 2008) and those that appear to reflect common and persistent misconceptions that obstruct understanding (e.g., Alters and Nelson 2002; Evans 2001). For example, students of biology tend to hold on to Lamarckian-esque notions (i.e., the inheritance of acquired characteristics) that are difficult to change even after direct instruction (e.g., Bishop and Anderson 1990). In this sense, then, Lamarckianism is a conventional wisdom in that it is a simplistic and widely accepted false proposition. Thus, we included two constructs encompassing knowledge, evolutionary misconceptions and evolutionary knowledge, with the suspicion that they may behave differently from each other.

Distrust of the Scientific Enterprise

Additionally undermining the acceptance of evolutionary theory might be one’s general attitudes toward the scientific enterprise and its methods (Rutledge and Warden 1999), especially as the enterprise is associated with evolution (Lombrozo et al. 2008; Downie and Barron 2000; Scott 2004).

Relevance of Evolutionary Theory

Additional obstacles to accepting evolutionary theory are attitudes regarding its relevance (e.g., Wilson 2005). Regarding attitudes, we queried respondents about their attitudes on evolutionary theory’s explanatory relevance. We hypothesize that the more participants know about the theory of evolution, the more they will find it relevant for fields evoking it (e.g., zoology, botany, and human affairs). Intentionally, the items were graded from the relatively non-controversial application to plants, biology, and animals and then stepping into applications involving humans such as human origins, human behavior, and ultimately the humanities as a discipline (Wilson 2005). Many authors have found little opposition to evolution when it is applied to domains outside the purview of humans. When it comes to human origins or behavior, however, there is resistance even in college students (Sinclair et al. 1997; Richards 2008).

Social Objections

An additional obstacle to the acceptance of evolutionary theory is its perceived implications. Many students reason that if natural selection acts on variability within a species, and that some of that variability is “better” than other variants for the ecological context, then such variability must exist in humans. From there, it is a short leap to racist and sexist conclusions, many believe. Thus, the implications of the theory for human affairs are seen as unsavory. Early fallacious applications of Darwin’s theory (e.g., Spencer and social evolution) seem to suggest that the theory has a racist, sexist past (see also Eugenics movement). Understandably then, social objections abound, and they may come from both the left and right (as recent history suggests; Segerstråle 2001). Indeed, after course instruction, students appear to be even more aware of the historical baggage associated with the theory, and endorsement of these items increases (e.g., O’Brien et al. 2009; see also Brem et al. 2003).

Demographics

Though there is a perception (perhaps accurate) that people of the “bible belt” would be less accepting of evolution, that predictability may be amplified when additional demographic variables are considered such as the education of parents, size of graduating class, whether the community is more rural or less rural, etc. In these senses, with its large rural community and multitude of very small high schools, Kansas provides an excellent opportunity to explore these stereotypes.

Personality

Personality reflects the basic ways in which humans differ in relatively stable patterns of acting and experiencing. Many psychologists endorse what is known as the five-factor model which describes five fundamental dimensions of personality (i.e., neuroticism, agreeableness, conscientiousness, extroversion, and openness to experience; McCrae and John 1992). Among other qualities, openness to experience represents intellectual curiosity and preference for variety. As such, we measured openness to experience because it has been previously found to be related to the willingness to entertain difficult subject matter like the theory of evolution (Sinatra et al. 2003).

Materials and Methods

We developed the present instantiation of the survey over the course of three semesters. The present analyses are based on data drawn from two classes (Child Psychology and Social Psychology) via a web interface. The classes were chosen based on the fact that they both attract a wide variety of students representing many majors and disciplines and that neither course exposes students to an evolutionary curriculum of any note.

Participants

In total, 371 undergraduates at the university participated and represented nearly 40 declared majors. Three hundred twenty-seven were drawn from Child Psychology (37 majors comprised this class, 32% were declared Psychology majors) and 44 came from a Social Psychology course (11 majors were represented, 63% were declared Psychology majors). The average age was 20.67 years; 102 were men and 269 were women. Additionally, the average high school graduating class was 374.21 (SD = 321.71), and the most frequent response for both the participant’s father‘s (N = 109, 30.36%) and mother’s (N = 126, 34.24%) education was 4-year college degree. Finally, the average rating for the degree to which one’s pre-university environment was rural was 3.45 (SD = 1.62) on a seven-point scale. For both classes, the students were invited to participate for extra course credit and were asked to complete the survey outside of class time via an easy-to-access link emailed to them. The students reported informally that it took them about 25 minutes to complete.

Materials

The Evolutionary Attitudes and Literacy Survey

The survey consists of 17 pages of web-presented items where respondents rated the degree to which they agree or disagree with 104 statements³ on a 1 to 7 scale (1 = strongly disagree, 4 = neither agree nor disagree, 7 = strongly agree). Five statements were removed from the current analyses because of very low item variance for these items that was accounted for by their corresponding constructs. The bulk of the survey was organized to measure 16 meaningful constructs introduced above: Self-exposure to evolution, youth exposure to evolution, religious activity⁴, young earth creationist beliefs, moral objections, political ideology, political activity, attitudes toward life, intelligent design fallacies, knowledge about the scientific enterprise, genetic literacy, evolutionary knowledge, evolutionary misconceptions, distrust of the scientific enterprise, relevance of evolutionary theory, and social objections. (Example items are provided below. The survey in its entirety can be found in Table 1). Key demographic variables were also assessed and explored here, including gender, size of high school graduating class, father’s education, mother’s education, the degree to which one’s pre-university environment was rural, and number of siblings.

The constructs were as follows: (1) Self-exposure to evolution (five items): I read science magazines featuring evolution (e.g., Discover, National Geographic, Nature). (2) Youth exposure to evolution (four items): How much training in evolution did you receive in high school? I have visited natural history museums on field trips or with family. (3) Religious activity (six items): To what degree does religion impact your daily life? How much do you believe in God? (4) Young earth creationist beliefs (nine items): The Earth isn’t old enough for evolution to have taken place. Present animal diversity can be explained by the Great Flood. (5) Moral objections (five items): People who accept evolution do not believe in God. Darwinism strips meaning from our lives. (6) Conservative self-identity (i.e., political ideology; five items): To what degree are you conservative? In general, how do you self-identify politically? (7) Political Activity (six items): To what degree are you politically active? To what degree do your political views influence your daily life? (8) Attitudes toward life (three items; all from Miller et al. 2006): All stages of human life—embryo, fetus, child, adult—should have the same legal protections. Life begins at conception. (9) Intelligent design fallacies (11 items): Complex biological systems cannot come about by slight successive modifications (i.e., they are irreducibly complex). Natural selection cannot create complex structures; it is like a tornado blowing through a junkyard and creating a 747. (10) Knowledge about the scientific enterprise (six items): Scientific ideas can be tested and supported by feelings and beliefs. For scientific evidence to be deemed adequate, it must be reproducible by others. (11) Genetic literacy (nine items; six items directly from Miller et al. 2006): Today it is not possible to transfer genes from one species of animal to another. Ordinary tomatoes do not have genes, whereas genetically modified tomatoes do. (12) Evolutionary knowledge (seven items): Increased genetic variability makes a population more resistant to extinction. The more recently species share a common ancestor, the more closely related they are. (13) Evolutionary misconceptions (six items): Natural selection is a random process. Characteristics acquired during the lifetime of an organism are passed down to that individual’s offspring. (14) Distrust of the scientific enterprise (seven items; three drawn from Rutledge and Sadler 2007): Contemporary methods of determining the age of fossils and rocks are untrustworthy. Evolutionary theorists believe that if something is natural then it is good or right. (15) Relevance of evolutionary theory (nine items): The theory of evolution helps us understand plants. Evolutionary theory is highly relevant for the humanities (e.g., history, literature, philosophy). (16) Social objections (six items): The theory of evolution has contributed to racism. Applying the theory of evolution to human affairs implies we are not fully in control of our behavior.

The Big Five Inventory

In its entirety, the Big Five Inventory (BFI; Benet-Martinez and John 1998) is a collection of 44 short descriptive phrases designed to assess openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. Only the ten-item scale for openness to experience (e.g., “I see myself as someone who is an ingenious, deep thinker”) was used for the present study as it conceptually addresses the “breadth, depth, originality, and complexity of an individual’s mental and experiential life” (John et al. 2008, p. 120). The BFI is widely used, has been internationally validated, and is currently available in at least ten languages.

Analytic Methods

Prior to analyses, we screened the data for univariate and multivariate outliers, and normality. None of the items demonstrated significant skew or kurtosis (i.e., no values were greater than ±2 in either measure of non-normality). Additionally, we examined scatter plots of all item pairs within each hypothesized construct and found support for linear relations and bivariate normality among them. Overall, <5% of the data were missing. Thus, missing data were inputted one time via Markov chain Monte Carlo estimation using Proc MI in SAS, version 9.1.3 (Enders 2010; SAS Institute 2005).

Because the 16 constructs were developed a priori, a confirmatory factor analysis (CFA) was used across the 104-scale items in order to verify that each scale item (i.e., manifest indicator) loaded on the latent construct that it was developed to measure. For example, a participant’s response to the item “individuals don’t evolve, species do” was hypothesized to be an indicator of evolutionary knowledge. Therefore, a loading was specified between the latent construct evolutionary knowledge and the scale item. With CFA, the collection of hypothesized paths is estimated to determine if the items’ covariations are adequately accounted for by their corresponding latent constructs.

We fit a second-order CFA model to ease interpretation of the scale and increase parsimony. This higher order CFA model was fit to the unit-weighted factor scores of the item-level CFA and was used to determine if some of the 16 lower order latent factors possess covariations that can be accounted for (i.e., further explained) by a higher order factor. Specifically, we took the mean of all of the items within a particular construct and used these construct means as the indicators for the higher order CFA. This approach gives the same generalizability as a simultaneously fitted model but has the advantages of an evaluated model fit separately for the lower order and the higher order levels. This approach also has added benefits in that it requires fewer parameter estimates, reduces sampling error, and decreases the likelihood of correlated residuals (Little et al. 2002) compared to item-level analyses.

Once the structure of the higher order factors was confirmed via the CFA model, we conducted sequential latent regressions for each higher order factor. Because the constructs are corrected for measurement error, latent regressions provide unbiased estimates of the regression relations among constructs. Latent regressions also allow for the predictive role of numerous latent variables (e.g., demographics and openness to experience) to be examined across multiple dependent variables (in this case, the six higher order factors of the Evolutionary Attitudes and Literacy Survey (EALS)). The first set of predictors consisted of the measured demographic variables. Using item parcels (Little et al. 2002), we examined whether the nine-item openness to experience construct accounts for unique variance above and beyond demographic variables. In other words, the unique effect of openness to experience was examined as a predictor for each of the six higher order factors of the EALS. All factor analyses and latent regressions were conducted using maximum likelihood estimation via the software package Mplus, version 5.2.

Results

Confirmatory Factor Analysis

Table 1 displays all items with their variance accounted for by the underlying factor for the EALS. Additionally, the latent factors are accordingly named and displayed with their alpha reliabilities, means, and standard deviations. Because the model was identified by fixing the variances of the latent constructs to 1.00, the standardized factor loading for a particular item can be calculated by taking the square root of the item’s variance explained by the underlying factor. Overall, the item-level CFA supported our hypothesized model with acceptable fit (χ² (5,132, N = 371) = 10,304.92, p < 0.0001; comparative fit index (CFI) = 0.79; Tucker–Lewis fit index (TLI) = 0.78; RMSEA = 0.052_{(0.051,0.054)}, SRMR = 0.065). Although the CFI and TLI were low, the other fit statistics demonstrate acceptable model fit. In order to ensure that the CFA produced the best fitting model, modification indices were examined. These indices were relatively low and mainly included within-construct correlations, which should be expected with a large model.

Table 1 Means, standard deviations, and alpha coefficients for each construct, as well as item variance explained for each item of the EALS

Table 2 contains the lower order latent factor correlations. These lower order correlations were examined to ensure there were 16 distinct constructs. A high correlation (r = 0.99) between intelligent design fallacies and distrust of the scientific enterprise was a potential concern, and thus, a new nested model was created to test if this correlation could be considered 1.00. For the nested model, all other latent correlations with the constructs intelligent design fallacies and distrust of the scientific enterprise were equated. For example, the correlation between intelligent design fallacies and evolutionary knowledge was forced to equal the correlation between distrust of the scientific enterprise and evolutionary knowledge. This process was repeated for the remaining relationships between these constructs so that only one set of latent correlations was estimated for these constructs instead of two. Finally, the latent correlation between intelligent design fallacies and distrust of the scientific enterprise was fixed to 1. Thus, this new model contained fewer parameter estimates than the original CFA (i.e., nested within the original CFA) and could be tested for improved fit. A chi-square difference test of the resulting model (χ ²_diff (15, N = 371) = 42.466, p < 0.0001) suggested that these two constructs should not have their correlations equated and that they indeed have differential relationships with other constructs. Thus, based on these results as well as strong theoretical support, the two constructs were judged to be distinct enough from one another to keep them separate.

Table 2 Latent correlations between 16 lower order constructs

Hierarchical Structure

Figure 1 displays a path diagram depicting the higher order factor model. Overall, the hierarchical CFA produced very good model fit (χ² (203, N = 371) = 421.61, p < 0.0001, CFI = 0.95, TLI = 0.93, RMSEA = 0.054_{(0.047,0.061)}, SRMR = 0.048). Although two higher order constructs each significantly accounted for the variance of only a single lower order construct (political activity and evolutionary misconceptions), four other higher order constructs contained multiple lower order constructs. Specifically, a higher order construct (hereafter referred to as Political/Religious Conservatism) was positively indicated by conservative self-identity (political ideology), religious activity, attitudes toward life (pro-life direction), intelligent design fallacies, and young earth creationist beliefs and was negatively indicated by relevance of evolutionary theory. Another construct, hereafter referred to as Creationist Reasoning, was positively indicated by intelligent design fallacies, young earth creationist beliefs, moral objections, social objections, and distrust of the scientific enterprise. The construct Knowledge/Relevance was positively indicated by relevance of evolutionary theory, genetic literacy, evolutionary knowledge, and knowledge about the scientific enterprise. Lastly, both self-exposure to evolution and youth exposure to evolution were positively indicated by a higher order factor we refer to as Exposure to Evolution.

Fig. 1

Hierarchical model of the EALS

Inter-Higher Order Factor Correlations

Table 3 contains the latent correlations for the higher order factors. As can be seen in Table 3, Creationist Reasoning was positively correlated with Political/Religious Conservatism (r = 0.687) and negatively correlated with Knowledge/Relevance (r = −0.798) and Exposure to Evolution (r = −0.588). Accordingly, Political/Religious Conservatism was also negatively correlated with Knowledge/Relevance (r = −0.459). Though Exposure to Evolution was positively associated with Knowledge/Relevance (r = 0.555), Evolutionary Misconceptions was wholly uncorrelated with Exposure to Evolution (r = 0.013).

Table 3 Correlations among the six latent constructs representing evolutionary attitudes and literacy

Latent Regressions

Table 4 displays the R² before and after openness to experience was added as a predictor, as well as the standardized regression coefficients (β) for each predictor after openness to experience was added. All significant values are based on the Wald statistic.

Table 4 Latent regressions for the six higher order EALS constructs

As a set, the demographic variables only predicted a small amount of variance for each construct, with Evolutionary Misconceptions having the least amount of variance explained (R² = 0.02) and Exposure to Evolution having the most variance explained (R² = 0.083). No demographic variables significantly predicted Political Activity. High school size significantly predicted Political/Religious Conservatism (β = −0.176, p < 0.01). Gender (being male versus being female) significantly predicted Knowledge/Relevance (β = 0.137, p < 0.05). Father’s education significantly predicted Creationist Reasoning (β = −0.126, p < 0.05). Finally, gender (β = 0.139, p < 0.05) and father’s education (β = 0.167, p < 0.01) significantly predicted Exposure to Evolution.

The increase in R² when openness to experience was added as a predictor varied across each construct. Openness to experience was a significant positive predictor for Political Activity (β = 0.208, p < 0.001, R² = 0.064), Knowledge/Relevance (β = 0.316, p < 0.001, R² = 0.152), and Exposure to Evolution (β = 0.258, p < 0.001, R² = 0.168). Conversely, openness to experience was a significant negative predictor for Creationist Reasoning (β = −0.248, p < 0.001, R² = 0.118) and Political/Religious Conservatism (β = −0.152 p < 0.05, R² = 0.092).

Discussion

The present study had two goals: to confirm the structure of our survey instrument in order to maximize its utility as an assessment tool and then to use the said structure to explore predictors of knowledge and attitudes in Kansas university students. Regarding the first and primary goal, our 104-item survey distilled down well into 16 multi-item constructs (all with excellent reliabilities) in this large sample of university students from varied disciplines. Specifically, the absence of correlated residuals, few dual-factor loadings between items, as well as low modification indices demonstrated both convergent and divergent validity. Similar items loaded uniquely onto a single construct (i.e., convergence) and, as demonstrated in Table 2, were distinct from other constructs (i.e., divergence). Moreover, the hierarchical SEM analysis showed the 16 constructs to have a theoretically meaningful and quantitatively coherent higher order factor structure. For the sake of parsimony and the avoidance of redundancy (because of the high correlations among the 16 constructs), we can now focus our discussion on this higher order structure with which we addressed our secondary goal of predicting attitudes and knowledge of Kansan university students.⁵

Higher Order Survey and the Structure of Cognition and Attitudes

Political Activity

Political activity, though it did not map onto any other constructs in our model, was positively correlated with Exposure to Evolution (0.436), negatively correlated with Creationist Reasoning (−0.273), and openness to experience alone accounted for the bulk of its variance accounted for (4%; see Table 4). These results seem to suggest that politically active individuals, regardless of political self-identity, are critical knowledge seekers and as such may be attracted intellectually to the politically charged topic of evolution (however, Political Activity does not appear to be significantly related to Knowledge/Relevance in this sample).

Political/Religious Conservatism

Given the sociopolitical atmosphere surrounding evolution education over the last decade, we are not surprised to see a high positive correlation between Creationist Reasoning and Political/Religious Conservatism (see also O’Brien et al. 2009). Similarly, the significant correlation between Political/Religious Conservatism and Exposure to Evolution stands to reason: The conservatively oriented who engage in creationist reasoning (replete with moral and social objections, as well as a distrust of the scientific enterprise) do not presently seek out exposure to evolution, nor were they exposed to it early on. Accordingly, both Creationist Reasoning (especially) and Political/Religious Conservatism are negatively associated with Knowledge/Relevance in this university sample.

Exposure, Knowledge, and Attitudes

At the same time, Exposure to Evolution is strongly positively associated with Knowledge/Relevance; that is, those who have been exposed and choose to expose themselves to evolution seem to have some understanding and views of its relevancy (see also O’Brien et al. 2009; Moore and Cotner 2009). This point might be of particular interest to educators, though this relationship should be interpreted with caution because causal direction cannot be assured. Positive attitudes may cause people to seek out information, exposure can lead to positive attitudes, or some other variable can cause them both (see, for example, the discussion of openness to experience below). Moreover, self-reports of exposure may be biased depending on one’s attitudes; those positively inclined may either report or remember more exposure experiences.

Evolutionary Misconceptions

An additional novelty offered by the present survey is the Evolutionary Misconceptions higher order factor. Perhaps seeming paradoxical at first, Evolutionary Misconceptions is uncorrelated with Exposure to Evolution. To try to make sense of these patterns (which should be validated on additional samples), we offer some speculations about cognitive biases.

A significant body of empirical research suggests that misconceptions are common, difficult to change, and manifest in predictable ways (D’Avanzo 2003; Donnelly et al. 2007; Sinatra et al. 2008). These cognitive biases can be thought of as cognitive organizational “rules of thumb” and as such reflect “common sense” thinking (Sinatra et al. 2008). The most prominent naïve views relating to evolution are essentialism (species and their traits are immutable), teleology (species evolve toward “perfection”), and intentionality (evolution is due to the intention or “need” of the species). Informal explanations of natural selection often ascribe agency to evolutionary processes (Moore et al. 2002)⁶, and such falsehoods may become more believable with repeated exposure (i.e., the “illusion of truth”; Halpern 2003). In the end, the null relationship between Evolutionary Misconceptions and Exposure to Evolution in this university sample suggests that we may not be able to predict falling prey to these falsehoods based on one’s exposure to evolutionary theory. At the very least, this relationship suggests that educators should be mindful of the ways they describe evolutionary processes to students of all levels. At the same time, Evolutionary Misconceptions moderately negatively related to Creationist Reasoning (r = −0.257). In contrast to the possible process described above, creationists’ misunderstandings of evolution probably derive from fundamental religious beliefs that are perceived to be at odds with evolutionary theory, be that reasoning sound or not.

Kansas Demographics

Though stereotypes about Kansans might suggest that those living rurally or coming from very small high schools would be more susceptible to political inactivity, religious conservatism, negative attitudes, and lack of literacy and exposure, little evidence supported these clichés in the present university sample. The latent regressions showed that high school size was modestly related to conservatism (the smaller the class, the higher endorsement of conservatism). But whether those participants were raised in a rural community or came from large families appears to be largely immaterial. Similarly, the education of the parents played less of a role than some might expect; education of the mother was not significantly related to any of the constructs, and education of the father slightly negatively related to Creationist Reasoning and positively associated with Exposure to Evolution. Gender, in contrast, was associated with both Knowledge/Relevance and Exposure to Evolution (favoring males).

Openness to Experience

Demographic variables (gender, rural, education of parents) played little role in predicting Knowledge/Relevance or Creationist Reasoning. On the other hand, the personality factor openness to experience (a stable aspect of personality) appears to play a rather large role in this Kansas university sample (see also Sinatra et al. 2003; Sá et al. 1999). Namely, openness to experience positively predicted Political Activity, Knowledge/Relevance, and Exposure to Evolution and negatively predicted Creationist Reasoning and Political/Religious Conservatism (see Kruglanski et al. 2006; Jost et al. 2003). In fact, a good deal of the variance accounted for in these factors was due to personality, especially for Knowledge/Relevance (10% of the variance uniquely), Creationist Reasoning (approximately 6% uniquely accounted for), and Exposure to Evolution (approximately 7% uniquely).

These patterns raise an important question: Can educators increase openness to experience with the possible trickle-down effect of students pursuing, understanding, and finding relevant their science education? Although openness to experience has generally been found to normatively increase through adolescence and young adulthood (Roberts et al. 2006; Lüdtke et al. 2009), significant changes in openness to experience over the college years appear to be due to a small proportion of individuals (Robins et al. 2001). Perhaps most instructive on this point is the work addressing creativity enhancement in so far as creativity is a central feature of openness to experience. Work in this area has addressed, among other things, fostering intrinsic motivation and collaborative atmospheres, teacher qualities, attitudes, and behaviors that enhance instructional effectiveness and innovative and flexible curricular approaches (see Esquivel 1995 for review).

Implications for Educators

A number of implications for educators can be derived from the present study. First, for example, Political/Religious Conservatism seems to hinder understanding of and attitudes of relevance toward evolutionary theory. Some authors argue that understanding is more important than acceptance (e.g., Ingram and Nelson 2006; Rutledge and Sadler 2007), and we agree to a point. But, knowledge and attitudes appear to be strongly linked, as evidenced by our singular Knowledge/Relevance higher order factor (see also Sinatra et al. 2003; Lawson and Worsnop 1992 for an extended discussion of cause and effect).

Moreover, standard practices of teaching philosophy of science have been criticized as inadequate (Moore et al. 2002; Alters and Nelson 2002) as they can fail to provide learners with an adequate organizational framework for understanding evolution (Turner and Sullenger 1999). Put simply, to understand basic principles in science and to overcome common misconceptions, one must possess sound scientific schema from which to confront existing cognitive biases and to integrate and accurately reproduce scientific knowledge. As Carl Sagan observed, “In the course of their training, scientists are equipped with a baloney detection kit.…What’s in the kit? Tools for skeptical thinking.…What skeptical thinking boils down to is the means to construct, and to understand, a reasoned argument and—especially important—to recognize a fallacious or fraudulent argument” (Sagan 1995; p. 196). Without skeptical thinking, students’ understandings will more easily fall prey to pervasive cognitive biases, a situation well understood by the Intelligent Design movement.

Limitations

Ultimately, we may come to find that our derived structure is limited to our Kansas university sample (but preliminary analyses on data from New York students suggest that this concern may be unwarranted). Additionally, the very high correlation between the lower order constructs of intelligent design fallacies and distrust of the scientific enterprise should be interpreted with caution. Although the chi-square difference test indicated a significant difference between the nested model where the constructs were equated and the model where they were distinct, this difference test may reflect a type I error as the chi-square difference test can produce significant differences when testing within large samples. Therefore, a clear distinction between intelligent design fallacies and distrust of the scientific enterprise subscales cannot be assured. Future research should explore the relationship between these two constructs with new and different samples.

Conclusions and Future Directions

We hope that the present survey will provide a valuable tool for educators to assess some of the issues regarding scientific learning (especially as it pertains to evolution) specific to their unique regional populations. The relationship of the present survey to widely used epistemological measures would also be of interest to educational psychologists (e.g., Schommer-Aikins 2004). To readers who (understandably) find the survey in its present version still too long for everyday use (students tell us it takes approximately 25 minutes to complete), we are in the process of creating an informed short form in part based on the analyses presented here. Moreover, to fill in some gaps regarding causal directions, longitudinal analyses are needed. Indeed, to this end, we have already collected time 1 time 2 data on over 1,000 Kansas university students.

Regarding our educational mission, to us the message is clear: We should be cognizant not only of what we teach, but how we teach (e.g., O’Brien et al. 2009) to best educate and enthrall our students about the scientific enterprise, to illuminate evolutionary theory and its fascinating application to human affairs, and to arm them against fallacious, pseudo-scientific messages.