Measurement of Executive Functioning and High Intellectual Ability in Childhood: A Comparative Meta-Analysis

Abstract

From a neuroconstructivist approach and a developmental model of high intellectual ability (HIA), it is argued that the management of intellectual resources through executive functioning (EF) is one of the factors influencing the expression of high potential. The main objective is to determine the effectiveness of measures of executive functioning used comparing schoolchildren with HIA and those of average intelligence. A meta-analysis was carried out on a selection of 17 studies for a total sample of 1518 children with either HIA or an average level of intelligence. Pooled estimates of effect size revealed a significant difference favoring the HIA individuals in the two components of EF related with WM verbal (d = 1.015), and WM visual-spatial (d = 0.709). Other components did not show significant differences: inhibition (d = −0.014), flexibility (d = 0.068), and planification (d = −0.038). The empirical heterogeneity was very high. It is concluded that these instruments show a degree of measurement impurity, which condition their validity and reliability, and that schoolchildren with HIA display better executive functioning in the components of verbal and visual-spatial working memory.

1. Introduction

Research advances in human intelligence in the fields of genetics, neuroscience, and psychology have redefined the concept of high intellectual ability (HIA) as a high potential for cognitive functioning resulting from a complex interplay of factors, including genetic and environmental covariates. High biological potential, expressed epigenetically throughout an individual’s development, is a necessary but insufficient condition for the emergence of HIA. Each human being has a unique developmental trajectory, shaped by endogenous and exogenous modulating factors that determine how this potential is expressed in adulthood [1].

Current neurocomputational [2] and genetic [3,4] models of HIA suggest that genetics provide a biological basis, including a distinctive cytoarchitecture and differences in brain functioning. This is one predictor of HIA, but biological factors are not immutable, nor are they sufficient to guarantee that high potential will be expressed in high ability. This expression is relatively stable, because biological potential is acted upon by other factors over the course of an individual’s development, depending on his or her particular traits and contextual influences [5]. In short, HIA is not a static property hardwired in the brain but the product of an ongoing interaction with modulating factors that influence how high neurobiological potential translates into complex functions and the construction of representations that allow the brain to become more efficient.

Among the various endogenous modulating factors is the management of intellectual resources. This is one factor that may explain the gap occasionally observed between the level of competence expected based on high potential and suboptimal performance on the part of certain individuals with HIA.

Executive functions (EFs) are vital to the management of intellectual resources. In close conjunction with metacognition [6], they play a key role in our understanding of how people consciously regulate their thoughts and behaviours, and they are also associated with academic performance. Definitions of executive functions still require greater precision. There are a number of different models that together identify a heterogeneous set of high-level cognitive processes. According to certain experts [7,8], all of these processes share three core components: inhibitory control, working memory (WM), and cognitive flexibility. These core components provide a foundation for other higher-order processes, such as reasoning, problem solving, planning, and decision making [9].

There is a continuum between the two kinds of EF [10]: cool regulation, associated with cognitive processes, such as applying reasoning or information processing to the completion of tasks; and hot regulation, associated with the management of affective and emotional responses in decision making and behavioural control. The concepts of hot and cool self-regulation [11] have been similarly applied to temperament [12] and to executive attention [13], introducing new operational and relational variables for these interconnected processes.

Executive functioning is linked to the maturation and activation of the prefrontal cortex and other regions of the brain that communicate with it [14,15]. The activation and efficiency of these regions can be adversely affected by circumstances, such as loneliness, stress, sadness, or poor physical health; personal and social modulating factors, therefore, can have an impact on the effectiveness of executive functioning.

Quite apart from the debate over their unity and diversity [16], there are substantial individual differences in executive functioning. Despite the genetic basis and neural correlates of EFs, studies suggest that environmental factors give rise to significant polymorphisms in their expression, as noted earlier with respect to HIA.

1.1. Executive Functions and High Intellectual Ability

Executive functions have an impact on whether high potential is ultimately expressed as HIA [17]. Two particularly interesting questions are their role in the achievement of excellence and the relationship between executive dysfunction and poor performance. A review of the specialized literature bears out the assumption that studying executive functioning can help us understand the trajectories in which HIA is expressed [17], lending support to the current trend to conceptualise it as a process in development [5,18]. Other studies [19] have detected variations in executive functioning between people of different intellectual profiles, where high potential is inferred from a set of constituent skills, for instance, in gifted individuals or those displaying particular talents.

Consequently, it is important to ensure that measures of executive functioning—generally, behavioural rating scales or cognitive tasks—are both valid and reliable. If this is not the case, any results or conclusions that come from these studies will only add to the conceptual haziness that inhibits our understanding of the role of executive functioning in the management of intellectual resources in those with HIA and its influence on the expression (or non-expression) of potential or competence in outstanding performance.

1.2. Measures of Executive Functions

A great many neuropsychological measures of executive functioning have been put forward, but these tend to be overly broad and vague as to which cognitive processes they are intended to capture [20,21]. The lack of construct operationalisation, the complexity of the processes involved, inconsistencies in the administration of batteries and tests, and the use of a single task to evaluate different components are some of the reasons why this kind of research is so complex [22].

Although some experts [9] seek to demonstrate how the three core components relate to cognitive assessment tasks, analysing their validity with respect to what is being measured, the reality is that both the validity (functional and ecological) and the reliability of these instruments are limited. The concept of task impurity reminds us that not all tasks measure what they claim to measure and that they all too often measure other components, too [16]. This can occur when multiple components are mobilised in completing a particular task, or because of similarities with how a task would be approached in everyday life, including the involvement of non-executive processes. Moreover, there tends to be only a modest correlation between the various ways of measuring a single EF component, and internal consistency reliability and test–retest reliability are low [23], with a handful of exceptions [24].

These difficulties suggest that measures of executive functioning are weak instruments for capturing a construct that is itself, perhaps, still poorly defined and operationalised [25].

The main objective of this study is to determine whether the use of meta-analysis might help to answer some of these questions surrounding how executive functioning is measured in schoolchildren with HIA or an average level of intelligence. Our main research questions are as follows: 1. Are all measures of executive functioning found in the literature equally effective? 2. Are there differences in executive functioning between schoolchildren with HIA and those with average intelligence.

2. Materials and Methods

The first step in the meta-analysis was to search for comparative studies that sought to measure differences in executive functioning between individuals with HIA and those with an average level of intelligence, by querying a specific set of databases that were deemed a sufficient starting point.

2.1. Study Selection Criteria

To be included in the analysis, a study had to meet the following six criteria: (a) be empirical in nature; (b) include participants with HIA and those of average intelligence, with no developmental disorders; (c) encompass a participant age range of 6 to 12 years; (d) employ standardised instruments for measuring executive functions and intellectual competence; (e) be published in English or Spanish between 1998 and 2019; and (f) provide the means and standard variations of their results. Any studies that did not meet these criteria were discounted.

2.2. Identification of Qualifying Studies

The search was conducted between March and June 2019, based on: (a) electronic databases (Web of Science, Scopus, Google Scholar, PsycINFO, Medline, ERIC, and Dialnet); (b) journals that publish articles related to HIA and psychology (Journal for the Education of the Gifted, Gifted Child Quarterly, Roeper Review, International Journal of Psychological Research, International Journal of Neuroscience, and Revista de Psicopedagogía); (c) a review of the references given in each article, with the aim of identifying additional studies that could potentially be included; and (d) the TESEO doctoral thesis database.

The search terms used were: executive function(s), intellectual ability(s), cognitive ability(s), executive functioning, intelligence, executive control, executive dysfunction, giftedness, talent, high intellectual ability, high achievement, high IQ, superior students, advanced students, working memory, inhibition, and flexibility. These terms were combined with Boolean operators (AND, NOT, OR, XOR) in order to make the search more precise, selective, and defined.

The initial electronic search yielded a total of 8560 texts pertaining to the research questions outlined above. To avoid omitting any articles published between 1995 and 2019 that were not included in these electronic databases, a manual search of two of the selected journals specialising in HIA (Gifted Child Quarterly and Roeper Review) was also carried out. Once duplicates had been removed and the abstract of each study reviewed using inclusion and exclusion criteria, 24 texts remained.

All texts were found to contain the statistical data needed to establish effect size. An initial inspection of these 24 texts led to 7 being excluded, leaving a final set of 16 articles and 1 doctoral thesis.

As it was expected that the selected studies would employ more than one measure of executive functioning with the same participants, as well as different scoring methods for the same task, and given that including such measures would have violated the principle of sample independence [26], parallel meta-analyses were conducted for all EF components that appeared in the studies. There were only a small number of qualifying studies for each EF component, so the planned analysis of moderating variables was abandoned. Figure 1 illustrates the search, selection, and inclusion process followed.

Data from each selected article were coded in accordance with the following variables: (a) authors and date; (b) number of participants (with HIA and with average intelligence); (c) age range; (d) measure used to identify participants with HIA; (e) components of executive functioning; and (f) cognitive tasks used to measure each component. To ensure acceptable data quality, interobserver reliability between two independent observers was calculated using the kappa coefficient. This produced a k of 0.89, confirming the reliability of the coding.

The statistical analysis was conducted as follows.

2.3. Effect Size Calculation

We calculated the effect size of each study on each component of executive functioning in the articles analyzed. The chosen index was the standarized mean difference, or Cohen’s d [27]. This is the most suited index to measure the difference between two groups in a continuous variable. Furthermore, many studies in this area provide means and standard deviations, the statistics necessary for calculations. The difference between the group means appears in the numerator of the formula. The way which we replaced in that formula for this meta-analysis was such that positive values reflect better performance of the group with HIA over the group with an average level of intelligence. The d values were obtained using the practical meta-analysis effect size calculator [28]. The calculated values were subsequently adjusted for bias, to correct for the well-known trend of d to overestimate the parameter [29].

2.4. Statistical Methods

Statistical analysis was performed assuming a random effects model. A random (rather than fixed) effects model was chosen because this model allows for generalizing the differences in EF beyond the specific set of studies included here. Also, this model was chosen because it is a more conservative model with respect to statistical inferences, as compared to a fixed effect model [30].

The pooled effect size was obtained by weighting the individual estimates by the inverse of variance method. Furthermore, for the between studies specific variance was estimated by the method of moments [31]. Heterogeneity was assessed using the Q [32] and I² statistica [33]. The Q statistic allows testing whether the observed variance exceeds what is expected from mere sampling under a fixed parametric value. The I² statistic describes the heterogeneity in relative terms.

Statistical analysis was done through the Lipsey and Wilson SPSS macros [34] and the figures (forest plot and funnel plot) were obtained with the R package [35]. As the number of studies is relatively small, we only analized the threat of publication bias through the visual inspection of the funnel plot.

Sensitivity analysis for outliers was done by recalculating the confidence interval of the pooled effect size after eliminating them. This analysis allows to ascertain whether the decision of including them or not could potentially compromise the results of the meta-analysis.

3. Results

The final set of k = 18 estimates summed a total sample of 1518 participants, of whom 591 displayed HIA and 927 displayed an average level of intelligence. The age range of participants was between 6 and 12 years. To identify participants with HIA, 78% of studies used a formal measure of intelligence, yielding a single IQ index, whereas 11% also used formal measures with a separate score for creativity. The remainder (11%) used multiple informal measures of intellectual competence, including teachers’ nominations, parents’ questionnaires, or participation in programs aimed at pupils with HIA.

We observed a great deal of variety in the EF components evaluated and the measurement instruments used. Eleven studies are focused exclusively on a single component, whereas 6 looked at several components. Inhibition is examined in 10 reports, verbal WM in 9, visual-spatial WM in 8, planning and flexibility in 3, and decision making in 1. The tasks presented to participants are also highly varied, comprising 21 separate measurement instruments. A total of 12 tasks were used to measure inhibition, 13 for verbal WM, 8 for visual-spatial WM, 2 for planning, 2 for flexibility, and 1 for decision making. Furthermore, there were 2 studies where the same instrument was used to measure different components (flexibility and inhibition, verbal WM and inhibition). The selected studies were carried out in Europe (6), North America (5), Asia (4), and Latin America (2). Given that this meta-analysis involves studies with distinct samples, containing participants of different ages in different countries, and took a variety of approaches and data extraction, the results may be generalized.

As we can see in

Table 1. Display the features of the different studies included in this meta-analysis, the EF components evaluated, and the instruments used.

Studies	Sample HIA	Sample Average	Age Range	HIA Identification	Intelligence Measure		Components FE	Instruments
					HIA	Average
Alloway & Elsworth, 2012 [36]	44	38	9–11	NAGC WISC	M = 36.7 SD = 7.58	M = 99.79 SD = 7.04	WM verbal WM visual-spatial	Listening Recall Task Spatial Recall Task
Berg & McDonald, 2018 [37]	23	28	5–8	WIAT-II Raven	M = 121.74 SD = 10.11	M = 107.64 SD = 7.62	Inhibition WM visual-spatial WM verbal Flexibility	Colour Word Inhibition Quantity-digits Inhibition Mapping and Directions Task Visual Matrix Task Auditory Digits Sequence Task Semantic Categorisation Task Making Trail Task
Calero et al., 2007 [38]	24	23	6–11	Kaufman Brief Intelligence Test (K-BIT)	Range IQ 136–160	Range IQ 95–115	WM verbal	Yuil et al. Task
Duan et al., 2009 [39]	15	15	11–13	Raven	M = 54.60 SD = 2.29	M = 43.20 SD = 3.97	Inhibition	Go/No-go Task
Haring, 2016 [40]	27	41	10	NIO	Range NIO 42–54	Range NIO 31–38	WM visual-spatial	Lion Game
Hoard et al., 2008 [41]	46	250	5–7	Raven	M = 126 SD = 5	M = 107 SD = 8	WM verbal WM visual-spatial	Working Memory Test Battery for Children
Jhonson et al., 2003 (older) [42]	40	57	8.67–11.92	WISC-III CCAT	Percentile 97	-----	Inhibition	Stroop Trail making Spatial location
Jhonson et al., 2003 (younger) [42]	17	35	6.75–9.25	WISC-III CCAT	Percentile 99	------	Inhibition	Stroop Trail making Spatial location
Khosravi et al., 2016 [43]	24	24	12	WISC	Range IQ 110–130	Range IQ 90–110	WM verbal	Digits (WISC)
Kornmann et al., 2015 [44]	42	39	8–12	Teachers’ Giftedness nominations	M = 112.26 SD = 11.68	M = 11.95 SD = 14.85	WM verbal WM visual-spatial	Listening Span Task N back task Spatial span
Li et al., 2017 [45]	98	125	7–12	Cattell´s Culture Fair Intelligence Test	M = 38.43 SD = 3.26	M = 27.99 SD = 6.76	Decision making	IOWA Gambling Task
Liu et al., 2011,a [46]	20	21	12.6–13.4	WISC Creativity Observation Cattell´s Culture Fair Intelligence Test	M = 43.2 SD = 1.7	M = 34.5 SD = 1.9	Inhibition	Go-no go
Liu et al., 2011,b [47]	15	13	8.6–10.5	Standford–Binet Intelligence test Creativity Observation Cattell´s Culture Fair Intelligence Test	M = 47.6 SD = 5.2	M = 39.8 SD = 1.3	Inhibition	Go-no go
Montoya-Arenas et al., 2010 [48]	32	30	7–11	WISC-III	M = 138.6 SD = 1.3	M = 101.8 SD = 5.7	Flexibility Inhibition WM verbal WM visual-spatial Planning	WCST Stroop Oral Word Fluency Semantic/Phonological Desing Fluency Test Tower of Hanoi
Montoya-Arenas et al., 2018 [49]	32	43	7–11	WISC-III	M = 138.6 SD = 1.3	M = 104.5 SD = 6.7	Flexibility Inhibition WM verbal WM visual-spatial Planning	WCST Stroop Oral Word Fluency Semantic/Phonological Desing Fluency Test Tower of Hanoi
Swanson, 2006 [50]	50	77	6–9	WIAT WRAT-III	M = 126.78 SD = 5.25	M = 99.21 SD = 8.22	WM verbal WM visual-spatial Inhibition	Backward Digit span (WSIC) Listening/Sentence span Digit/Sentence span Updating Visual Matrix Mapping Randon Letter Generation Randon Number Generation Catagorical Fluency Letter Fluency
Urben et al., 2018 [51]	18	20	12	WISC	IQ > 125	IQ < 95	Inhibition	Stop-Signal Task
Vogelaar et al., 2019 [52]	24	48	9–11	Teachers and parents´ Gifted Nominations Raven	M = 47.58 SD = 5.04	M = 39.63 SD = 6.59	Planning	Tower of London

HIA = High Intellectual Ability; WISC-III = Wechsler Intelligence Scale for Children; CCAT = Canadian Cognitive Abilities Test; WIAT = Wechsler Individual Achievement Test; WRAT-III = Wite Range Achievement Test; NAGC = National Association for Gifted Children; NIO = Nederlandse Intelligentietest voor Onderwijsniveau; WAIS = Wechsler Abbreviated Scales of Intelligence.

, there is a great dispersion in the measurements, which suggests a high degree of impurity. Furthermore, studies using multiple measures for a single EF component reported that there was no correlation between different measures.

Figure 2 shows the forest plot with the 18 estimates. To avoid dependence problems, we have included only one estimate from each study. When the study reported more than one, we choose the most representative core components of EF. The component represented for each study is also specified in a separate column in the figure. It is obvious that there is substantial variability between the studies, with values ranging from d = 2.97 to d = −1.04. Visual inspection revealed that heterogeneity was much greater in the earlier than in more recent studies. To highlight this effect, we have ordered the studies in the figure according to the year of publication. Studies published since 2015 show less heterogeneity, with component effect sizes ranging from d = 0.95 to d = −0.21.

The heterogeneity analysis yielded a significant value [Q (17) = 193.8990, p < 0.01; I² = 91.23; τ² = 0.63374], indicating diversity across the studies. This was to be expected since dependent variables that assess the different components of EF are included in this set. We have estimated the combined effect size for each of the EF components studied. This involved 10 estimates for the component of inhibition, 9 for verbal WM, 8 for visual-spatial WM, 3 for flexibility, and 3 for planning. Decision-making was excluded, as only one study sought to measure this component.

Table 2. Combined effects for the set of studies reporting estimates of differences in inhibition, verbal WM, visual-spatial WM, flexibility, and planning. For each case they are also included the Q statistic and the specific variance estimate (τ²).

Component	k	d	CI 95%		Q (p)	τ2
			UL	LL
Inhibition	10	−0.014	−0.342	0.314	33.609 (0.0001)	0.1984
WM verbal	9	1.015	0.602	1.428	53.153 (0.000)	0.3294
WM visual-spatial	8	0.709	0.270	1.147	53.535 (0.0000)	0.3427
Flexibility	3	0.068	−0.220	0.356	0.777 (0.6781)	0.0000
Planification	3	−0.038	−0.310	0.242	0.685 (0.7100)	0.0000

k: number of studies; d: effect size; WM: working memory; CI: confidence interval; τ²: specific variance estimate.

shows the pooled estimates and the heterogeneity statistics.

The effect sizes vary widely between components. A statistically significant effect was observed in two components, verbal WM (d = 1.0149; CI: 0.6018–1.4278) and visual-spatial WM (d = 0.7085; CI: 0.2703–1.1468). Comparing the two groups, participants with HIA display better performance in verbal WM and visual-spatial WM tasks. There are no significant differences in the other components: inhibition, flexibility, and planning. The high level of heterogeneity in the inhibition and WM components suggests the need for an analysis of the moderating variables, but they were not carried out here due to the small number of studies (and the corresponding low power).

Visual inspection of the funnel plots of the WM components (as shown in Figure 3) suggest that certain results are at odds with the majority and so can be considered as potential outliers (2 in the verbal set and 1 in the visual-spatial set). They are the points lying beyond the right tail of each plot.

As the inclusion of a small number of outlying studies could produce an artificial effect in the results, a sensitivity analysis was carried out in order to evaluate their potential distortive effect. After eliminating the two outlying studies on verbal WM, the estimated value d = 1.0149 (CI 95%: 0.6018–1.4278) became d = 0.6790 (CI 95%: 0.5139–0.8442), indicating a smaller but statistically significant effect.

After eliminating the only outlying study on visual-spatial WM, the estimated value d = 0.7085 (CI 95%: 0.2703–1.1467) became d = 0.4917 (CI 95%: 0.2164–0.7670); again, the effect is smaller but still statistically significant. As a result, the sensitivity analysis done after visual inspection of the funnel plots reinforce the conclusions. The existence of robust differences in the WM components does not depend on the inclusion of studies with rare results.

4. Discussion

Executive functioning is linked to the management of intellectual resources in HIA, in keeping with its current definition as a process in development [5]. Executive dysfunctions may explain the substantial variability in the expression of high potential, as other authors have suggested [53]. This may be one of the factors that can lead to a gap between high potential and performance. It is important to have access to pure, valid, and reliable measures of executive functions in order to understand their role in the management of extensive intellectual resources and in the wide variability in the expression of high potential, so that the appropriate educational approach can be adopted.

This analysis sought to generate new insights into this question. The meta-analysis described above met the overall objective and provided answers to the main research questions. In terms of the effectiveness of the measures used in this set of specialist studies, the findings reported here corroborate those of other authors who have raised the problem of measurement impurity [6,9,16]. The effect size values obtained indicate a certain level of dispersion in the measures, both among schoolchildren with HIA and those with an average level of intelligence. The high values yielded by the heterogeneity analysis cast doubt on whether these tests are truly measuring the aspect they were designed to measure. This heterogeneity, although significant, has decreased in recent years. However, this does not necessarily mean that the reliability of EF measures can now be taken for granted.

A qualitative analysis of the selected studies shows that heterogeneity can refer to: (a) the use of different tasks to measure the same component, (b) the use of a single task to measure different components, and (c) the use of different tasks to measure the same component without correlation between the scores for each task. These points call into question the validity of the construct, as well as the reliability and validity of the measures used, as other researchers have concluded [9,16,20,21,54].

The EF components evaluated in the selected studies can be understood to comprise the three core components [8,9], plus planning and, in one study, decision making. In other words, there is also dispersion in terms of the construct measured.

This heterogeneity raises serious concerns about the effectiveness of current measures of executive functions, and it introduces some doubt as to whether they can be used reliably as an indicator of resource management in individuals with HIA. It is not clear precisely what they are measuring or if there are other non-executive components also captured by the same instrument.

These results must be born in mind when considering whether the two populations do indeed display differences in executive functioning. Based on this study, it can be cautiously concluded that there are statistically significant differences in the verbal WM component and the visual-spatial WM component between participants with HIA and those with an average level of intelligence, but not in the other EF components studied.

In interpreting these results, one should be mindful that, in most cases (89%), HIA is identified using tests that return an IQ index or using informal instruments. Only 11% of studies adopted a formal multidimensional measure, as current thinking recommends [18]. Again, this may affect the validity of conclusions about the link between EF and HIA.

This meta-analysis has a number of limitations, one of them being the small number of comparative studies included, which ruled out any analysis of moderating variables, of the power of the contrast, or the power of publication bias. Even so, results obtained suggest that further research is required to develop pure measures of executive functioning that can improve our understanding of the role of executive management in the optimal expression of high potential. In this way, both individuals and societies can avoid the losses represented by unrealised potential.