Convergent and Ecological Validity of Mother and Child Reports of Children’s Depressive Symptoms: Evidence from a Diverse Sample of Mother–Child Dyads

Abstract

This study investigated the convergent and ecological validity of a multi-informant approach through retrospective measures and ecological momentary assessment (EMA) from mother–child dyads. In 202 mother–child dyads, mothers completed parent-proxy retrospective measures of their child’s depressive symptoms and their own depressive symptoms. Children completed self-report retrospective measures of their own depressive symptoms, self-esteem, and sleep quality; and self-report EMA of affect across 8 days. Results showed that parent-proxy and child self-report retrospective measures of depressive symptoms were weakly positively correlated (r = 0.23); this association was stronger for non-Hispanic mothers compared to Hispanic mothers (p = 0.048). Parent-proxy retrospective measures were moderately positively associated with the mothers’ own retrospectively reported level of depressive symptoms (r = 0.33). Parent-proxy and child self-report retrospective measures were negatively associated with the children’s average EMA happiness (p = 0.001 and p = 0.003, respectively), but only the children’s retrospective measures were positively associated with EMA sadness (p = 0.001). In multivariable models, the children’s retrospectively reported depressive symptoms were significantly associated with lower self-esteem and reduced sleep quality, while parent-proxy retrospective reports were only associated with the mother’s own depressive symptoms. Overall, the current study provides evidence for the convergent and ecological validity of children’s self-reports of depressive symptoms. However, there was limited validity for parent-proxy reports, and parent-proxy reports may be highly influenced by the mothers’ depression.

1. Introduction

Depressive symptoms in childhood are associated with a range of negative outcomes (e.g., alcohol use, disordered eating, obesity) [1,2]. However, they are subject to a number of measurement challenges in comparison to depressive symptoms in adulthood [3]. Therefore, in order to appropriately assess and increase our understanding of depressive symptoms in children, it is essential to utilize reliable and valid measures of depressive symptoms. Many studies use a multi-informant approach to assess depressive symptoms in children, which involves having multiple observers report on depressive symptoms (e.g., child, parents, teachers) [4]. Most commonly, studies use children and parents as informants. A recent meta-analysis found that child and parent reports of internalizing symptoms (e.g., summary score of anxiety, depression, and related symptoms) were only moderately related to each other (r = 0.26) [4], thus indicating that these measures are assessing different components of internalizing symptoms, including depressive symptoms; although, it is unclear what exactly these components are.

Child self-report and parent-proxy reports of depressive symptoms offer different strengths and limitations. Although subject to retrospective recall biases, evidence suggests that adults can report on their own symptoms with an acceptable level of reliability and validity [5]. This may extend to some degree to children; however, children typically have more difficulty recalling information and understanding various psychological concepts (e.g., feeling worthless, feeling restless), which may limit the adequacy of their self-reports of depressive symptoms [6]. Having parents provide reports of their child’s depressive symptoms is one way to alleviate this limitation. Yet, there are a number of concerns with parent reports. Specifically, parents are not always present with their children, and thus depressive symptoms may go unnoticed in children. Additionally, the parent’s own depression may affect their reports of their child’s depression [7,8]. Finally, parents may not adequately identify, or may misunderstand, their child’s depressive symptoms, whereas children’s reports are based on their own experience and thus may be more accurate.

To improve the assessment of children’s depressive symptoms, this study was guided by several research questions that address the ecological and convergent validity of parent-proxy and self-report of child depressive symptoms.

First, how are parent-proxy reports of child depressive symptoms associated with children’s self-report of depressive symptoms? It is expected that there will be a weak correlation between these measures, given recent meta-analytic findings [4].

Second, how are parent-proxy retrospective reports of children’s depressive symptoms associated with parents’ retrospective self-report of depressive symptoms? Consistent with previous research [7,8], it is expected that these measures will be moderately correlated.

Third, how are parent-proxy and children’s retrospective self-reports of depressive symptoms associated with real-time reports of depressive affect (i.e., decreased happiness and increased sadness) in the natural environment via ecological momentary assessment (EMA; i.e., a test of ecological validity)? Ecological validity refers to the extent that a measure is associated with related symptoms in individuals’ daily lives [9]. EMA is a real-time data capture methodology that uses repeated measures to collect information from individuals throughout the course of their daily lives [10]; associating EMA data with self-report questionnaires will allow for the examination of ecological validity. It is expected that children’s retrospective self-report of depressive symptoms will be more strongly associated with EMA-reported depressive symptoms (i.e., happiness, sadness) compared to parent-proxy retrospective report.

Fourth, do related child retrospective self-report measures (i.e., sleep, stress, and self-esteem) predict children’s retrospective self-report and parent-proxy report of depressive symptoms co-varying for the other dyad member’s report of depressive symptoms and the mother’s retrospective self-report of depressive symptoms (i.e., a test of convergent validity)? Convergent validity refers to the extent to which a measure is related to similar constructs [11]. Sleep, stress, and self-esteem are useful indicators of the convergent validity of depressive symptom measures, as they are important components of depression [12]. There were no specific hypotheses for this research question.

Finally, given the diverse sample with regard to ethnicity, we examined ethnicity (i.e., Hispanic versus non-Hispanic) as a moderator in all study aims.

2. Method

Participants and Procedure

The current study used baseline data from 202 mother–child dyads enrolled in the Mothers and their Children’s Health (MATCH) study, which included an ethnically diverse group of mothers and children recruited in Southern California schools through flyers distributed to classrooms and information booths at school and community organized events. Children (51% girls) were, on average, 9.60 years old (SD = 0.92; range = 8–12), and mothers were, on average, 40.98 years old (SD = 6.13; range = 26–57). Forty-nine percent of mothers and 54% of children were Latinx. Most mothers were married (67.7%), followed by never married (14.3%), divorced (10.0%), separated (7.5%), and widowed (0.5%). Most mothers worked full-time (57.6%), followed by part-time (25.8%), and not employed, retired, a student, or other (16.6%). The median household income was between $65,000–74,999. Informed consent was obtained from parents, and assent was obtained from children for all participants included in the study. At baseline, both mothers and children completed self-report questionnaires and eight days of EMA via mobile phones. The Institutional Review Board at the University of Southern California approved this study (#HS-12-00446).

3. Measures

3.1. Measures Completed by Mothers

Parent-proxy measure of child depressive symptoms. The Diagnostic and Statistical Manual (DSM)-oriented Depressive Problems subscale of the Child Behavior Checklist (CBCL) [13] was used as the parent-proxy measure of a child’s depressive symptoms in the current study. The CBCL has been shown to be a reliable and valid retrospective measure of children’s internalizing symptoms across various cultures [13,14], and a psychometrically sound measure of DSM-oriented diagnoses, including depression, in English and Spanish samples [15,16]. In our sample, the Cronbach’s alpha was 0.65, slightly below the recommended 0.70. Removing the items “sleeping trouble” and “doesn’t eat well” improved the alpha to 0.70. Analyses were run using the modified scale. The remaining items were averaged, with higher scores meaning more depressive symptoms.

Mothers’ self-reported depressive symptoms. The Center for Epidemiological Studies—Depression scale (CES-D) [17] is a 20-item self-report assessment of depressive symptoms, including restless sleep, poor appetite, and feeling lonely. Response options range from 0 (rarely or none of the time) to 3 (most or almost all the time). Scores range from 0–60, with higher scores indicating greater depressive symptoms. The CES-D has adequate psychometric properties and good sensitivity and specificity for identifying clinical depression [18]. The Cronbach’s alpha was 0.93.

3.2. Measures Completed by Children

Child self-report measure of child depressive symptoms. The Revised Children’s Anxiety and Depression Scale (RCADS) [19] is a self-report retrospective measure assessing affective symptoms. The Major Depression subscale (10 items), used in the current study, assesses a range of depressive symptoms, such as feeling sad or empty, trouble sleeping, and a lack of energy. Children responded to items on a four-point scale ranging from 0 (never) to 3 (always). Items were averaged, with higher scores indicating more depressive symptoms. The RCADS has been shown to have good reliability and validity [20]. The Cronbach’s alpha in the current study was 0.81.

Life events. The Child and Adolescent Survey of Experiences (CASE) [21] is a self-report measure of positive and negative life events; only the negative life events score was used for the current analyses. Children indicated whether 38 different events had occurred in the past six months. Examples of events included moving, illness, being bullied, divorce, and making a new friend. Participants used a scale from 0 (did not happen) to 6 (really bad) to rate each item. Items were summed, with higher scores indicating more life stress. The Cronbach’s alpha in the current study was 0.88.

Sleep quality. The Jenkins Sleep Scale (JSS) [22] assessed children’s sleep quality with four items (i.e., trouble falling asleep, waking up at night, trouble staying asleep, and feeling tired after waking). Children responded to items on a rating scale from 0 to 5 as to how often each symptom occurred over the past month, from 0 (not at all) to 5 (22–31 days). Items were summed, with higher scores indicating more sleep difficulties. Research suggests that the JSS has adequate reliability and validity in adults [23]. The Cronbach’s alpha in the current study was 0.75.

Self-esteem. The Rosenberg Self-Esteem Scale (RSES) [24] assessed global self-esteem through 10 items, with response options ranging from 1 (strongly disagree) to 4 (strongly agree). Items were summed, with higher scores indicating better self-esteem. The RSES has excellent psychometric properties [25]. The Cronbach’s alpha in the current study was 0.83.

EMA of depressive affect. Children received EMA survey prompts for seven days. Prompts were generated in stratified random sampling windows, with one prompt randomly occurring during each window. Weekdays included three prompts after school (i.e., 3–4 p.m., 5–6 p.m., and 7–8 p.m.); Weekends included seven prompts (i.e., 7–8 a.m., 9–10 a.m., 11 a.m.–12 p.m., 1–2 p.m., 3–4 p.m., 5–6 p.m., and 7–8 p.m.). The survey required approximately 2–3 min to complete, and if surveys were prompted during incompatible times, participants had the option to delay. Up to two reminder prompts were initiated within 10 min of the initial prompt, after which time the survey was closed. Sadness and happiness were measured with two items: “Right before the phone went off, how (1) sad and (2) happy were you feeling?”. Affect items were each presented to the participant on a unique screen, and the response options consisted of a four-point scale ranging from 1 (not at all) to 4 (extremely). EMA sadness and happiness reports were aggregated into a mean score representing the child’s average level of sadness and happiness.

Pubertal development. The Pubertal Development Scale (PDS) [26] was used as a self-report measure of pubertal development. The PDS assesses aspects of pubertal development. Girls were asked four questions regarding growth spurts, body hair, changes in skin, and breast growth, and boys were asked about body growth, body hair, skin changes, voice changes, and beard growth. Children responded using a 4-point scale ranging from 0 (not yet started), 1 (recently started), 2 (started a while ago), to 3 (already completed). This PDS demonstrates adequate psychometric properties [26].

4. Statistical Analyses

Analyses were conducted in SPSS version 24. Descriptive statistics were calculated for the study variables, and the inspection of histograms was conducted to screen for normality. For the first three research questions, bivariate correlations were used to evaluate the hypotheses. For the fourth research question, two general linear models (GLMs), which are flexible analysis methods with regard to normality violations, were calculated. The GLMs used the parent-proxy measure (i.e., CBCL scores) and the child retrospective self-report measure (i.e., RCADS scores) as dependent variables. Independent variables included: the opposite dyad member report of the child’s depressive symptoms and the mother self-report of depressive symptoms; demographic covariates (i.e., gender, age, and pubertal status); and the child self-report of sleep quality, self-esteem, and stress. Exploratory analyses were conducted for all aims to examine for interactions with the mother’s ethnicity (Hispanic versus non-Hispanic). The significance level was set at p < 0.05.

5. Results

Descriptive statistics and bivariate correlations between the study variables are displayed in

Table 1. Descriptive statistics among study variables.

	1	2	3	4	5	6	7	8
1. Child self-report of depressive sx	-	0.23 **	0.13	0.24 **	−0.23 **	0.59 ***	−0.64 ***	0.10
2. Parent-proxy report of depressive sx		-	0.33 ***	0.13	−0.29 **	0.12	−0.26 ***	0.17 *
3. Mother self-report of depressive sx			-	0.13	−0.04	0.03	−0.22 **	0.24 **
4. EMA sadness				-	−0.19 **	0.18 *	−0.14	0.12
5. EMA happiness					-	−0.15	0.14	−0.16 *
6. Child sleep quality						-	−0.42 ***	0.11
7. Child self-esteem							-	−0.11
8. Child stress								-
M	0.53	1.00	7.80	1.20	3.07	3.56	31.88	8.32
SD	0.46	0.16	7.94	0.27	0.66	3.88	5.31	9.22
Minimum	0.00	0.20	0.00	1.00	1.00	0.00	13.00	0.00
Maximum	2.40	2.00	54.00	2.40	4.00	20.00	40.00	68.00
Skewness	1.42	1.66	2.05	2.23	−0.59	1.82	−0.78	2.44
Kurtosis	2.80	11.48	6.74	5.79	−0.20	3.73	0.64	9.66

Note. This table includes analyses for research questions 1–3; sx = symptoms; EMA = ecological momentary assessment. *** p < 0.001, ** p < 0.01, * p < 0.05.

. The child self-report, parent-proxy report, and mother self-report of depressive symptoms were all non-normal. The child self-report of depressive symptoms and parent-proxy reports were weakly, yet significantly positively, correlated, r = 0.23, p < 0.01. Parent-proxy reports were moderately positively correlated with the mother self-report of depressive symptoms, r = 0.33, p < 0.001; and the child self-report of depressive symptoms was unrelated to the mother self-report of depressive symptoms, r = 0.13, p > 0.05. The child self-report of depressive symptoms was significantly positively related to ecological momentary assessed sadness (r = 0.24, p < 0.01), and negatively related to ecological momentary assessed happiness (r = −0.23, p < 0.01). Parent-proxy reports were unrelated to ecological momentary assessed sadness (r = 0.13, p > 0.05), but significantly negatively related to ecological momentary assessed happiness (r = −0.29, p < 0.01).

In the first GLM (see

Table 2. General linear model of children’s self-report of depressive symptoms.

	B	SE	CI	p
Parent-proxy report of child’s depressive sx	0.02	0.18	[−1.27, 2.82]	0.72
Mother’s depressive sx	0.01	0.01	[−0.01, 0.02]	0.36
Mother’s ethnicity	0.08	0.09	[−0.11, 0.26]	0.41
Age	0.01	0.06	[−0.10, 0.12]	0.89
Gender	−0.08	0.10	[−0.27, 0.11]	0.42
Pubertal status	0.08	0.08	[−0.07, 0.23]	0.30
Child’s stress	0.01	0.01	[−0.01, 0.02]	0.32
Child’s sleep quality	0.05	0.01	[0.03, 0.08]	<0.001
Child’s self-esteem	−0.06	0.01	[−0.08, −0.04]	<0.001

Note. This table includes analyses for research question 4; sx = symptoms; B = beta; SE = standard error; CI = confidence interval.

), a gamma function was used to account for the non-normality of the data. The child self-report of depressive symptoms was associated with lower self-esteem and reduced sleep quality only (ps < 0.001). Parent-proxy reports and mother self-reports, demographic variables, and child self-report of stress were unrelated to child self-report of depressive symptoms. In the second GLM (see

Table 3. General linear model of parent-proxy report of child depressive symptoms.

	B	SE	CI	p
Child’s self-report of depressive sx	0.04	0.33	[−0.61, 0.68]	0.92
Mother’s depressive sx	0.04	0.02	[0.01, 0.07]	0.01
Mother’s ethnicity	−0.20	0.22	[−0.64, 0.23]	0.37
Age	0.07	0.13	[−0.18, 0.33]	0.57
Gender	−0.02	0.23	[−0.47, 0.43]	0.93
Pubertal status	−0.09	0.18	[−0.45, 0.26]	0.62
Child’s stress	0.001	0.01	[−0.02, 0.02]	0.93
Child’s sleep quality	0.01	0.03	[−0.05, 0.07]	0.69
Child’s self-esteem	−0.03	0.03	[−0.09, 0.02]	0.28

Note. This table includes analyses for research question 4; sx = symptoms; B = beta; SE = standard error; CI = confidence interval.

), a negative binomial function was used to account for the non-normality of the data. The parent-proxy report of child depressive symptoms was only associated with the mother self-report of depressive symptoms (p = 0.01); parent-proxy reports were unrelated to all other variables in the model.

Exploratory analyses revealed an interaction between the mother’s ethnicity and the child’s report of depressive symptoms, predicting the mother’s report of their child’s depressive symptoms, B = 1.18, SE = 0.59, p = 0.048. Follow-up stratified correlation analyses revealed a significant correlation between depressive symptom reports for non-Hispanic mothers (r = 0.39, p < 0.001), yet no significant correlation among Hispanic mothers, (r = 0.09, p = 0.39). Interactions were non-significant for the other aims.

6. Discussion

The current study examined the validity of the multi-informant approach to assessing children’s depressive symptoms using data from a diverse sample of mother–child dyads. Consistent with a recent meta-analysis [4], the parent-proxy report and child self-report of depressive symptoms were only weakly associated. Although there is some overlap, this correlation suggests that retrospective measures completed by mothers and children are not assessing the construct in the same way, which may reflect differences between the parent-proxy versus self-report measure or RCADS. The RCADS and CBCL assess many similar depressive symptoms (e.g., sadness, sleep) but several different symptoms (e.g., guilt, suicidal thoughts). Our findings, as well as those of previous studies [7,8], suggest that parent-proxy reports may be affected by the parent’s own level of depressive symptoms. Further, research has found that there may be a sizable number of children whose depressive symptoms go unnoticed by their parents [27]. Also, some research indicates that the parent–child discrepancy in the symptom score may be an important factor associated with depression course and treatment [28].

It is widely known that self-report questionnaires are highly susceptible to retrospective recall biases; parent-proxy reports of child depressive symptoms may reflect a recall bias where parents have difficulty separating their own and their child’s experiences. Further, exploratory analyses showed that there was only concordance between a non-Hispanic mother’s reports of depressive symptoms and their child’s report of their own depressive symptoms. Factors that may play a role in explaining this lack of concordance in Hispanic mothers may include acculturation, parent–child culture and language discordance, defined family roles, and parenting styles [29], but more research is needed to elucidate the reasons for this lack of concordance.

Both parent-proxy reports and child retrospective self-reports of depressive symptoms were similarly associated with happiness in the natural environment, measured with EMA, such that higher depressive symptoms were associated with less happiness. The RCADS and CBCL may be similarly assessing the anhedonia (i.e., loss of pleasure and positive affect) [30] facet of depressive symptoms. However, only children’s self-reports of depressive symptoms were related to real-time reports of sadness in the natural environment. Therefore, child self-reports (specifically the RCADS) appear to have more ecological validity with regard to a wider range of depressive affect.

In multivariable models, parent-proxy reports of child depressive symptoms were only associated with the mother’s own depressive symptoms, and child retrospective self-reports of depressive symptoms were associated with their child self-reports of lower self-esteem and reduced sleep quality. These models provide evidence for the convergent validity of child retrospective self-reports of depressive symptoms, as poor self-esteem and sleep problems are two of the core components of depression [31]. However, results suggest that parent-proxy reports lack construct validity and are most associated with their own reports of depressive symptoms.

The strengths of this study include a diverse sample with regard to race and ethnicity. In addition, a strength of this study was the use of both self-report surveys and EMA to study both convergent validity and ecological validity of depressive symptom measures. There are several limitations worth noting. This study used data from a community-based sample, and thus, children had relatively low levels of depressive symptoms. Also, this study did not compare self-report and parent-proxy measures to clinical assessments of depression. More research is needed to determine if these results are similar in clinical samples and if the measures are related to clinical assessments.

There are several important avenues for future research. First, research is needed to determine if the findings reflect parent-proxy report versus child self-report or use of the RCADS versus the CBCL. Perhaps parent-proxy report using the RCADS may have better convergent and ecological validity. Second, future studies should examine the utility of using more proximal reports from parents on their child’s depressive symptoms using EMA, which would be less affected by recall biases. Finally, future studies should examine the predictive validity of parent-proxy and child self-report of depressive symptoms by examining the utility of the scores on these measures in predicting negative child mental and physical health outcomes longitudinally.

In a community-based study, our results show that the child retrospective self-report of depressive symptoms has the strongest evidence for validity in terms of convergent and ecological validity. Oppositely, evidence for the validity of parent-proxy reports of child depressive symptoms was rather weak. Therefore, these findings provide support for the use of child self-report of depressive symptoms for assessment in research and clinical practice.