How do Runaway Adolescents and their Parents Perceive the Family? Measurement Invariance in the Family Functioning Scale

School of Social Work, University of Texas at Austin, Austin, Texas, USA

*Corresponding Author:

Sanna J Thompson
School of Social Work
The University of Texas at Austin
1925 San Jacinto, Austin, Texas 78712, USA
Tel: 512-232-0604
E-mail: SannaThompson@utexas.edu

Received Date: October 23, 2013; Accepted Date: November 18, 2013; Published Date: November 24, 2013

Citation: Thompson SJ (2013) How do Runaway Adolescents and their Parents Perceive the Family? Measurement Invariance in the Family Functioning Scale. J Child Adolesc Behav 1:117. doi: 10.4172/2375-4494.1000117

Copyright: © 2013 Thompson SJ. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Child and Adolescent Behavior

Abstract

Abstract
Exploring perceptual discrepancies and how they affect family life may be particularly relevant when household
dynamics have broken down to the point that a child runs away or is forced to leave the home. This study explored
the sources of differences between parents’ and youths’ reports of family functioning and, more generally, illustrates
how to perform confirmatory factor analysis methods to purge error due to measurement invariance. Basic concepts
and methods that are often needed in social work research are described. Creating uni-dimensional and configurable
invariant subscales resulted in improved significance for the Conflict subscale over traditional methods. All other
subscales remained significantly different between parents and youths at all levels of invariance. Findings highlight
the caution required in using any scale to measure the same construct for two different populations, such as parents
and children.

Keywords

Runaway youth; Measurement invariance; Family functioning; Perceptual differences; Confirmatory factor analysis; Parent-youth dyads

Introduction

Research exploring parent and adolescent relationships in normative and clinical populations suggests that family members often have divergent perspectives [1]. Parents who have invested time and energy in maintaining integrated family relationships are more likely to report fewer family difficulties, whereas adolescents striving for independence and increased separation from parents are more apt to have unenthusiastic opinions of family life and relationships [2,3]. Although these disparate perspectives of family functioning may be typical, they can lead to family discord, especially when household dynamics are highly conflictual and disorganized. Among families encountering significant stressors, divergent perspectives may be more pronounced [4].

Exploring perceptual discrepancies and how they affect family life may be particularly relevant when household dynamics have broken down to the point that a child runs away or is forced to leave the home. Runaway youths report high rates of family disorganization, ineffective parenting, parental deprivation, family conflict, parental abuse, criminality, and substance abuse [5,6]. However, most of what is known concerning these family situations is based solely on reports by the youth as information from parents is hampered by difficulties in identifying, accessing, and gaining their consent [7,8]. Runaway youths also are reluctant to permit contact with their parents, perhaps out of fear of being reported to authorities and forced to return to abusive homes [6,9]. Once contacted, parents are often unwilling to discuss family situations, likely due to their own frustrations concerning the decaying connections with their child [7].

The few studies that have queried both runaway youth and their parents have noted a significant parent-child divergence of opinions concerning the family environment [6-8]. Compared to their runaway adolescent children, parents perceive more positive family functioning and less negativity and distress concerning family life, but also report a greater number of youths’ behavior problems. Thus, parents and adolescents have unique and often disparate views of relationships and functioning within the family. Previous research, however, has not yet explored whether parent-child differences in evaluation of family functioning are based on quantitative differences (i.e., disagreement about the family’s score on a scale of family functioning) or qualitative differences (i.e., a divergent understanding of the meaning of family functioning as a construct).

To capture differences between parents and adolescent children on a particular scale, researchers typically average scale items to create separate scale means for parents and youths, and then proceed with further analysis. This approach yields quantitative mean differences between groups, but these differences are valid only under the assumption that both groups understand the scale questions similarly. This measurement equality/invariance assumption holds when the mean difference between two groups reflects true individual differences in the measured trait, rather than discrepancies due to perception differences or measurement error [10]. As parents and youths’ understanding of specific scale items diverge, this assumption is likely to be violated, leading to potentially inaccurate estimates of quantitative group differences. Thus, only through demonstrating measurement invariance can a scale be deemed to measure the same attributes or construct across groups [11].

This study aimed to explore the sources of differences between parent and youth reports of family functioning and illustrate how to perform confirmatory factor analysis methods to purge error due to measurement invariance. Basic concepts and methods that are apt to be most useful to the applied social work researcher are described.

Methods

Study sites and sample

Between September 1999 and August 2001, youths were recruited for participation in a larger study to evaluate runaway adolescents admitted to two service agencies: emergency youth shelter and juvenile detention center. Parents of the youths were also contacted and recruited to participate. The two agencies were located in a mid-sized urban city in Western New York and served similarly-aged youth.

Emergency youth shelter: Within 48 hours of admission, the shelter contacted each youth’s parent or guardian, sought parental consent for their teen’s participation, and recruited the parent for participation in an interview similar to that of their son/daughter. Upon parents’ consent, youths were approached and recruited for participation. A convenience sample of 156 adolescents and 64 parents completed several brief self-report measures. Only youths whose parents completed the measures were included in the current analysis (n=64).

Juvenile detention center: Study participants were drawn from consecutive youth entrants to the county juvenile detention center. Although this facility provided residential guardianship to youth who had committed a criminal or status offense, only those youths 12 to 17 years of age who were admitted for non-criminal offenses and reported a runaway episode during the previous six months were recruited. Parents of these youths were recruited to participate in a similar interview as their child. 121 youths met inclusion criteria and 101 of their parents agreed to participate. Only youth-parent dyads were included in this analysis (n=101).

Measurement

The Family Functioning Scale (FFS) provided the core measurement of parent and youths perceptions for the current analysis [12]. This instrument was designed to measure general dimensions of family functioning and is based on an eclectic, integrative view of the functioning of families by incorporating five family functioning domains: Family Rituals, Communication, Positive Family Affect, Conflict, and Relationship Worries. The 40-item scale is scored on a 7-point Likert scale (1=never to 7=always) and summed for the five subscales. This scale was chosen due to its ease of availability, no cost, and acceptable reliability [13]. See Table 1 for items and subscales. The Family Functioning Scale has been used in several studies, including previous analyses of the larger study [14,15].

Analysis strategy

Initial analyses determined the reliability of each subscale and compared parents and youths’ means in the standard fashion (i.e., ignoring potential measurement invariance). Next, measurement invariance tests were conducted to determine whether parents’ and youths’ reports assessed the same construct/domain of family functioning in a qualitatively similar fashion. Measurement invariance is typically assessed using factor analysis. Early studies testing the measurement invariance assumption used exploratory factor analysis (EFA) to create a factor solution separately for each group. This strategy, however, has two disadvantages. First, EFA cannot be used to test pre-existing hypotheses regarding the number of factors or the particular items that load on each factor; second, it cannot explicitly compare factor solutions between two groups. Given the drawbacks of EFA, confirmatory factor analysis (CFA) has been increasingly used as it offers a more rigorous and hypothesis-driven investigation of measurement invariance [10]. Using structural equation modeling in AMOS 7.0, CFA assessed measurement invariance on three different levels with increasing rigor [16].

The least stringent level, configural invariance, requires that the two groups (parents and youths) have the same number of factors with the same items loading on each factor [10]. When configural invariance is met, it indicates that the same basic concept is being measured in each group; however, because this level of invariance does not guarantee that the measurement of the latent variable is the same for each group, comparisons between groups are not possible.

The next level, metric invariance requires that unstandardized factor loadings on corresponding items are equal for both groups [17]. This level of invariance provides evidence that parents and youths interpret scale items similarly; that is, the construct is measured in the same way for both groups and score differences reflect actual responses not attributable to measurement error.

The third and most rigorous level, scalar invariance assesses whether both the factor loadings and intercepts for corresponding items are equal between groups [18]. Scalar invariance implies that the two groups do not have differing response biases for scale items; thus, the measure may be used to assess cross-group mean differences on observed scores. The current analysis illustrates the steps involved in testing each invariance level and demonstrates the degree of discrepancy between parents’ and youths’ reports affected by the lack of configural, metric, or scalar invariance. The purging of items that indicated qualitative differences due to measurement invariance were completed, with discrepancies between results of the initial and final analysis identified.

Analyses and Results

Traditional comparison of means

As this sample included paired groups, reliability was calculated separately for youths and parents [19]. Results indicated reliabilities ranged from α=0.70-0.77 for parents and from α=0.74-0.89 for adolescents across the five subscales. Comparing the means on the original scale using paired-sample t-tests for parents and youths, results showed that parents scored significantly higher on subscales of Positive Affect, Rituals, Communication, and Relationship Worries, but parents and youths did not differ on the Conflict subscale. Correlations between parents’ and youths’ reports were low, ranging from r=0.11 (Rituals) to r=0.24 (Positive Affect).

All parents’ subscales and several youths’ subscale reliabilities fell below the recommended cut-point of 0.80. Examination of the itemtotal correlations for each subscale for each group did not suggest obvious items to drop from each subscale to increase reliability; thus, exploratory factor analysis was conducted with each subscale to identify items that may not load strongly on the primary factor. Examination of the communalities indicated that at least two items within each subscale were poor candidates for factor analysis (communalities<.20) for parents and youth. These items were deleted; resulting in 5-6 items and a single-factor solution for each subscale (Table 1). This procedure did not strongly improve the subscale reliabilities, but did ensure that each subscale was accessing a unidimensional construct. A set of paired t-tests for the revised subscales indicated that parents’ scores remained significantly higher than adolescents’ scores on the revised Ritual, Communication, Positive Affect, and Relationship Worries subscales; difference between parents and youths on the Conflict subscale was significant, with youths scoring higher.

Table 1: Original scale items and their retention at each level of invariance.

Configural invariance

Confirmatory factor analysis was conducted to assess whether each domain was measured similarly for youths and parents. Due to the relatively small sample size, each subscale was tested separately. For each subscale, a structural equation model tested the hypothesis that a one-factor model was appropriate for the reduced set of items identified in the exploratory factor analysis. Each model included parents and youths’ reports for each subscale with parent and adolescent factors allowed to correlate. To control for non-independence of responses within the parent-youth dyad, the error terms of parallel items were also allowed to correlate. Each model was evaluated using the following criteria: a non-significant (p>.05) chi-square, TLI and CFI>.95, and RMSEA<.06. As shown in Table 2, the configural invariance assumption was upheld for each of the five subscales.

Table 2: Configural invariance results for all subscales.

Metric invariance

To test the metric invariance assumption that factor loadings of corresponding items are equal between groups, cross-group equality constraints were imposed on the relevant loadings. For example, in the model for the Positive Affect subscale, the loading for the parental “respected” item and the youth “respected” item were constrained to the same value; similar constraints were imposed for the remaining corresponding items. For each subscale, the metric invariance model was evaluated by a nested model comparison with the configural invariance model. A non-significant difference in chi-square and using the change in CFI (ΔCFI) with values between 0 and -.01 indicate support for metric invariance [20]. As shown in Table 3, only the Communication subscale did not meet criteria for metric invariance.

Table 3: Initial metric invariance results for all subscales.

When the metric invariance assumption is not met, invoking partial metric invariance is used as it allows the non-invariant corresponding items to be freely estimated. Alternatively, items are deleted that are not metric invariant and the model is re-calculated. Either approach requires identification of parent-youth item pairs that are metric invariant or equal. To determine the invariance of a given pair, equality constraints are imposed on that pair; the remaining pairs of items are free to vary and the single-item metric invariance model is compared to the baseline configural invariance model. A non-significant chi-square difference and a ΔCFI value (0 to -.01) provide support for metric invariance of the given parent-youth item pair. By rotating through each pair one at a time, problematic parent-youth item pairs are identified. Only the Communication subscale required revision to meet the full metric invariance assumption. “Communication” subscale item pairs indicated that variables of “angry” and “relationships” did not pass the chi-square difference criterion, although they did pass the ΔCFI criterion.

When freed in the model, both “angry” and “relationships” had higher loadings for youths than for parents, indicating that these items are more important to youth’s understanding of Family Communication than they are to parents. With conflicting evidence from the chi-square difference test and the ΔCFI, however, it was unclear whether one or both items should be dropped from the scale. Accordingly, a partial metric invariance model was calculated on the worst offender, “angry” and cross-group equality was constrained for all items, except “angry”. This model compared favorably with the baseline configural invariance model, Δχ² (3)=5.95, p=.114; ΔCFI=-0.005. Thus, the “relationship” item was retained and the item measuring “angry” was dropped from the subscale. The full metric invariance assumption was met for the revised four-item subscale, Δχ² (3)=6.07, p=0.11; ΔCFI=-0.000.

Scalar invariance

To test the full scalar invariance assumption that intercepts of corresponding items are equal between groups, cross-group equality constraints were imposed on the intercepts of each parent-youth item pair. For each subscale, the full scalar invariance model was evaluated against the final full metric invariance model for that subscale. As shown in Table 4, only the Rituals and the revised 4-item Communication subscales met the full scalar invariance assumption.

Table 4: Complete scalar invariance results for all subscales.

For the remaining three subscales, partial scalar invariance models weretested with each model constraining the intercepts of only one parent-youth item pair (Table 5). Differences between parents’ and youths’ intercepts capture discrepancies in each group’s propensity to score low or high on the item, apart from the influence of family functioning. For example, parents tended to score lower on the “respected” item (measuring their feeling of respect from family members) than did their teenage children, regardless of the degree of Positive Affect in the family.

Table 5: Single-item scalar invariance results for three subscales.

Results suggested that item intercepts differed on the Positive Affect subscale for the variable “respected” (parent intercept=-1.48, youth intercept=-.75) and “listen” (parent intercept=-.42, youth intercept=.03). The intercept for the item “respected” clearly differed between groups; while the results for “listen” were mixed (the chi-square difference test was significant, while the change in CFI was small). To further explore whether it was appropriate to constrain the parentyouth intercepts of “listen” to equality, a partial scalar invariance model with parent-youth equality constraints for the intercepts of “listen” and all other items except “respect” was tested. This model did not perform well against the metric invariance model (Δχ² (4)=15.77, p<.01; ΔCFI=- 0.016), indicating that it was necessary to release constraints on at least one more item. Releasing an additional constraint on “listen” yielded mixed results (Δχ² (3)=9.83, p<.05; ΔCFI=-0.009); however, the overall model fit for this partial scalar invariance model was good (χ² (55)=72.62, p=.06; CFI=.977). The items of “respected” and “listen” were dropped from the scale and all models were reanalyzed on the revised 4-item scale. These analyses resulted in a poor complete scalar invariance model when compared to the complete metric invariance model (Δχ² (3)=7.98, p<.05; ΔCFI=-0.013); however, each remaining variable in the model, when tested individually, showed non-significant differences between the intercepts. The complete scalar invariance model showed good overall model fit (χ² (21)=25.71,p=.22; CFI=.987). Thus, the exclusion of the items “respected” and “listen” yielded a Positive Affect subscale that was comparable between parents and children.

For the Conflict subscale, item intercepts differed on variables of “fight” (parents intercept=1.31, youths intercept=.87), “hit” (parent intercept=-.97, youth intercept=-.37), and “use” (parent intercept=.64, youth intercept=.05). A partial invariance model was tested allowing only these three item intercepts to differ between parents and youths; it performed well against the complete metric invariance model (Δχ² (2)=5.25,p=.07; ΔCFI=-0.01). Dropping the three items and comparing the new 3-item subscale’s scalar invariance model to its metric invariance model resulted in good nested model fit (Δχ² (2)=5.72,p=.06; ΔCFI=-0.003) and good overall fit for the complete scalar invariance model (χ² (9)=9.35,p=.41; CFI=.997).

On the Relationship Worries subscale, item intercepts differed only for the variable “angry” (parent intercept=1.54, youth intercept=1.03). A partial scalar invariance model allowing only the intercept of “angry” to differ between the two groups fared marginally worse than the complete metric invariance model (Δχ² (2)=6.82,p=.08; ΔCFI=-0.013). However, dropping this item from the scale and reanalyzing the complete scalar invariance model resulted in good nested model fit (Δχ² (3)=5.34, p=.15; ΔCFI=-0.013) and good overall fit (χ² (21)=29.31,p=.11; CFI=.955).

Comparison of results

In order to evaluate the results of the traditional comparison of means versus the invariance approaches, revised scales were calculated based on the items retained at each level of invariance (Table 1). Creating unidimensional and configurally invariant subscales resulted in improvement for the Conflict subscale over traditional methods; however, other subscales remained significantly different between parent/youth dyads at all levels of invariance.

Discussion

This study aimed to evaluate differences between parents and youths’ perceptions by testing measurement invariance/equality on a measure of family functioning. As measures are often used with the presumption that they are measuring concepts similarly across various respondent groups, it is important to evaluate this assumption. Even if a measure provides valid information for various respondent groups, it does not guarantee that the measure functions the same way across diverse groups [17]. This may be especially true of measures used with parent and youth respondents.

In this sample of runaway youths and their parents, results confirmed that parents’ perceptions of various facets of family functioning differed from their child’s view. For example, in terms of evaluations of family Communication, the item “I tell people in my family when I am angry with them” was more meaningful for adolescents than for parents. This discrepancy implies that the item has a different meaning for these two groups and requires further qualitative exploration. These findings may suggest that among adolescents, the item reflects feelings of trust and security, while it reflects a sense of discipline or authority for parents. Until the underlying meaning of the disclosure of anger and its relationship to family functioning is more fully understood and clearly measured, however, the item should not be used to compare youths’ and parents’ perceptions of the quality of family communication.

In terms of evaluating family positive effect, adolescents and their parents agreed on the nature and measurement of the construct, but displayed divergent biases in their reports of its meaning. Parents, regardless of the reality of their own family’s functioning, were less likely to feel that family members listen to and respect one another. In order to compare perceptions of two members of the same family, this group-level bias must be addressed. When the “respected” and “listen” items were dropped in our study, the parent-youth difference in perception of positive affect widened slightly. Therefore, removal of a negative bias common among the parents revealed an even stronger parent-youth gap in terms of understanding their family’s functioning.

Although some differences between groups in construct measurement and item bias were statistically significant in this study, they were not strong enough to substantially alter the conclusions that would be drawn from the standard practice of ensuring construct unidimensionality. Overall, it appears that parents have higher mean scores on subscales measuring the positive aspects of family functioning, such as mood or affect of the household, adherence to rituals and traditions, and communication. Conversely, adolescent children had higher scores on issues of conflict within the family. These findings, consistent with other studies may reflect parents’ investment in portraying the family ecology in a more positive manner than their children [1,7]. Even though runaway youths consistently report numerous chronic personal and family difficulties that precipitated their running away from parental homes, parents may feel family dynamics improve once the adolescent departs [5]. This change results in household functioning reaching a more satisfactory level and parents feeling more positive about the remaining family member’s interactions. On the other hand, some suggest that the stressors high-risk families encounter on a daily basis keep them from attending to household dynamics, resulting in misperceptions and misapprehensions among family members [2,4]. Exploring whether this is the situation among families with a runaway adolescent would aid understanding developmental pathways through which family discord results in adolescents prematurely leaving home.

Findings of this study highlight the caution required in using any scale to measure the same construct for two different populations. Although the lack of metric and scalar invariance for some of the Family Functioning Scale subscales did not affect the significance of group differences in this study, they did affect the size of some differences. Thus, researchers should be cautious and examine invariance assumptions when comparing constructs between very distinct populations, such as those from different cultures or life stages.

References

1. Yu S, Clemens R, Yang H, Li X, et al. (2006) Youth and parental perceptions of parental monitoring and parent-adolescent communication, youth depression, and youth risk behaviors. Social Behavior and Personality 34: 1297-1297
2. Pelton J, Forehand R (2001) Discrepancy between mother and child perceptions of their relationship: I. Consequences for adolescents considered within the context of parental divorce. Journal of Family Violence 16: 1-15.
3. Youngstrom E, Loeber R, Stouthamer-Loeber M (2000) Patterns and correlates of agreement between parent, teacher, and male adolescent ratings of externalizing and internalizing problems. J Consult Clin Psychol 68: 1038-1050.
4. Pelton J, Steele R.G, Chance M.W, Forehand R (2001) Discrepancy between Mother and Child Perceptions of Their Relationship: II. Consequences for Children Considered within the Context of Maternal Physical Illness. J Family Violence 16: 17-35.
5. Thompson SJ (2004) Risk/protective factors associated with substance use among runaway/homeless youth utilizing emergency shelter services nationwide. Subst Abus 25: 13-26.
6. Whitbeck L B(2009)Mental health and emerging adulthood among homeless young people. New York, NY: Psychology Press.
7. Safyer AE, Thompson S, Maccio E, Zittel-Palamara K, Forehand G (2004) Adolescent and parent perceptions of runaway behavior: Problems and solutions. Child and Adolescent Social Work Journal 21: 493-510
8. Slesnick N, Prestopnik JL (2004) Perceptions of the Family Environment and Youth Behaviors: Alcohol-Abusing Runaway Adolescents and Their Primary Caretakers. Fam J Alex Va 12: 243-253.
9. McManus HH, Thompson SJ (2008) Trauma Among Unaccompanied Homeless Youth: The Integration of Street Culture into a Model of Intervention. J Aggress Maltreat Trauma 16: 92-109.
10. Vandenberg RJ, Lance CE (2000) A Review and Synthesis of the Measurement Invariance Literature: Suggestions, Practices, and Recommendations for Organizational Research. Organizational Research Methods 3: 4-70.
11. Rusticus S, Hubley A (2006) Measurement Invariance of the Multidimensional Body-Self Relations Questionnaire: Can We Compare Across Age and Gender? Sex Roles 55: 827-842.
12. Tavitian M, Lubiner JL, Green L, Grebstein LC, Velicer WF (1987) Dimensions of family functioning. Journal of Social Behavior and Personality 2: 191-204.
13. Corcoran K ,Fischer J (2000) Measure for clinical practice: A sourcebook (3rd ed. Vol. 1). New York: The Free Press.
14. Thompson SJ, Windsor LC, Zittel-Palamara K (2005) Family reunification among two groups of runaway adolescents utilizing emergency shelters. Family Preservation Journal 8: 95-109.
15. Thompson SJ, Zittel-Palamara KM, Maccio E (2004) Runaway youth utilizing crisis shelter services: Predictors of presenting problems. Child and Youth Care Forum 33: 387-404.
16. Arbuckle JL (2006) Amos 7.0 user's guide. Chicago: SPSS Inc.
17. Milsap R.E 2010 Statistical approaches to measurement invariance. New York: Taylor and Francis Group
18. Kim E.W, Yoon M (2011) Testing measurement invariance: A comparison of multipe-grou categorical CFA and IRT. Structural Equation Modeling 18: 212-228.
19. Springer DW, Abell N, Hudson WW (2002) Creating and Validating Rapid Assessment Instruments for Practice and Research: Part 1. Research on Social Work Practice 12: 408-439.
20. Cheung G.W, Rensvold R.B (2002) Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance. Structural Equation Modeling 9: 233-255.