Reliability and Validity of the Eleven Item Kutcher Adolescent Depression Scale, Chinese Version (KADS-11CV)

• Depression evaluations scales however not only need to demonstrate reliability and validity, but also should be sensitive to symptom change over time, be completed quickly and demonstrate time utility for screening studies. In China, there are few studies of Depression scales that address these important considerations.

• The KADS-11CV is a short and user-friendly self-report tool that suggests good clinical and research utility with excellent reliability and validity in regards to the diagnosis of adolescent Depression in China.

• KADS-11CV can be applied in Chinese health-care settings and in epidemiological research into adolescent Depression in China.

Adolescence is a time during which individuals experience significant physiological, psychological and social changes. Concurrently, it is also a period in which mental disorders frequently onset. Rates of Depression increase rapidly over the decade following puberty and Depression is considered to be one of the most burdensome illnesses of young people [1-3]. Depression is associated with numerous negative academic, social, personal and vocational outcomes for young people. It is a risk factor for early mortality, suicide and a host of other illnesses, including but not limited to: substance abuse; diabetes and heart disease [4,5]. Effective interventions for the treatment of adolescent Depression are available and can be applied in both specialty mental health services and primary care [6,7].

Presently, there are few tools specifically designed to assess adolescent Depression in China. Two scales commonly suggested for use are the Children’s Depression Inventory (CDI) and the Reynolds Adolescent Depression Rating Scale (RADS). The CDI (Kovacs, 1992) is a 27 item scale that was derived from the Beck Depression Inventory (BDI) in 1977 to measure the severity of depression mood experienced by children or adolescents aged 7 to 17 years old. Typically the CDI takes about 15 minutes to complete, and the range of possible scores is 0 to 54 (Kovacs, 1992). It was not developed to address the assessment of youth (ages 13-20 years) and thus is limited to pediatric care settings. A number of Chinese studies of the CDI have demonstrate it to have good Cronbach alpha coefficients (ranging from 0.84 to 0.88 and good test-retest reliability (ranging from 0.79 to 0.81) [8-11]. The RADS [12] is a 30 item scale developed in 1981 to assess depressive symptoms in adolescents aged 13 to 18 years. It also takes about 15 minutes to complete and due to its age range it is limited to pediatric populations. Its possible total scores range from 30 to 120 [12]. In China, one study of a revised version of the RADS demonstrated good construct validity with a Cronbach α coefficient of 0.88, and a testretest correlation coefficient of 0.79 [13].

Depression evaluation scales however not only need to demonstrate reliability and validity, but also should be sensitive to symptom change over time and demonstrate utility in assessment of treatment outcome [14]. Furthermore, they should be able to be completed quickly and acceptable to the young person and clinician’s using the tool, and for researchers they should demonstrate time utility for application in population based screening studies. In China, there are few studies of Depression scales that address these important considerations.

The Kutcher Adolescent Depression Scale (KADS) is a self-report scale designed to assess the severity of adolescent depression. It is available in three versions: the original 16-item version, an 11-item version, and a 6-item version. The original version of KADS (KADS-16) contains 16 items, which collectively assess the frequency of occurrence and the severity of 16 core symptoms of adolescent depression.

The 16-item version has been tested in two studies, one of which enabled assessment of the diagnosis accuracy of each item, the other one assessed the sensitivity to change of each item. On the basis of the data from the former study, a 6-item version of the scale (KADS-6) taking approximately four minutes to complete was developed, optimized for depression diagnostic accuracy and deemed suitable for both clinical and epidemiological use [15]. On the basis of the data from the latter study, an 11-item version of the scale (KADS-11) was developed, for epidemiological use, clinical diagnosis and monitoring of treatment effects over time [16].

In a clinical trial study, Brooks et al. [16] further demonstrated that the KADS-11 not only provided a clinically useful diagnostic ascertainment tool and symptom severity measure, but that it was highly sensitive to change over time in a treatment setting. Compared to other concurrently applied measures of treatment outcome the KADS-11 demonstrated excellent sensitivity to symptom change over time and with a Cronbach’s alpha of 0.84, good concurrent validity and brevity of application was well suited to the clinical setting and acceptable to clinicians and subjects alike [16].

Based on these findings, we hypothesized that the KADS-11 can efficiently and effectively assess depressive mood for adolescents on the Chinese Mainland. We translated and back translated the KADS-11 to provide a Chinese version (KADS-11CV), tested its reliability and validity and calculated its optimal cut-off value and sensitivity/ specificity as well as addressing its clinical utility for use in the assessment of adolescent depression.

Participants

Following established ethical and organizational permission, the study was conducted in regular public schools from 6 provinces (Heilongjiang, Liaoning, Hebei, Henan, Jiangxi, Hunan) and one municipality (Shanghai). Three thousand four hundred students aged eleven to seventeen were selected by randomly stratified sampling from these schools. 220 students of this number did not complete the protocol, leaving a total of 3, 180 (Heilongjiang, n=454; Liaoning, n=497; Hebei, n=553; Henan, n=264; Jiangxi, n=771; Hunan, n=399; Shanghai, n=242), 93.5% of the initial sample. Of this number, 1661 (52.2%) were males, 1519 (47.8%) were females, mean age 13.85 years, SD=1.88 years).

The study was approved by the ethic committees of Shanghai Jiaotong University Affiliated Mental Health Center, and all the parents of participants signed an informed consent.

Measures

Development of the KADS-11CV: The original English version of the KADS-11 was translated into Chinese by one of the researchers, fluent in both languages. Then, another researcher also fluent in both languages back translated the KADS-11 into English. This version was then collaboratively reviewed by both investigators who agreed upon any modifications of words or sentences to ensure the Chinese version was appropriated adapted for the Chinese population. They came to consensus on the Chinese version of the KADS-11CV which is found in appendix one. As in the original version, the KADS-11CV is an 11 item self-report depression assessment scale with every item similarly scored from 0 to 3 according to the frequency of the occurrence of the symptom, 0 (hardly never), 1 (much of the time), 2 (most of the time), 3 (all of the time). The total score of the KADS-11CV is the sum of all the individual 11 item scores and can range from 0 to 33. The KADS-11CV can be completed and hand scored in 5 minutes.

CDI (Children’s Depression Inventory): CDI is a 27-item self-rated inventory that assesses depressive symptoms during the previous two weeks. Each item is scored from 0 to 2, and the total score (maximum of 54) is obtained by adding all items, with higher scores indicate greater severity. According to previous publications [17], we chose a total CDI score of ≥19 to define potential Depression, and were included in the following diagnosis interview. CDI scores have been shown to be reliable in our study, internal consistency α coefficient is 0.84. And the CDI scale has good discriminant validity, in our study the scale scores of depressed group is significantly higher than that of the non-depressed group (t=-33.31, P=0.000).

Procedure

Training: The researchers in our study were all trained by the study principal investigator at the Shanghai mental health center. The training course lasted two days, mainly including the characteristics of the CDI and KADS-11CV, the clinical presentations and diagnosis of Depression in adolescents, and notes of investigation process. At the end of the training course, we arranged some practice to make sure the consistency among the researchers’ evaluation.

Screening and diagnosis: Researchers from each of the selected study sites chose study schools using a random stratified sampling methodology. Students in the identified schools were asked to complete the CDI and KADS-11CV in their usual classroom time under the supervision of the researcher and the classroom teacher. For students whose CDI total scores totaled ≥19 (435 students) were interviewed by study psychiatrists and study diagnoses of Depression were assigned using DSM-IV criteria. A subsample of students (n=73) that randomly selected from the Shanghai site repeated this procedure one month later.

Statistical analysis: Data was imported into Epidata3.1 and analyzed using SPSS 17.0 and AMOS 17.0. Statistical methods employed were: correlation analysis, reliability analysis, two independent-samples T test, exploratory factor analysis, confirmatory factor analysis, receiver operating characteristic analysis. To test construct validity, the total sample was randomly divided into two parts, for the exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) respectively.

Reliability of the KADS-11CV

Internal consistency: Cronbach’s alpha (α) was high in the total sample for the KADS-11CV scale, α=0.84; All of the 11 items were significantly correlated with the total scores of the scale (p<0.01). Itemtotal correlations (r) ranged from 0.48 to 0.71. Subgroup analyses indicated that Cronbach’s alpha (α) in the girls were higher than that in the boys (girls: α=0.85, boys: α=0.83)

Split-Half reliability: Applying the parity split-half method, the splithalf reliability demonstrated a Pearson coefficient of 0.77 (P<0.01).

Test-Retest reliability: In the group of Shanghai students who completed a second application of the study (n=73) [18,19], the testretest Pearson coefficient for the one month interval was 0.77 (P<0.01).

Validity of the KADS-11CV

Concurrent validity: CDI has been widely used to address Depression and depressive symptoms in children and adolescents [20], and is well known to have substantial reliability and validity [9-11]. We compared the scores of the KADS-11CV to those from the CDI and established a significant positive association between the two instruments (r=0.740, P<0.01).

Construct validity: The total sample was randomly divided into two sections. One in which we conducted the EFA (n=1567) and one for which the CFA (n=1613) was determined. We confirmed that the split sample groups were not significantly different by analyzing sex (χ2=0.15, P=0.697), age (t=1.00, p=0.315) and CDI scale scores (t=1.53, P=0.125).

We determined the KMO index value (0.90) and Bartlett’s test of sphericity (p<0.05) to confirm that the data is robust enough for factor analysis. In the first sub-sample, two factors with eigenvalues >1 were extracted using the Principal Component Analysis method, these explained 48.6% of the variance (Table 1).

Table 1: Eigenvalues of factors and variance explained.

We determined that the absolute value of the correlation coefficient between the two factors is greater than 0.3(r=0.49), confirming that the two factors are reasonably correlated and on this basis applied the Oblique rotation method of analysis [21]. The Structure Matrix obtained from the Direct Oblimin (one component of the Oblique rotation), identified that factor 1 contains 5 items (item 1“low mood”, item 3“sleep difficulties”, item 4“decreased interest”, item 6“feeling tired”, item 8 “life is not very much fun”). We named this factor as “mixed states factor”. The second factor contains 6 items (item 2 “irritable”, item 5 “feeling of worthlessness”, item 7 “trouble concentrating”, item 9 “worried”, item 10 “physical feelings of worry”, item 11 “thoughts or actions about suicide or self-harm”). We named this factor as “cognitive-anxious factor” (Table 2).

Table 2: Structure Matrix of KADS-11CV.

We conducted a further analysis applying the CFA method to determine the fit of a single factor and a two factor model. Series of fitting indices are given in below table. Among these, RMSEA, GFI, AGFI, CFI, TLI and CN are all relatively stable fitting indices, and with the study sample changing larger, the values of these statistics change little [22,23]. With this analysis we determined that the two factor model fit the confirmatory sample (sample2) adequately (RMSEA<0.08, the values of GFI, AGFI, CFI, TLI are all higher than 0.90, CN>200). Furthermore, AIC and ECVI of the two-factor model are all lower than those of the single-factor model, and GFI, AGFI, CFI, TLI and CN of the two-factor model are all higher than those of the single-factor model (Table 3). The above results suggested that the two-factor model is better fit than the single-factor model.

Table 3: CFA results of the two-factor model and the single-factor model of the KADS-11CV

Criterion validity: Table 4 shows the comparison on individual item scores and total scores between the Depressed and non-Depressed groups. It demonstrates that the Depressed group scores significantly higher than the non-depressed group on each individual item and on the whole scale (P<0.001).

Table 4: Comparison of scores for depressed vs non-depressed groups.

ROC analysis and Cut-off points for the KADS-11CV

ROC analysis: The area under the ROC (AUC) analysis is an method that can be used to help to determine diagnostic accuracy. In the current study, the mean AUC of KADS-11CV was 0.938, CI=[0.908, 0.967, 95%], is considered to indicate adequate diagnostic accuracy [24]. The mean AUC of each item were also calculated and are presented in Table 5. Individual item AUC values ranged from 0.7 to 0.9. Of the 11 items, item 5 “feelings of worthlessness” presented the most adequate diagnostic accuracy, and item 4 “decreased interest” showed the least adequate diagnostic accuracy.

Table 5: AUC analysis of individual items

Cut-off points for diagnosis of depression with the KADS-11CV

Based on the Youden index and the location on the curve closest to the (0,1) point of the scale [24], and balancing the sensitivity and specificity of the scale, the current study determined the optimal cutoff point of KADS-11CV was a total score of ≥9. At this cut-off, sensitivity and specificity values for the KADS-11CV were 89% and 90%, respectively.

In this study, we tested the KADS-11CV for diagnostic utility in China with a large nationwide sample of young people. Our analysis demonstrated that a one-time application of the KADS-11CV demonstrated excellent reliability and validity in the evaluation of Depression in Chinese adolescents. The internal consistency α coefficient was 0.84, which was similar to that reported in a study of Canadian adolescents by Brooks et al. [16]. In that study, the Brooks and colleagues [16] evaluated the KADS-11 on seven occasions over the course of an 8 weeks duration psychopharmacology treatment study. In their study, the KADS-11 demonstrated similar and excellent internal consistency on each of the 7 occasions. Cronbach α coefficients ranged from 0.80 to 0.87, and the mean value over all applications was 0.84 [16]. Our study has furthered the work of Brooks et al. [16] investigation by demonstrating that in the KADS-11CV, all of the 11 items were significantly correlated with the total scores of the scale (p<0.01), the spilt-half reliability coefficient was 0.77 (P<0.01), the test-retest Pearson’s coefficient was 0.77 (P<0.01), the difference between the basline and follow-up assessment may be due to the random error. These additional evaluations increase our comfort with the value of the KADS-11CV measure.

In our analysis, the EFA suggests that the KADS-11CV consists of 2 main factors. Our further application of the CFA results showed that two-factor model is a better fit than the single factor model. This analysis which to our knowledge has previously not been applied to the KADS-11 [16]. We have called these two factors the “mixed states factor” and the “cognitive-anxious factor”. Further research using the KADS-11CV in clinical samples is necessary to determine if any dimensions of Depression, such as: family history of depression; comorbidity; response to treatment; relationship to life events; etc. may be uniquely related to one of these factors.

Furthermore, in our analysis, the KADS-11CV demonstrated good concurrent validity compared to the well-studied and widely deployed but much longer and more cumbersome to use CDI with the correlation coefficient between KADS-11CV and CDI being 0.74. Additionally, the Depressed group scored significantly higher than the non-Depressed group on each KADS-11CV item and the totally scale score as well, which shows the KADS-11CV has good criterion validity.

The results of our ROC analysis demonstrates that KADS-11CV has good diagnostic accuracy in adolescent Depression (AUC>0.9) and this positive relationship extends to each individual item (mean AUC values of each item ranged from 0.7 to 0.9). Of all the KADS- 11CV items, item 5 “feelings of worthlessness” presented the highest diagnostic accuracy, item 11 “thoughts/actions about suicide/self- harm” ranks seventh, and item 4 “decreased interest” showed the lowest diagnostic accuracy. This finding is not consistent with a previously reported Canadian study on the KADS-11 by LeBlanc et al. [15]. In that study, the investigators tested the KADS-11 in a school based sample of adolescents and based on their AUC analysis reported that item 11 “thoughts/actions about suicide/self-harm” ranked highest. We do not know why these findings differ, but one hypotheses that bears future testing is that perhaps Chinese students (for cultural or other reasons), tend to be more secretive about such emotionally charged issues as “suicide” or “self-harm” [25,26].

The current study also determined the optimal cut-off point of the KADS-11CV in the Chinese adolescent population as ≥9. At this score, the single application of the KADS-11CV sensitivity and specificity determinations for a clinical diagnosis of Depression were 89% and 90%, respectively. Previous studies before have not determined diagnostic cut-off points of the single time point administered KADS-11. Our findings in this large sample suggest that a single point application of the KADS-11CV in young Chinese people may be both an excellent clinical and epidemiological research application of the KADS-11CV, as its brevity lends itself to both situations equally.

The KADS-11CV is a short and user-friendly (for both patients and clinicians alike) self-report tool that suggests good clinical and research utility with excellent reliability and validity in regards to the diagnosis of adolescent Depression in China. Based on the findings of this study and the results of earlier published research on the KADS-11, we can comfortably recommend the KADS-11CV for application in Chinese health-care settings and in epidemiological research into adolescent Depression in China. Further study of the value of the KADS-11CV in clinical research in this population, in particular its sensitivity to change with therapeutic intervention, should be conducted.

This study was supported by a grant of the 2012 general program of National Natural Science Foundation of China (Grant Number: 81271510), a grant of 2012 Shanghai Municipal Health Bureau Youth Project (Grant Number: 20124y042), and 2013 Shanghai Mental Health Center featured specialty development project (Grant Number: 2013-YJTSZK-04).

The authors have no conflicts of interest to disclose.