Advertisement

Detecting Depression and Anxiety Among Adolescents in South Africa: Validity of the isiXhosa Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7

Open AccessPublished:October 20, 2022DOI:https://doi.org/10.1016/j.jadohealth.2022.09.013

      Abstract

      Purpose

      Screening tools such as the Patient Health Questionnaire-9 (PHQ-9) and the Generalized Anxiety Disorder-7 (GAD-7) could potentially be used in resource-limited settings to identify adolescents who need mental health support. We examined the criterion validity of the isiXhosa versions of the PHQ-9 and GAD-7 in detecting depression and anxiety among adolescents (10–19 years) in South Africa.

      Methods

      Adolescents were recruited from the general population and from nongovernmental organizations working with adolescents in need of mental health support. The PHQ-9 and GAD-7 were culturally adapted and translated into isiXhosa and administered to 302 adolescents (56.9% female). The Kiddie Schedule for Affective Disorders and Schizophrenia was administered by trained clinicians as the gold standard diagnostic measure for depression and anxiety.

      Results

      For the PHQ-9, the area under the curve was 0.88 for the full sample of adolescents (10–19 years old). A score of ≥10 had 91% sensitivity and 76% specificity for detecting adolescents with depression. For the GAD-7, the area under the curve was 0.78, and cutoff scores with an optimal sensitivity-specificity balance were low (≥6). A score of ≥6 had 67% sensitivity and 75% specificity for detecting adolescents with anxiety.

      Discussion

      The culturally adapted isiXhosa version of the PHQ-9 can be used as a valid measure for depression in adolescents. Further research on the GAD-7 for use with adolescents is recommended.

      Keywords

      Implications and Contributions
      This study determined the psychometric properties of the culturally adapted isiXhosa versions of the Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7 for a range of cutoff scores, for use with adolescents. These findings make a meaningful contribution to establishing tools to measure adolescent mental health at a population level in South Africa and other low- and middle-income countries.
      Adolescence is a particularly vulnerable period for mental health, with almost half of all mental disorders developing before the age of 18 years [
      • Solmi M.
      • Radua J.
      • Olivola M.
      • et al.
      Age at onset of mental disorders worldwide: Large-scale meta-analysis of 192 epidemiological studies.
      ]. Although major commitments have been made to improve mental health research globally [
      • Patel V.
      • Saxena S.
      • Lund C.
      • et al.
      The Lancet Commission on global mental health and sustainable development.
      ], we still know little about the mental health of the majority of the world's children and adolescents [
      • Erskine H.
      • Baxter A.
      • Patton G.
      • et al.
      The global coverage of prevalence data for mental disorders in children and adolescents.
      ]. Better prevalence data from low- and middle-income countries (LMICs) are urgently needed, particularly from sub-Saharan Africa where data coverage is essentially nonexistent [
      • Erskine H.
      • Baxter A.
      • Patton G.
      • et al.
      The global coverage of prevalence data for mental disorders in children and adolescents.
      ]. Research on adolescent mental health in LMICs is hampered by the lack of validated measurement tools that can be used at a population level. A recent systematic review reported high rates of depression (26.9%) and anxiety (29.8%) among the general population of adolescents in sub-Saharan Africa [
      • Jörns-Presentati A.
      • Napp A.-K.
      • Dessauvagie A.S.
      • et al.
      The prevalence of mental health problems in sub-Saharan adolescents: A systematic review.
      ]. However, most studies used screening tools, many of which have not been validated for use with adolescents in these settings. Cultural adaptation of tools and appropriate validation efforts in LMICs are needed to ensure that reported prevalence rates do not underestimate or overestimate the burden of the problem [
      • Kohrt B.A.
      • Kaiser B.N.
      Measuring mental health in humanitarian crises: A practitioner’s guide to validity.
      ]. Accurate data on prevalence are important to ensure that resources for mental health services are appropriately allocated [
      • Hayes J.
      • Carvajal L.
      • Hijazi Z.
      • et al.
      You can’t manage what you do not measure – why adolescent mental health monitoring matters.
      ].
      This study was conducted as part of the United Nations Children's Fund's Measurement of Mental Health Among Adolescents at the Population Level (MMAP) initiative that aims to develop and validate measurement tools that support large-scale collection of robust data on adolescent mental health [
      • Carvajal L.
      • Ahs J.W.
      • Requejo J.H.
      • et al.
      Measurement of mental health among adolescents at the population level: A multicountry protocol for adaptation and validation of mental health measures.
      ]. To allow for effective integration into national data-collection efforts, measures need to be brief and easy to administer, using suitable language and phrasing for the population and setting. In addition, the tools should lend themselves to cross-cultural adaptation and for multicountry comparisons. Two of the measures being validated for adolescents as part of the MMAP initiative are the widely used Patient Health Questionnaire-9 (PHQ-9), a screening tool for depression, and the Generalized Anxiety Disorder-7 (GAD-7), a screening tool for anxiety. Both tools have been used with adolescents in a handful of sub-Saharan African countries [
      • Adjorlolo S.
      Generalised anxiety disorder in adolescents in Ghana: Examination of the psychometric properties of the Generalised anxiety disorder-7 scale.
      ,
      • Anum A.
      • Adjorlolo S.
      • Kugbey N.
      Depressive symptomatology in adolescents in Ghana: Examination of psychometric properties of the patient health questionnaire-9.
      ,
      • Buckley J.
      • Otwombe K.
      • Joyce C.
      • et al.
      Mental health of adolescents in the era of antiretroviral therapy: Is there a difference between HIV-infected and uninfected youth in South Africa?.
      ,
      • Fatiregun A.A.
      • Kumapayi T.E.
      Prevalence and correlates of depressive symptoms among in-school adolescents in a rural district in southwest Nigeria.
      ,
      • Osborn T.L.
      • Venturo-Conerly K.E.
      • Wasil A.R.
      • et al.
      Depression and anxiety symptoms, social support, and demographic factors among Kenyan high school students.
      ], but they have not been validated for use among adolescents in South Africa.
      Our study examined the criterion validity of the culturally adapted isiXhosa versions of the PHQ-9 and GAD-7 in detecting depression and anxiety among adolescents in Khayelitsha, South Africa.

      Methods

      Setting

      The study was conducted in Khayelitsha, a large periurban neighborhood outside of Cape Town, South Africa. Khayelitsha, meaning “new home” in isiXhosa, was originally established for Black people under the Apartheid government. The area remains affected by inadequate service delivery and high rates of socioeconomic deprivation [
      • Smit W.
      • de Lannoy A.
      • Dover R.V.
      • et al.
      The built environment and non-communicable diseases in Khayelitsha, Cape Town.
      ], with many households affected by domestic violence, assault, rape, and murder [
      • Cassidy T.
      • Lloyd S.
      • Bowman B.
      Violence, alcohol and symptoms of depression and in Cape Town’s poorest communities: Results of a community survey. UCT working Paper.
      ]. As such, adolescents are commonly exposed to high-stress living environments, placing them at greater risk of poor mental health.

      Participants

      Participants were isiXhosa-speaking adolescents between the ages of 10 and 19 years, living in Khayelitsha. To effectively assess the measures' psychometric stability [
      • Wolf E.J.
      • Harrington K.M.
      • Clark S.L.
      • et al.
      Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety.
      ], we aimed for a sample of 300 adolescents across the younger (10–14 years) and older (15–19 years) age range, with an even split between male and female adolescents. We aimed to obtain an enriched sample or a specific proportion of participants likely to have anxiety or depression at a 2:1 ratio to participants without these conditions. We conducted rolling recruitment to ensure that an adequate number of adolescents likely to have symptoms of depression or anxiety were included, using three methods of recruitment. First, we used school-based recruitment where the research team presented the project to one class (usually with 40–50 students) in each grade (grades 4–11). Information sheets were distributed to take home, and interested families were asked to return the completed form to the school. Second, we recruited older adolescents directly from existing community networks, such as street committees. Third, to increase the number of adolescents in the sample likely to have depression or anxiety, we recruited directly from local nongovernmental organizations that work with adolescents in need of mental health support.

      Measures

      Depression and anxiety screening

      We used the isiXhosa versions of the PHQ-9 [
      • Kroenke K.
      • Spitzer R.L.
      • Williams J.B.
      The PHQ-9: Validity of a brief depression severity measure.
      ] and GAD-7 [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.W.
      • Löwe B.
      A brief measure for assessing Generalized Anxiety Disorder: The GAD-7.
      ], which were adapted for use (Box 1). Both measures ask participants to rate how often over the last two weeks they were bothered by specific symptoms, with scores ranging from 0 (not at all) to 3 (nearly every day). For the PHQ-9, we used a slight variation to item 7 (“trouble concentrating on things like school work, reading, or watching television” as opposed to “...reading the newspaper or watching television”), similar to the PHQ-A (a modified version for adolescents).
      Transcultural translation and adaptation
      We used a systematic transcultural translation and adaptation process [
      • Carvajal L.
      • Ahs J.W.
      • Requejo J.H.
      • et al.
      Measurement of mental health among adolescents at the population level: A multicountry protocol for adaptation and validation of mental health measures.
      ] to produce an isiXhosa version of the Measurement of Mental Health Among Adolescents at the Population Level questionnaire. Original items were translated from English to isiXhosa by bilingual local mental health experts via group consensus, with a focus on cultural nuances and age-appropriate language. Translated items were presented to adolescents and their caregivers via focus group discussions (n = 6) and cognitive interviews (n = 18). Participant feedback was incorporated into an updated version of the questionnaire, which we presented to a group of local researchers for final revisions. Lastly, we conducted a blind back translation to ensure that translated items retained the original meaning. The adaptation process resulted in the following key changes:
      • (1)
        The response category of “several days” (a score of one on the Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7) was changed in the translated version to “a few days”;
      • (2)
        Items were reframed from statements (“I feel…”) to questions (“how often do you feel…?”);
      • (3)
        Wording of certain items was adapted to increase clarity for adolescents;
      • (4)
        Visual aids were produced to assist with respondents' understanding of time frames and response categories (Figure A1).
        Figure thumbnail gr1
        Figure 1Endorsement of PHQ-9 items by the K-SADS depression diagnosis and endorsement of GAD-7 items by the K-SADS anxiety diagnosis (means and 95% CI).
      The adapted versions of Patient Health Questionnaire-9 and Generalized Anxiety Disorder-7 in English and isiXhosa are available in the Supplemental Annex.

      Broader mental health

      The PHQ-9 and GAD-7 were included as part of a larger MMAP questionnaire, consisting of United Nations Children's Fund-developed measures to assess suicidal ideation and behavior, functional limitations, mental health care, and connectedness [
      UNICEF
      ].

      Diagnostic assessment

      We used the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS) as the gold standard diagnostic measure for depression and anxiety.
      The K-SADS is a semistructured interview for children aged 6–18 years, designed to assess ongoing psychopathology, including mood and anxiety disorders [
      • Kaufman J.
      • Birmaher B.
      • Brent D.
      • et al.
      Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): Initial reliability and validity data.
      ]. Criteria-based algorithms determine the presence of current disorders, in line with the Diagnostic and Statistical Manual of Mental Disorders [
      American Psychiatric Association
      Diagnostic and statistical manual of mental disorders: DSM-5.
      ]. We used the depression module and a selection of anxiety modules (generalized, social, and separation) that correspond with anxiety disorders that commonly impact this age group.
      The K-SADS was translated into isiXhosa by mental health professionals fluent in both languages through a process of group consensus. We piloted the translated version with adolescents, under the supervision of a clinical psychologist, to refine the translations and to establish inter-rater reliability.

      Procedures

      The study protocol was approved by the Health Research Ethics Committee at Stellenbosch University (N19/08/104) and the Western Cape Department of Education (20200206-4132). The MMAP questionnaire was administered as an interview by local trained data collectors, fluent in English and isiXhosa, experienced in working with adolescents in Khayelitsha. The training focused on informed assent and consent procedures, questionnaire administration, data management, and referral and emergency protocols. The K-SADS was administered as the diagnostic assessment by trained and supervised social workers, fluent in isiXhosa and experienced in working with children and adolescents. Similar to many LMICs, South Africa is characterized by severe human resource shortages for mental health [
      ]. Specialists such as psychiatrists and psychologists—particularly those who speak local languages such as isiXhosa—are in exceptionally short supply. As a result, no isiXhosa-speaking psychologist, psychiatric nurse, or clinical social worker was available to conduct the diagnostic assessments for the study. Two social workers administered the K-SADS, under the supervision of a registered clinical psychologist (D.G.). They received extensive training over a three-week period, including training on the presentation and identification of depressive and anxiety disorders, interviewing techniques, K-SADS administration, and referrals.
      A three-day pilot of assessment procedures took place to identify administration or logistical issues before starting the study. Six adolescents (three in the younger age group and three in the older age group) participated in the pilot.
      Data collection took place at a community research center in Khayelitsha between May and November 2021. Following informed assent or consent, adolescents were interviewed in isiXhosa, using the MMAP questionnaire. Trained data collectors captured participant responses using a preprogrammed questionnaire on tablet devices. Alerts were programmed to flag the need to activate a referral protocol if participants were considered to be at risk. Interviews were audio-recorded for quality control purposes. After a refreshment break, adolescents proceeded to see a social worker for the K-SADS interview, with scores captured on tablet devices. Following the interview, the social worker and data collector met to discuss the need for referral, and the social worker facilitated referrals to relevant services. The entire assessment visit lasted 1.5–2 hours. Any K-SADS interviews that could not be completed on the same day as the MMAP interview were scheduled within a 48-hour window. As a token of appreciation, all participants received an R160 (∼11 USD) supermarket voucher.
      Online data submissions were reviewed daily to track progress and referrals. A random selection of audio-recordings from each data collector were reviewed weekly, with constructive feedback provided during team meetings. K-SADS assessments were video-recorded to enable a detailed review of cases and ongoing training during weekly supervision sessions.

      Statistical analysis

      Descriptive statistics were completed for the full sample and stratified by younger and older age groups. Based on the K-SADS results, the following diagnostic categories were used for the analysis: depression diagnosis (evidence of a major depressive disorder), anxiety diagnosis (evidence of generalized anxiety, separation anxiety, and/or social anxiety disorder), any diagnosis (evidence of depression and/or anxiety), and no diagnosis.
      To assess the criterion validity of the PHQ-9 and GAD-7 against a clinician's diagnosis, we used total scores to construct a receiver operating characteristic curve and calculated the area under the curve (AUC) for each test, using the K-SADS as the gold standard. For AUC calculations, we report exact binomial 95% confidence intervals (95% CI). To make determinations related to the tests' validity, we calculated psychometric properties (Table A1) for a range of cutoff values. We completed all analyses using Stata, version 17 (Statcorp LP, College Station, Texas).

      Results

      The sample consisted of 302 adolescents (56.9% female), with 43% younger and 57% older adolescents (Table 1). Over half of the sample (56%) were recruited from organizations working with adolescents in need of mental health support, and 44% were recruited via school or community avenues.
      Table 1Participant demographic information
      10–14 Years old (n = 130)15–19 Years old (n = 172)Total sample (n = 302)
      Age
       Mean (SD)12.09 (1.43)17.02 (1.52)14.90 (2.86)
      Gender
       Male50 (38.5%)78 (45.3%)128 (42.4%)
       Female80 (61.5%)92 (53.5%)172 (56.9%)
       Other02 (1.2%)2 (0.7%)
      Recruitment source
       School/Community66 (50.8%)67 (39%)133 (44%)
       NGO64 (49.2%)105 (61%)169 (56%)
      Home language
       isiXhosa128 (98.5%)167 (97.1%)295 (97.7%)
       English/other2 (1.5%)5 (2.9%)7 (2.3%)
      School language
       isiXhosa93 (71.5%)111 (64.5%)204 (67.5%)
       English/other37 (28.5%)61 (35.5%)98 (32.5%)
      Education (highest completed grade)
      Missing: n = 3.
       Grade 1–448 (37.2%)048 (16%)
       Grade 5–778 (60.5%)18 (10.6%)96 (32.1%)
       Grade 8–103 (2.3%)103 (60.6%)106 (35.5%)
       Grade 11–12049 (28.8%)49 (16.4%)
      PHQ-9 (depression symptom screen)
       Mean (SD)7.05 (5.28)7.31 (5.19)7.20 (5.22)
       Moderate to severe symptoms (score of 10+)39 (30%)58 (33.7%)97 (32.1%)
      GAD-7 (anxiety symptom screen)
       Mean (SD)4.28 (3.85)5.63 (4.74)5.05 (4.42)
       Moderate to severe symptoms (score of 10+)15 (11.5%)39 (22.7%)54 (17.8%)
      Suicide attempt (ever)
      Missing: n = 2.
      7 (5.4%)30 (17.5%)37 (12.3%)
      Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS)
       No diagnosis118 (90.7%)127 (73.8%)245 (81.1%)
       Depression diagnosis3 (2.3%)20 (11.6%)23 (7.6%)
       Anxiety diagnosis9 (6.9%)36 (20.9%)45 (14.9%)
       Any diagnosis12 (9.2%)45 (26.1%)57 (18.9%)
      NGO = nongovernmental organization; SD = standard deviation.
      a Missing: n = 3.
      b Missing: n = 2.
      Based on severity thresholds of the original PHQ-9 [
      • Kroenke K.
      • Spitzer R.L.
      • Williams J.B.
      The PHQ-9: Validity of a brief depression severity measure.
      ], 32.1% of adolescents (65% female) were categorized with moderate to severe depression symptoms (a total score of 10 or higher). Using the GAD-7 [
      • Spitzer R.L.
      • Kroenke K.
      • Williams J.B.W.
      • Löwe B.
      A brief measure for assessing Generalized Anxiety Disorder: The GAD-7.
      ], 17.8% (57% female) of adolescents were categorized with moderate to severe anxiety symptoms (a total score of 10 or higher). In addition, 12.3% of adolescents—predominantly older adolescents—reported that they had previously attempted suicide.
      Using the K-SADS as the diagnostic tool, 7.6% of adolescents were diagnosed with depression, and 14.9% were diagnosed with anxiety. Eleven adolescents (3.6%) were diagnosed with both anxiety and depression.
      During the assessment visit, six participants were identified as actively suicidal (had a plan and intended to carry out the plan) and were immediately referred to the nearest hospital for assistance.

      Psychometric analysis

      Results are presented for the full sample and for older adolescents (15–19 years old). Due to the small number of younger adolescents with a diagnosis according to the K-SADS (n = 3 for depression; n = 9 for anxiety), we were not able to make generalizable statements about the PHQ-9's or GAD-7's performance with this age group.
      Using the PHQ-9 to discriminate between adolescents with and without a depression diagnosis on the K-SADS, the AUC (Figure A2) was 0.88 for the full sample (95% CI 0.81–0.95) and 0.88 for older adolescents (95% CI 0.80–0.96).
      Using the GAD-7 to discriminate between adolescents with and without an anxiety diagnosis on the K-SADS, the AUC was 0.78 for the full sample (95% CI 0.71–0.85) and 0.79 for older adolescents (95% CI 0.71–0.87) (Figure A3).
      Table 2 shows the test characteristics of the PHQ-9 and GAD-7 using the K-SADS as the gold standard. The PHQ-9 performed best for discriminating depression among the full sample, with the highest diagnostic odds ratio (OR) of 33.89 at a cutoff score of 10 or greater, with a sensitivity of 0.91, specificity of 0.76, and an overall accuracy of 0.77. For older adolescents, the highest diagnostic OR of 31.24 resulted from a cutoff score of 11 or greater. At this cutoff, the PHQ-9 had a sensitivity of 0.90 and a specificity of 0.78 for detecting older adolescents with major depression on the K-SADS, with an overall accuracy of 0.79.
      Table 2Validation psychometrics of the PHQ-9 and the GAD-7 from comparison with the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS)
      Cutoff scoreSensitivitySpecificityPPVNPVPLRNLRDiagnostic ORYouden's Index (J)TP (%)TN (%)FP (%)FN (%)Accurately classified (%)
      Total PHQ-9 score
       Full sample (10–19 years old)≥70.910.620.170.992.430.1417.400.540.070.580.350.010.65
      ≥80.910.670.180.992.740.1321.000.580.070.620.310.010.69
      ≥90.910.730.220.993.350.1228.050.640.070.670.250.010.74
      ≥100.910.760.240.993.860.1133.890.680.070.710.220.010.77
      ≥110.740.820.260.974.210.3213.300.560.060.760.160.020.82
      ≥120.740.880.340.986.250.3021.120.620.060.810.110.020.87
      ≥130.570.920.370.967.170.4715.190.490.040.850.070.030.89
       Older adolescents (15–19 years old)≥70.900.550.210.981.980.1810.830.450.100.480.400.010.59
      ≥80.900.640.250.982.490.1615.870.540.100.560.320.010.67
      ≥90.900.670.260.982.740.1518.360.570.100.590.290.010.70
      ≥100.900.740.310.983.420.1425.200.640.100.650.230.010.76
      ≥110.900.780.350.984.020.1331.240.680.100.690.200.010.79
      ≥120.700.850.380.964.630.3513.090.550.080.750.130.030.83
      ≥130.700.890.470.966.650.3419.830.590.080.790.090.030.87
      Total GAD-7 score
       Full sample (10–19 years old)≥40.820.650.290.952.320.288.440.470.120.550.300.030.67
      ≥50.760.700.310.942.520.357.230.460.110.600.250.040.71
      ≥60.670.750.320.932.640.455.910.410.100.640.220.050.74
      ≥70.620.780.330.922.860.485.910.400.090.670.190.060.76
      ≥80.580.840.380.923.540.507.010.410.090.710.140.060.80
      ≥90.490.870.400.913.810.596.490.360.070.740.110.080.81
      ≥100.400.910.440.904.470.666.780.310.060.770.080.090.83
       Older adolescents (15–19 years old)≥40.860.490.310.931.670.295.850.350.180.380.410.030.56
      ≥50.810.630.360.922.150.316.900.430.170.490.300.040.66
      ≥60.720.680.380.902.280.415.620.410.150.540.250.060.69
      ≥70.670.740.400.892.520.455.560.400.140.580.210.070.72
      ≥80.640.770.430.892.800.475.990.410.130.610.180.080.74
      ≥90.640.820.480.903.480.447.860.460.130.650.150.080.78
      ≥100.530.850.490.873.590.556.480.380.110.670.120.100.78
      FN = false negative; FP = false positive; GAD-7 = Generalized Anxiety Disorder-7; NLR = negative likelihood ratio; NPV = negative predictive value; OR = odds ratio; PHQ-9 = Patient Health Questionnaire-9; PLR = positive likelihood ratio; PPV = positive predictive value; TN = true negative; TP = true positive.
      Compared to the PHQ-9, validity was slightly weaker for the GAD-7: The optimal cutoff value for maximizing sensitivity without loss of specificity was a score of six or greater for older adolescents and for the full sample. For the full sample, a cutoff score of six had a sensitivity of 0.67 and a specificity of 0.75, with a diagnostic OR of 5.91 and an overall accuracy of 0.74. For older adolescents, a cutoff score of six had higher sensitivity (0.72) but lower specificity (0.68), with a slightly lower diagnostic OR of 5.62 and overall accuracy of 0.69.

      Item analysis

      Item means for the PHQ-9 and GAD-7 were calculated separately for adolescents who were diagnosed as having an anxiety or depressive disorder versus adolescents who did not receive any diagnosis. For the anxiety analysis, we excluded adolescents with depression who did not have anxiety. For the depression analysis, we excluded adolescents who had anxiety without comorbid depression. Table 3 presents the discriminant ability of items on the PHQ-9 and GAD-7 for adolescents with and without a diagnosis according to the K-SADS results. For the PHQ-9, the majority of items performed well by showing significant differences between respondents with and those without a depression diagnosis, with the exception of item 5 (“poor appetite or overeating”) and item 8 (“moving or speaking slowly/being fidgety or restless”). For the GAD-7, item 5 (“so restless that it is hard to sit still”) did not discriminate between diagnosed and undiagnosed groups.
      Table 3Discriminant ability of PHQ-9 and GAD-7 items for adolescents with and without diagnosis on the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS) for the full sample and subsample of 15–19 years old
      PHQ-915–19 Years oldFull sample
      Mean (SD)Mean (SD)Mean (SD)Mean (SD)
      Item #DescriptionNo diagnosis (n = 127)Depression (n = 20)No diagnosis (n = 245)Depression (n = 23)
      PHQ 1Little interest or pleasure in doing things0.96 (0.99)1.70 (1.03)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      1.11 (1.09)1.78 (1.04)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      PHQ 2Feeling down, depressed, or hopeless0.91 (1.03)2.35 (0.93)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.83 (0.99)2.39 (0.89)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      PHQ 3Trouble falling asleep, staying asleep, or sleeping too much0.83 (1.05)1.70 (1.08)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.89 (1.09)1.70 (1.06)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      PHQ 4Feeling tired/having little energy0.59 (0.76)1.90 (1.21)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.64 (0.80)1.96 (1.19)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      PHQ 5Poor appetite or overeating0.56 (0.83)1.05 (1.05)0.64 (0.95)1.17 (1.07)
      PHQ 6Feeling bad about yourself/that you are a failure/letting people down0.59 (0.88)1.85 (0.99)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.55 (0.87)2.00 (1.00)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      PHQ 7Trouble concentrating0.87 (0.95)1.90 (1.17)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.87 (0.98)1.83 (1.19)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      PHQ 8Moving or speaking slowly/being fidgety or restless0.31 (0.74)0.60 (0.88)0.40 (0.79)0.65 (0.83)
      PHQ 9Thoughts that you would be better off dead/hurting yourself0.20 (0.60)1.30 (1.22)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.18 (0.54)1.26 (1.14)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      Total PHQ-9 score5.80 (4.29)14.35 (5.22)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      6.10 (4.50)14.74 (5.01)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD-715–19 Years oldFull sample
      Mean (SD)Mean (SD)Mean (SD)Mean (SD)
      Item #DescriptionNo diagnosis (n = 127)Anxiety (n = 36)No diagnosis (n = 245)Anxiety (n = 45)
      GAD 1Feeling nervous, anxious, or on edge0.51 (0.76)1.33 (1.15)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.45 (0.71)1.20 (1.14)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD 2Not able to stop or control worrying0.62 (0.86)1.39 (1.08)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.52 (0.80)1.27 (1.07)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD 3Worrying too much about different things0.58 (0.78)1.31 (1.14)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.56 (0.79)1.36 (1.17)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD 4Trouble relaxing0.43 (0.74)1.08 (1.20)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.44 (0.75)1.11 (1.13)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD 5So restless that it is hard to sit still0.55 (0.89)0.94 (1.04)0.57 (0.85)0.87 (1.04)
      GAD 6Easily annoyed/irritable1.08 (0.99)2.19 (1.12)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      1.05 (1.04)1.93 (1.21)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD 7Feeling afraid as if something awful might happen0.53 (0.72)1.47 (1.18)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      0.49 (0.70)1.42 (1.18)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      Total GAD-7 score4.31 (3.81)9.72 (5.21)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      4.09 (3.68)9.16 (5.06)
      p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      GAD = Generalized Anxiety Disorder; PHQ = Patient Health Questionnaire; SD = standard deviation.
      a p < .05, T-tests for mean differences for diagnosed and undiagnosed groups, with Bonferroni correction for multitesting (9 comparisons for depression items and seven comparisons for anxiety items).
      Figure 1 provides PHQ-9 item means by K-SADS diagnostic status, comparing adolescents with no diagnosis to those with a depression diagnosis. The three most frequently endorsed items on the PHQ-9 among undiagnosed adolescents were item 1 (Little interest or pleasure in doing things), item 2 (Feeling down, depressed, or hopeless), and item 7 (Trouble concentrating). All three items performed well to distinguish between depressed and nondepressed adolescents.
      Figure 1 also provides GAD-7 item means by K-SADS diagnostic status, comparing undiagnosed adolescents to those with an anxiety diagnosis. The three most frequently endorsed items on the GAD-7 among undiagnosed adolescents were item 6 (Easily annoyed/irritable), item 2 (Not being able to stop or control worrying), and item 3 (Worrying too much about different things). Similar to the PHQ-9, all three items successfully discriminated between diagnosed and undiagnosed adolescents.

      Adjustments for population prevalence

      Using cutoff values that balance sensitivity and specificity, Figure 2 shows what would be reflected in the outcomes of reported prevalence rates on the PHQ-9 and GAD-7 based on estimates of true prevalence rates. When adjusting for false positives and false negatives, the PHQ-9 and GAD-7 identified prevalence rates for depression and anxiety can be adjusted to approximate what the true prevalence may be in the population. For example, if a prevalence of 36% is identified for depression among 15- to 19-year-old adolescents, the estimated true prevalence is likely closer to 20%. The degree of adjustment differs based on prevalence because of the contribution of false positive versus false negatives to the estimates made. The positive predictive value and negative predictive value for individual adolescents also vary by prevalence rates (also included in Figure 2). Policy-makers can use algorithms or figures such as this to make adjusted prevalence estimates when allocating resources and designing programs.
      Figure thumbnail gr2
      Figure 2Detected prevalence rates for anxiety and depression using the PHQ-9 and GAD-7 at different estimated true prevalence rates.

      Discussion

      Our study is the first to evaluate the psychometric properties of the PHQ-9 and GAD-7 against a diagnostic interview for use with adolescents in South Africa. This validation exercise of the culturally adapted isiXhosa versions of the PHQ-9 and GAD-7 makes a meaningful contribution to establishing tools to measure adolescent mental health at a population level in South Africa and potentially other LMICs.
      Our sample included adolescents from the general population as well as those attending nongovernmental organizations that provide adolescents with mental health support. We determined the psychometric properties of the PHQ-9 and GAD-7 for a range of cutoff scores. Cutoff scores should be selected based on the intended use of the tool for different applications. For example, the cutoff scores appropriate for screening adolescents to include in interventions may differ from cutoff scores used for determining population prevalence.
      Measures with high sensitivity should be prioritized when risk factors to health and safety are serious [
      • Kohrt B.A.
      • Kaiser B.N.
      Measuring mental health in humanitarian crises: A practitioner’s guide to validity.
      ]. For adolescent populations in LMICs, using tools with the highest possible sensitivity is crucial given that self-harm is among the top five causes of death and that depressive and anxiety disorders are among the leading causes of disability [
      World Health Organization
      Global Health Estimates: Life expectancy and leading causes of death and disability.
      ]. At the same time, high specificity is also important in population-based research, to reduce the likelihood of overburdening resource-deprived health systems with high numbers of false positives. Achieving high specificity without compromising sensitivity is therefore a key priority for measuring adolescent mental health at a population level. We provided estimates for adjusting population prevalence rates based on the tools' psychometric properties. Of note, high reported prevalence rates need substantial adjustment to prevent overestimation of the population burden.
      Regarding cutoff scores that balance sensitivity and specificity, we identified a cutoff score of 10 or higher on the PHQ-9 to indicate a potential diagnosis of depression. Using this cutoff, the PHQ-9 demonstrated high sensitivity (91%) and good specificity (76%) for detecting depression among adolescents aged 10–19 years.
      The GAD-7 demonstrated a 78% chance for discriminating between adolescents with and without an anxiety disorder. Cutoff scores with an optimal sensitivity-specificity balance were low (a score of six or more). Using this cutoff, the GAD-7 demonstrated moderate sensitivity (67%) and good specificity (75%). Limited validation research with adolescents is available to help us make sense of these findings. In Finland [
      • Tiirikainen K.
      • Haravuori H.
      • Ranta K.
      • et al.
      Psychometric properties of the 7-item Generalized Anxiety Disorder Scale (GAD-7) in a large representative sample of Finnish adolescents.
      ] and Ghana [
      • Adjorlolo S.
      Generalised anxiety disorder in adolescents in Ghana: Examination of the psychometric properties of the Generalised anxiety disorder-7 scale.
      ], the GAD-7 demonstrated factorial and construct validity in adolescents; however, neither study assessed criterion validity against a diagnostic assessment.
      It is possible that the GAD-7 performed poorly compared to the PHQ-9 because of differences in the duration for these conditions in the K-SADS for a clinical diagnosis. The PHQ-9 and K-SADS use a 2-week period of symptoms. However, while the GAD-7 uses a 2-week period, the K-SADS requires 6 months of symptoms for a diagnosis of generalized anxiety disorder.
      Information on individual item performance helps to identify items that do not discriminate between adolescents with and without a condition that could potentially be removed. Most items demonstrated a good discriminant ability, with the exception of “poor appetite or overeating” on the PHQ-9 and items related to movement on the PHQ-9 and GAD-7 (“moving or speaking slowly/being fidgety or restless” and “so restless that it is hard to sit still”). In two other LMICs (Nepal and Nigeria), items related to appetite also performed poorly in discriminating depression among adolescents [
      • Kohrt B.A.
      • Jordans M.J.
      • Tol W.A.
      • et al.
      Validation of cross-cultural child mental health and psychosocial research instruments: Adapting the depression self-Rating scale and Child PTSD symptom scale in Nepal.
      ]. It is possible that adolescents in general are more likely to struggle to sit still or to experience appetite fluctuations than adults, and therefore, these items did not work well to distinguish between adolescents with and without depression or anxiety when using the PHQ-9 or GAD-7. This notion however requires further exploration.

      Strengths and limitations

      The results should be considered in light of the following strengths and limitations. First, the adaptation and translation work conducted to produce the measures for validation was extensive and included local experts, adolescents, and their caregivers in the process. Another strength is the use of a comprehensive, high-quality diagnostic interview as the gold standard. In addition, we demonstrated that other cadres of mental health workers can be trained and supervised to conduct diagnostic assessments, an important finding in light of the shortages of mental health specialists in South Africa and many other LMICs.
      Because the study took place in one area in South Africa, using one language (isiXhosa), results may not be generalizable to other adolescent populations in South Africa or other countries. We used targeted recruitment to include adolescents likely to experience depression or anxiety, and our prevalence rates are possibly higher than what might be seen when conducting screening in the general population. Despite these efforts, we did not achieve the desired ratio between diagnosed and undiagnosed adolescents, leading to smaller sample sizes for analysis. Our study included an extremely small sample of younger adolescents with diagnoses. Future studies with larger samples of younger adolescents with depression and anxiety are needed to draw conclusions about the performance of the PHQ-9 and the GAD-7 in this age group.
      We emphasized categorical analyses of disorders for the purpose of public health reporting, which typically requires categorical classification for reporting and policy-making. However, it is important to consider more continuous uses of these scales for tracking clinical improvements and for epidemiological research on risk and protective factors.

      Conclusion

      In global health, we cannot effectively manage what we do not measure [
      • Hayes J.
      • Carvajal L.
      • Hijazi Z.
      • et al.
      You can’t manage what you do not measure – why adolescent mental health monitoring matters.
      ]; therefore, addressing adolescent mental health starts with identifying appropriate tools to collect valid and reliable data on the prevalence of adolescent mental health conditions. These tools should enable both clinicians and researchers to engage with people in their home language and be adapted to the local context. In low-resource settings—both in research and practice—lines are often blurred between screening and diagnostic tools due to the lack of specialists and appropriate tools. Determining the validity of measurement tools and correctly interpreting that information is important to prevent tools from being used inappropriately. If we continue to use brief, self-report screening measures to make decisions about mental health service provision, it is important to confirm that these measures are valid and reliable in order to provide an accurate reflection of the mental health burden among adolescents and identify those at risk. This study found that the culturally adapted PHQ-9 tool had high sensitivity and specificity for both younger and older adolescents and could be used for population-level assessments of prevalence of depression among adolescents in the study setting. Further research is needed on adapting the GAD-7 to accurately capture the prevalence of generalized anxiety among adolescents in South Africa.

      Acknowledgments

      The authors are grateful to all the adolescents and their caregivers whose contributions made this study possible. The authors would like to acknowledge the hard work and perseverance of the Khayelitsha research team who conducted the study under exceptionally challenging circumstances: Athenkosi Mangele, Loyiso Ndamase, Luleka Sobekwa, Nonkululo Sikweyiya, Nozuko Matiso, Siphokazi Hlati, Vuyokazi Tasana, and Zwelebanzi Skiti, with the support of Zanele Siqabatiso and Zena Jacobs. Many members of this team were sick with COVID or lost family members due to COVID during the course of the study. The authors thank the various nongovernmental organizations and their staff for their partnership and support, both in referring adolescents to the study and for taking on new referrals identified through the study. Their services are invaluable to the Khayelitsha community. The authors would like to thank Prof. Jason Bantjes from Stellenbosch University for his input and guidance in establishing the suicide referral protocols for the study. The authors are grateful to the Khayelitsha Day Hospital and team, who assisted with imminent suicide referral cases.

      Funding Sources

      This work was funded by the UNICEF HQ through the “Data Strengthening Grant” provided by the Bill & Melinda Gates Foundation in 2017–2018. This work was supported, in whole or in part, by the Bill & Melinda Gates Foundation [ INV-001395 ]. Under the grant conditions of the Foundation, a Creative Commons Attribution 4.0 Generic License has already been assigned to the Author Accepted Manuscript version that might arise from this submission.

      References

        • Solmi M.
        • Radua J.
        • Olivola M.
        • et al.
        Age at onset of mental disorders worldwide: Large-scale meta-analysis of 192 epidemiological studies.
        Mol Psychiatry. 2022; 27: 281-295
        • Patel V.
        • Saxena S.
        • Lund C.
        • et al.
        The Lancet Commission on global mental health and sustainable development.
        Lancet. 2018; 392: 1553-1598
        • Erskine H.
        • Baxter A.
        • Patton G.
        • et al.
        The global coverage of prevalence data for mental disorders in children and adolescents.
        Epidemiol Psychiatr Sci. 2017; 26: 395-402
        • Jörns-Presentati A.
        • Napp A.-K.
        • Dessauvagie A.S.
        • et al.
        The prevalence of mental health problems in sub-Saharan adolescents: A systematic review.
        PLoS One. 2021; 16: e0251689
        • Kohrt B.A.
        • Kaiser B.N.
        Measuring mental health in humanitarian crises: A practitioner’s guide to validity.
        Confl Health. 2021; 15: 72
        • Hayes J.
        • Carvajal L.
        • Hijazi Z.
        • et al.
        You can’t manage what you do not measure – why adolescent mental health monitoring matters.
        J Adolesc Health. 2021; https://doi.org/10.1016/j.jadohealth.2021.04.024
        • Carvajal L.
        • Ahs J.W.
        • Requejo J.H.
        • et al.
        Measurement of mental health among adolescents at the population level: A multicountry protocol for adaptation and validation of mental health measures.
        J Adolesc Health. 2021; https://doi.org/10.1016/j.jadohealth.2021.11.035
        • Adjorlolo S.
        Generalised anxiety disorder in adolescents in Ghana: Examination of the psychometric properties of the Generalised anxiety disorder-7 scale.
        Afr J Psychol Assess. 2019; 1: a10
        • Anum A.
        • Adjorlolo S.
        • Kugbey N.
        Depressive symptomatology in adolescents in Ghana: Examination of psychometric properties of the patient health questionnaire-9.
        J Affect Disord. 2019; 256: 213-218
        • Buckley J.
        • Otwombe K.
        • Joyce C.
        • et al.
        Mental health of adolescents in the era of antiretroviral therapy: Is there a difference between HIV-infected and uninfected youth in South Africa?.
        J Adolesc Health. 2020; 67: 76-83
        • Fatiregun A.A.
        • Kumapayi T.E.
        Prevalence and correlates of depressive symptoms among in-school adolescents in a rural district in southwest Nigeria.
        J Adolesc. 2014; 37: 197-203
        • Osborn T.L.
        • Venturo-Conerly K.E.
        • Wasil A.R.
        • et al.
        Depression and anxiety symptoms, social support, and demographic factors among Kenyan high school students.
        J Child Fam Stud. 2020; 29: 1432-1443
        • Smit W.
        • de Lannoy A.
        • Dover R.V.
        • et al.
        The built environment and non-communicable diseases in Khayelitsha, Cape Town.
        Health Place. 2015; 35: 11-18
        • Cassidy T.
        • Lloyd S.
        • Bowman B.
        Violence, alcohol and symptoms of depression and in Cape Town’s poorest communities: Results of a community survey. UCT working Paper.
        (Available at:)
        https://open.uct.ac.za/handle/11427/28271
        Date accessed: October 23, 2022
        • Wolf E.J.
        • Harrington K.M.
        • Clark S.L.
        • et al.
        Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety.
        Educ Psych Meas. 2013; 76: 913-934
        • Kroenke K.
        • Spitzer R.L.
        • Williams J.B.
        The PHQ-9: Validity of a brief depression severity measure.
        J Gen Intern Med. 2001; 16: 606-613
        • Spitzer R.L.
        • Kroenke K.
        • Williams J.B.W.
        • Löwe B.
        A brief measure for assessing Generalized Anxiety Disorder: The GAD-7.
        Arch Intern Med. 2006; 166: 1092-1097
        • UNICEF
        Measuring mental health among adolescents at the population level (MMAP): An Overview. UNICEF, New York2021 (Available at:)
        • Kaufman J.
        • Birmaher B.
        • Brent D.
        • et al.
        Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): Initial reliability and validity data.
        J Am Acad Child Adolesc Psychiatry. 1997; 36: 980-988
        • American Psychiatric Association
        Diagnostic and statistical manual of mental disorders: DSM-5.
        5th ed. American Psychiatric Publishing, Washington2013
      1. Mental health atlas 2020. World Health Organization, Geneva2021 (Available at:)
        • World Health Organization
        Global Health Estimates: Life expectancy and leading causes of death and disability.
        (Available at:)
        • Tiirikainen K.
        • Haravuori H.
        • Ranta K.
        • et al.
        Psychometric properties of the 7-item Generalized Anxiety Disorder Scale (GAD-7) in a large representative sample of Finnish adolescents.
        Psychiatry Res. 2019; 272: 30-35
        • Kohrt B.A.
        • Jordans M.J.
        • Tol W.A.
        • et al.
        Validation of cross-cultural child mental health and psychosocial research instruments: Adapting the depression self-Rating scale and Child PTSD symptom scale in Nepal.
        BMC Psychiatry. 2011; 11: 127