If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Mental disorders are among the leading causes of disability among adolescents aged 10-19 years. However, data on prevalence of mental health conditions are extremely sparse across low- and middle-income countries, even though most adolescents live in these settings. This data gap is further exacerbated because few brief instruments for adolescent mental health are validated in these settings, making population-level measurement of adolescent mental health especially cumbersome to carry out. In response, the UNICEF has undertaken the Measurement of Mental Health Among Adolescents at the Population Level (MMAP) initiative, validating open-access brief measures and encouraging data collection in this area.
This protocol presents the MMAP mixed-methods approach for cultural adaptation and clinical validation of adolescent mental health data collection tools across settings. Qualitative activities include an initial translation and adaptation, review by mental health experts, focus-group discussions with adolescents, cognitive interviews, synthesis of findings, and back-translation. An enriched sample of adolescents with mental health problems is then interviewed with the adapted tool, followed by gold-standard semistructured diagnostic interviews.
The study protocol is being implemented in Belize, Kenya, Nepal, and South Africa and includes measures for anxiety, depression, functional limitations, suicidality, care-seeking, and connectedness. Analyses, including psychometrics, will be conducted individually by country and combined across settings to assess the MMAP methodological process.
This protocol contributes to closing the data gap on adolescent mental health conditions by providing a rigorous process of cross-cultural adaptation and validation of data collection approaches.
This protocol describes a rigorous mixed methods approach to ensure that tools adapted using the MMAP methodology have cultural and clinical relevance for the settings where they are used. A consistently used rigorous protocol like MMAP will allow for valid and reliable population-level comparisons of the prevalence of mental health conditions across settings. This protocol includes approaches for capturing information on mental health conditions on younger adolescents aged 10–14, an age group largely missing from many data collection efforts. The protocol was designed before COVID-19, meaning that modifications to validation steps to accommodate physical distancing or other preventive measures may be required.
Adolescence marks a period of dynamic physical growth and rapid brain development. It is when emotional learning, peer relationships, social influences, and increasing responsibility usher in a transition from childhood to greater independence. Mental health is fundamental to overall health and well-being during this sensitive period. But poor mental health during adolescence can also carry impacts that extend into adulthood [
]. Historically, though, mental health, including during adolescence, has been overlooked on the global development agenda.
The adoption of the United Nation’s Sustainable Development Goals (SDGs) in 2015, marks the first time that mental health has been included in a global accountability framework. SDG Target 3.4 calls for the reduction of premature mortality from noncommunicable diseases by one third by 2030 “through prevention and treatment, and promote mental health and well-being,” and Target 3.4.2 aims at reducing suicide mortality [
]. Staggeringly low data availability in regions like sub-Saharan Africa, parts of Asia and Latin America mean that the available data are representative of <1% of adolescents in these settings. Data collection efforts must be accelerated to fill this data gap on adolescent mental health in order to assess needs, respond appropriately, and be able to reach SDG targets. Such data are also essential to guide effective adolescent mental health policies and programs.
Collecting reliable mental health data across settings requires valid tools for measurement. In mental health, population-level measurement is conducted using scales that ask about symptoms. These scales need to be adjusted when used in different settings to account for nuances in language, local idioms, and expressions so that they accurately capture how adolescents express and experience symptoms [
For culturally adapted tools to be comparable across settings, validity must then ideally be established using a “gold-standard” measure that is calibrated across settings. Although the gold standard in mental health utilizes clinical diagnoses, lack of investment and resources dedicated to mental health means that only a limited number of tools for adolescent mental health are validated, and few have been conducted in LMICs, and even fewer across multiple LMIC settings. Collecting data at the country level using properly adapted and validated tools will contribute to the prioritization of evidence-based national policies and programs on adolescent mental health as well as prioritization for investment.
In response, United Nations Children’s Fund and partners established the Measurement of Mental Health Among Adolescents at the Population Level (MMAP) initiative [
]. The aim of MMAP is to improve quality and availability of adolescent mental health data. To this end, MMAP has developed and is testing a multistep standardized process for adaptation and gold-standard validation of adolescent mental health measures. These processes can be replicated to adapt and/or validate measures in new settings where the tools will be used. This work is being conducted initially in Belize, Kenya, Nepal, and South Africa, and involves validating measures for domains of adolescent mental health reflecting a core global indicator set [
The MMAP initiative supports the integration of these validated tools into national or subnational survey efforts, to encourage the collection of comparable and valid data on adolescent mental health, and enable assessments of trends over time and multicountry comparisons. This effort is a key step toward improved global monitoring of adolescent mental health.
This protocol paper describes the methodology for MMAP’s cross-cultural adaptation and validation procedures.
A mixed methods sequential design is used. The process consists of two phases, described in detail below. In the first phase, qualitative data collection activities are conducted to achieve cultural equivalence of the measures [
]. The objective of this phase is to achieve a simplification of the language and expressions that can be understood by the target population, across age groups, and taking in consideration various literacy levels, in the setting where the tool will be used.
Qualitative findings emerging from Phase 1 will be synthesized to inform the instruments used in Phase 2, the quantitative data collection. Phase 2 will entail quantitative gold-standard validation of the adapted measures using a clinical diagnostic assessment (Figure 1). In terms of the length of the process, it is expected to take about 6 months from beginning to end. Phase 1 can take about 2 months and Phase 2 about 3 months, including time for ethical approval, recruitment of participants training of relevant personnel like enumerators and clinicians. A month in between the two phases might be needed for logistics and coordination of aspects related to the second phase.
Step 1: translation and adaptation by bilingual experts.
Translation of the MMAP tools requires adaptation into suitable language and phrasing for the population and setting, rather than literal word-for-word translation which can result in wording with low comprehensibility for respondents. This step requires experienced translators working in English (the language of the original version of the tools) and to be fluent or native speakers of the target language where the tool is being tested. Translation can be done by an individual or by a group of translators. A group translation approach by three translators is suggested when possible as this allows for discussion among the group of translators to determine optimal terminology and resolve any differences in suggested wording by different translators.
Initial test tools will include the Revised Child Anxiety and Depression Scale-25 [
]. In addition, a suite of items from Patient-Reported Outcomes Measurement Information System item bank [reference] has also been tested in two of the validation sites. United Nations Children’s Fund–developed measures to collect data functional limitations, suicidal behavior, mental healthcare seeking, and connectedness will also be adapted and tested via the present MMAP protocol. This adaptation and validation protocol can be applied to any other tools that measure the same mental health conditions or additional conditions.
Step 2: mental health expert review
A team of 3–4 mental health clinicians with experience working with the target population and fluency in the target language then reviews the translated tool. This step ensures that the wording is easily and appropriately interpreted, and that symptoms and mental health–related concepts are appropriately conveyed in the translation. Experts may work together, or they may work independently to assess the translated instruments and then meet to discuss areas of discrepancy, ambiguous interpretation, or unclear phrasing. Experts make note of needs for revision using a TTA monitoring form [
]. The TTA form guides users through a critical assessment of each translated and adapted item through five equivalence domains (Box 1) in the local language and for the local context and documenting recommended changes.
Transcultural translation and adaptation equivalence domains
Comprehensibility (semantic equivalence): Review word choices for each item to ensure that they are understandable in the language of the participants and the target audience. This includes discussion about local idioms and phrasing.
Acceptability: Discuss whether respondents would find the item to be uncomfortable or offensive in any way, and whether the content of the question is acceptable by cultural, moral, and/or religious standards.
Relevance (content equivalence): Discuss whether an item and its description are relevant in the local culture, the target cultural group, and the living circumstances.
Completeness (criterion and conceptual equivalence): Discuss whether the culturally adapted version of the item captures the same concepts as the original item concept.
Technical equivalence: Discuss the methods of assessment, and whether they are comparable in the culture with respect to the data that they yield. For example, sentence structure; interrogative voice versus declarative; first person voice versus third person; question versus phrase; word choice for the levels; whether the items should have Likert scales, numbers, visuals, and so on.
A series of focus group discussions will be held to establish semantic, content, and technical equivalence of the mental health measures, and to gain an in-depth understanding of knowledge and beliefs surrounding mental health and its care.
Sample size and recruitment for focus group discussions
Four to eight focus groups will be conducted covering three types of respondents: (1) younger adolescents (10–14 years of age), (2) older adolescents (15–19 years of age), and (3) parents or caregivers of younger adolescents. The purpose of including parents in this validation is to gather evidence about the difference between direct proxy respondents and proxy respondents.
Adolescent participants meet in groups of 6–8 individuals, grouped according to age and gender, or as appropriate. Participants may be purposefully sampled based on recognition and recommendation by others (i.e., teachers or community leaders) for their potential willingness to discuss mental health issues, and also taking in consideration diversity in terms of varying levels of education, literacy, ethnicity, and socioeconomic background.
The focus group discussions are guided by the TTA approach mentioned above. If possible, perspectives of adolescents and caregivers of adolescents with lived experience increases the likelihood that terminology will reflect the local experiences. Focus group discussions are transcribed and coded for a priori TTA themes (Box 1) using content analysis.
Step 4: cognitive interviews
Cognitive interviews are conducted to garner participants’ understanding of the questionnaire items and response options, and to ensure alignment with the intended meaning of the items [
Sample size and recruitment for cognitive interviews
Cognitive interviews will be conducted one-on-one with 10–20 participants across target, such as males and females ages 10–14 and 15–19 and caregivers of younger adolescents in each country. Participants will be purposefully selected for fluency in the target language, and diversity representing varying levels of education, literacy, ethnicity, and socioeconomic background.
The cognitive interviews follow a semi-structured format discussing each item’s face and construct validity, assessing local understanding and appropriateness of items, explore how participants interpret items/questions, why they provide a specific response, and how items of interest fit in their life [
]. This process will be conducted using techniques such as “thinking aloud” and verbal probing to assess suitability and interpretation of the response options and understand the process whereby the response was chosen. To reduce participant burden, participants may be encouraged to focus on items they find difficult to understand or feel could be rephrased to improve clarity.
Interviews are recorded and transcribed, following the same TTA coding procedure as for focus groups. A TTA form will be used during interviews to document comments and suggestions during the interviews. Debriefing forms will be used for this purpose immediately following each interview.
Step 5: triangulation and integration of qualitative findings
Qualitative data gathered through activities described in Steps 1–4 are analyzed using qualitative methods as in Step 4, above. Findings are synthesized through framework analysis to guide recommendations for adaptation or modifications [
]. A revised, culturally adapted version of the questionnaire is then reviewed by the local research team.
Step 6: back translation
The back translator is an independent person who has not been involved in earlier processes of the project and who is not familiar with the original version of the tool. The back translation, as with the original translation, is intended to capture the conceptual equivalence of the item or phrase, rather than be word-for-word, and translates the tool back into the original language of the tool.
The research team reviews the back translation with local experts and those familiar with the tool, clearly noting and reconciling any differences or discrepancies between the back translation and the original tool and can be undertaken in an iterative process to refine the translation or correct any errors.
The final revised, culturally adapted version of the questionnaire for a specific language will then be collaboratively reviewed by the local team working in that language and the MMAP international research team.
Quantitative validation phase
The quantitative gold-standard validation phase comprises three steps: administration of the adapted tools, semi-structured diagnostic interviews by mental health professionals, and the psychometric evaluation of data.
For the gold-standard validation phase, survey questionnaires and clinical diagnostic interviews are administered to a sample of boys and girls across the age ranges of 10–14 and 15–19 years. The adolescent sample requires inclusion of an adequate number of persons with the target conditions (e.g., depression or anxiety) for comparison with “controls” who are presumed to not have the condition to ensure the possibility for the full range of psychometric analyses for validation against clinical diagnoses [
This is called an 'enriched sample' and for this protocol, a sample of 200 or more adolescents with symptoms of anxiety and/or depression is recommended.
The parent version of the questionnaire will be administered among a smaller sample of 100–175 parents/caregivers, depending on the site, of younger adolescents 10–14 years in at least one setting to assess the reliability of proxy report for younger participants.
Sample sizes of 300 or more for assessing psychometric stability of a measure are often recommended. The International Test Commission Guidelines for Translating and Adapting Tests (Second Edition) [
]. Efforts to test varied scenarios and data indicate sample sizes smaller than these “rules of thumb” may prove sufficient, while larger sample sizes typically increase power and precision. To balance demands of rigor with effective use of resources, 300 or more participants are suggested as suitable. Sample is also designed to have participants distributed across age and gender.
Recruitment for the quantitative validation data activities will be organized in collaboration with schools, community centers, nongovernmental organizations, and health clinics, depending on the setting. We will use the Community Case Detection Tool (CCDT) to garner referral of participants by teachers or community leaders. This approach will be used in Belize, Kenya, South Africa, and Nepal. The CCDT was developed in the context of mental health programs for LMICs with initial development in Nepal [
]. After a brief training, teachers or lay community people like youth program leaders will use the tool to assist in the referral of individuals with higher and lower likelihood for depression and anxiety for participation in the second phase of the MMAP process. This constitutes a two-stage sampling with higher sampling probability for screen positives. Depending on the context, snow balling sampling based on youth informants can also be used.
Step 7: administration of the adapted tools
This step will begin with training of enumerators to administer adapted tools. Taking in consideration target population diversity in terms of literacy levels, MMAP tools will be administered face-to-face by an interviewer at a location such as an office at a community center or clinic that affords privacy to the participants. Access to a counsellor will be arranged if participants experience distress or need support during or after the interview. A local procedure will offer adolescents referral, such as in the case of adolescents reporting current suicidal ideation, and a local emergency protocol will be followed if any participants experience distress or if care is needed urgently.
Step 8: diagnostic interview conducted by trained mental health professionals
A semi-structured diagnostic interview, the Kiddie Schedule for Affective Disorders and Schizophrenia (K-SADS-PL) (DSM-5) [
]. Participating clinicians will be trained in the administration of the K-SADS-PL and will complete a training process for ensuring inter-rater reliability. This is key for ensuring that clinicians are scoring participants in a consistent and comparable manner within and across settings. The components of the K-SADS-PL that will be administered may vary across sites, but will include, at a minimum, the depression module, and subsections of the anxiety module included generalized anxiety disorder, panic disorder, separation anxiety disorder, and social anxiety disorder.
Step 9: evaluation of psychometrics and validation analyses
Statistical analyses will be conducted to assess the psychometric characteristics for the MMAP tool and the performance of the tool using clinical diagnoses as the gold standard. These include diagnostic sensitivity and specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, diagnostic odds ratio, Youden’s index, area under the curve, and scale reliability [
]. These scores along with visualization of receiver operator characteristic curves will be used to determine an appropriate cut-off for intended use. For example, the need to detect as many adolescents as possible with a probable specific condition will require a cut-off with high sensitivity. However, when producing population estimates and determining cut-offs for health and education systems, it may be helpful to select a cut-off that balances sensitivity and specificity to avoid potential health system burden, costs, and stigmatization with high rates of false positives.
Positive predictive value of the Community Information Detection Tool will also be calculated in order to determine the potential utility of this tool for community-based proactive case finding and referral [
]. Data from countries and settings using the MMAP approach to TTA and validation of these tools will be combined for pooled psychometric analyses and to look at differential item functioning across settings and over time.
This work is being conducted in geographically and culturally diverse locations. Initial settings include Belize, Kenya, Nepal, and South Africa. In three of these settings the work will be conducted in a second language other than English: in Belize the work will also be conducted in Kriol, in Kenya the work will also be conducted in Kiswahili, and in South Africa the work will also be conducted in isiXhosa. In Nepal, the work will only be conducted in Nepali.
All participants will provide written informed consent, or written assent accompanied by parental consent for minors. The approaches and informed consent process reflect appropriate practices for the setting and target population. For example, material will be produced to ensure informed consent is suitably attained with lower literacy participants or their guardians and providing the youngest participants with simplified explanations of what they should expect if they agree to take part in the study activities.
In Kenya, South Africa, and Nepal, ethical approval will be provided by an ethical oversight board of the partner university, research institutions, or partner organization within each country. For the study activities in Belize, in addition to the Ethics board assembled by the Ministry of Health for this purpose, research ethics approval was granted by HML Institutional Review Board, an autonomous external ethics review committee in the United States authorized by the US Office for Human Research Protections within the US Department of Health and Human Services.
Providing resources and referrals
Ensuring privacy, safety, and convenience for participants is a priority. Referral procedures are developed specific to each setting in collaboration with local partner institutions, such as a Ministry of Health or local clinic, to offer support for individuals with a clinical level of distress or when otherwise indicated. Locally, appropriate protocols will be developed, in consultation and engagement with health providers in the country, to assist if any participants are experiencing acute distress or if care is needed urgently. Reflecting local experts and ethics board determinations, contact by a counselor within 24 hours may be offered, as well as informing caregivers of minors. Additionally, information regarding psychosocial support or mental health services may be provided to all individuals participating in the study.
The MMAP methodological approach has some limitations. First, it is based on clinical binary categories of adolescent depression and anxiety using existing criteria based on the psychiatric classification systems. There is increasing evidence that depression and anxiety exist along a continuum rather than a categorical absence versus presence [
] often called a dimensional approach. Therefore, future use of these tools could be for population level to capture continuum of mental health. Second, the adaptation and validation are conducted with samples of adolescents in a particular setting within countries, which might differ from other subpopulations, for example, urban versus rural or other geographic areas of the country. Third, the starting point DSM-5 and International Classification of Disease-11 categories for disorder. As more data are generated across different settings, we may find that diagnoses and diagnostic criteria for adolescents need to be refined. Therefore, the current approach is only part of broader and ongoing efforts needed to develop appropriate tools for global data collection related to adolescent mental health. Population-wide studies are needed to establish normative values for scores in a particular setting. Local health ministries and mental health programs can use population distributions to determine how best to allocate resources initially, for example, beginning with the 5% of the population with the highest scores as priority beneficiaries. Population norms are also helpful to refine cut-off scores because first steps, as described in this protocol, emphasize recruitment of “healthy” versus “distressed,” with adolescents on the borderline less likely to be included. Taken these limitations in consideration, the current process should be considered one step in a larger process to generate research in LMIC among adolescents and continuously refine procedures to identify who is most in need of support and treatment.
There are also a number of additional synergies and opportunities that can emerge from this work. MMAP methodologies could be adapted for use among vulnerable populations of adolescents like migrants, refugees, pregnant or adolescent mothers, adolescent living in institutions, sexual and gender minorities, and so on. This approach could also be adapted for use with younger populations. The tools that emerge from this effort can be tested for sensitivity to change for the purpose of using them for program evaluation purposes. This work is also timely and relevant given the new challenges the world is facing with COVID-19 pandemic and the impact on the mental health and well-being it is having on children and adolescents across the globe.
The validation of culturally adapted brief assessment tools for adolescent mental health conditions is essential for collecting valid and reliable data on mental health to guide prioritization of adolescent mental health policies, programming, and investment around the world. This protocol provides the methods that can be applied to arrive at such tools that can be used across populations and therefore allow for appropriate data collection, thus contributing to closing the data gap on adolescent mental health conditions. This is a rigorous process of cross-cultural adaptation and gold-standard validation of brief instruments that can be feasibly integrated into ongoing data collection efforts.
The authors are grateful to each member of the MMAP implementation teams and all of their supporting staff in Belize, Kenya, Nepal, and South Africa, for their hard work and commitment to implement the MMAP protocol. The authors are grateful to Priscilla Idele from the United Nations Children’s Fund Office of Research for her support in facilitating the Kenya field pilot work and technical guidance in the process. The authors are also grateful to Mark Jordans and Myrthe van den Broek from War Child Netherlands for making the CCDT available for use as part of the MMAP protocol. Katherine Ottman from George Washington University provided valuable inputs to qualitative aspects of the protocol. The authors acknowledge and thank members of the MMAP Technical Advisory Group for their contributions in shaping this work: Abiodun Adewuya, Claudia Cappa, Regina Guthold, Vikram Patel, George Patton, James Scott, Chiara Servili, Danuta Wasserman. The authors thank Bruce Chorpita for making the Revised Child Anxiety and Depression Scale tool available for use.
This work was funded by Bill and Melinda Gates Foundation – Data strengthening Grant provided to UNICEF HQ in 2017–2018. Field work in Kenya was funded by the Government of Italy, through the generous grant to the UNICEF Office of Research - Innocenti.