Advertisement

Rates, Patterns, and Predictors of Follow-up Care for Adolescents at Risk for Substance Use Disorder in a School-Based Health Center SBIRT Program

  • Timothy J. Ozechowski
    Correspondence
    Address correspondence to: Timothy J. Ozechowski, Ph.D., Division of Adolescent Medicine, Department of Pediatrics, University of New Mexico, MSC 10 5590, 1 University of New Mexico, Albuquerque, NM 87131-0001.
    Affiliations
    Division of Adolescent Medicine, Department of Pediatrics, University of New Mexico Health Sciences Center, Albuquerque, New Mexico

    Abt Associates, Division of Health and Environment, Albuquerque, New Mexico
    Search for articles by this author
  • Katherine Wilson
    Affiliations
    Division of Adolescent Medicine, Department of Pediatrics, University of New Mexico Health Sciences Center, Albuquerque, New Mexico
    Search for articles by this author

      Abstract

      Purpose

      To examine rates, patterns, and predictors of follow-up care for adolescents screened as being at risk for substance use disorder (SUD) in a school-based health center (SBHC) Screening, Brief Intervention and Referral to Treatment (SBIRT) program.

      Methods

      Electronic health records were extracted of adolescents who received health care services from one of three high school-based health centers implementing SBIRT. Patterns and predictors of engagement in follow-up care within 8 weeks following the week of a positive SUD risk screen were analyzed using item response theory (IRT) modeling.

      Results

      Out of 1,327 adolescents receiving SBHC services, 81.2% completed a health screening questionnaire. Of screened adolescents, 17.7% were positive for SUD risk. Across the 8-week follow-up period, 65.4% of adolescents at risk for SUD received at least one follow-up visit. IRT modeling indicated that high levels of engagement in follow-up care were characterized by contact with a behavioral health care (BHC) provider. The percentage of adolescents having follow-up contact with a BHC provider increased significantly after the onset of the COVID-19 pandemic. Engagement in follow-up care was predicted by risk for depression, history of suicidal behavior, being female, and previous sexual activity.

      Discussion

      SBHCs provide a favorable setting for screening and detecting adolescents at risk for SUD. Adolescents at risk for SUD should receive follow-up contact with a BHC provider. Enhanced follow-up engagement efforts may be warranted for adolescents at risk for SUD without risk for depression or suicidal history, as well as for females and those with previous sexual activity.

      Keywords

      Implications and Contribution
      This study examined rates, patterns, and predictors of follow-up care for adolescents screened as being at risk for substance use disorder (SUD) in school-based health centers implementing SBIRT. Results indicated that school-based health centers are conducive for screening a high proportion of adolescent patients and for detecting adolescents at risk for SUD. Contact with a behavioral health care provider is a central component of engagement in follow-up care.
      Over the past several decades, school-based health centers (SBHC) have emerged as a pivotal venue for providing health care services to youth and families in the United States [
      • Bronstein L.R.
      • Mason S.E.
      School-linked services: Promoting equity for children, families, and communities.
      ,
      • Kolbe L.J.
      School health as a strategy to improve both public health and education.
      ,
      • Lee R.M.
      • Gortmaker S.L.
      School interventions to support health behavior change.
      ,
      Substance Abuse and Mental Health Services Administration
      Ready, set, go review: Screening for behavioral health risk in school.
      ,
      • Kjolhede C.
      • Lee A.C.
      • Duncan De Pinto C.
      • et al.
      School-based health centers and pediatric practice.
      ,
      • Dunfee M.N.
      School-based health centers in the United States: Roots, reality, and potential.
      ,
      • Keeton V.
      • Soleimanpour S.
      • Brindis C.D.
      School-based health centers in an era of health care reform: Building on history.
      ,
      • Love H.E.
      • Schlitt J.
      • Soleimanpour S.
      • et al.
      Twenty years of school-based health care growth and expansion.
      ,
      • Weitzman C.
      • Wegner L.
      • The Section on Developmental and Behavioral Pediatrics
      • et al.
      Promoting optimal development: screening for behavioral and emotional problems.
      ,
      Council on School Health
      School-based health centers and pediatric practice.
      ]. SBHCs are freestanding clinics embedded in elementary, middle, and high schools, offering a range of primary care and behavioral health services. Since the late 1990s, the number of SBHCs in the United States has increased from approximately 1,100 to over 2,500, with the highest rate of the increase occurring in rural communities [
      • Love H.E.
      • Schlitt J.
      • Soleimanpour S.
      • et al.
      Twenty years of school-based health care growth and expansion.
      ]. Studies have shown SBHCs to be effective in increasing rates of health care service utilization, as well as improving medical and behavioral health outcomes for underserved youth [
      • Kjolhede C.
      • Lee A.C.
      • Duncan De Pinto C.
      • et al.
      School-based health centers and pediatric practice.
      ,
      • Dunfee M.N.
      School-based health centers in the United States: Roots, reality, and potential.
      ,
      • Love H.E.
      • Schlitt J.
      • Soleimanpour S.
      • et al.
      Twenty years of school-based health care growth and expansion.
      ,
      • Allison M.A.
      • Crane L.A.
      • Beaty B.L.
      • et al.
      School-based health centers: Improving access and quality of care for low-income adolescents.
      ,
      • Arenson M.
      • Hudson P.J.
      • Lee N.
      • et al.
      The evidence on school-based health centers: A review.
      ,
      • Bersamin M.
      • Coulter R.W.S.
      • Gaarde J.
      • et al.
      School-based health centers and school connectedness.
      ,
      • Choi K.R.
      • Easterlin M.C.
      Intervention models for increasing access to behavioral health services among youth: A systematic review.
      ,
      • Gibson E.J.
      • Santelli J.S.
      • Minguez M.
      • et al.
      Measuring school health center impact on access to and quality of primary care.
      ,
      • Knopf J.A.
      • Finnie R.K.C.
      • Peng Y.
      • et al.
      School-based health centers to advance health equity.
      ].
      Adolescent health experts have become increasingly interested in the utility of SBHCs for implementing health screening and intervention protocols, particularly the Screening, Brief Intervention, and Referral to Treatment (SBIRT) model for adolescents at risk for substance use disorder (SUD) [
      • Harris B.R.
      Communicating about screening, brief intervention, and referral to treatment: Messaging strategies to raise awareness and promote voluntary adoption and implementation among New York school-based health center providers.
      ,
      • Harris S.K.
      • Aalsma M.C.
      • Weitzman E.R.
      • et al.
      Research on clinical preventive services for adolescents and young adults: Where are we and where do we need to go?.
      ,
      • Mitchell S.G.
      • Gryczynski J.
      • Gonzales A.
      • et al.
      Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based program: Services and outcomes.
      ]. To date, SBIRT services for adolescents have been geared primarily for delivery in pediatric primary care (PPC) clinics [
      • Levy S.J.L.
      • Williams J.F.
      Committee on Substance Use and Prevention
      Substance use screening, brief intervention, and referral to treatment.
      ,
      • Bagley S.
      • Shrier L.
      • Levy S.
      Talking to adolescents about alcohol, drugs and sexuality.
      ]. However, studies have shown that behavioral health concerns such as substance use are the focus of a large proportion of visits with SBHC providers and that access to an SBHC increases the likelihood of youth receiving behavioral health care [
      • Keeton V.
      • Soleimanpour S.
      • Brindis C.D.
      School-based health centers in an era of health care reform: Building on history.
      ,
      • Gibson E.J.
      • Santelli J.S.
      • Minguez M.
      • et al.
      Measuring school health center impact on access to and quality of primary care.
      ,
      • Bains R.M.
      • Diallo A.F.
      Mental health services in school-based health centers: Systematic review.
      ,
      • Bains R.M.
      • Cusson R.
      • White-Frese J.
      • et al.
      Utilization of mental health services in school-based health centers.
      ,
      • Koenig K.T.
      • Ramos M.M.
      • Fowler T.T.
      • et al.
      A statewide profile of frequent users of school-based health centers: Implications for adolescent health care.
      ]. Moreover, youth exposed to SBIRT services in school settings report favorable attitudes and positive experiences with screening and intervention [
      • Chadi N.
      • Levy S.
      • Wisk L.E.
      • et al.
      Student experience of school screening, brief intervention, and referral to treatment.
      ,
      • Curtis B.L.
      • McLellan A.T.
      • Gabellini B.N.
      Translating SBIRT to public school settings: An initial test of feasibility.
      ,
      • McCarty C.A.
      • Gersh E.
      • Katzman K.
      • et al.
      Screening and brief intervention with adolescents with risky alcohol use in school-based health centers: A randomized clinical trial of the check yourself tool.
      ,
      • Maslowsky J.
      • Schulenberg J.E.
      Interaction matters: Quantifying conduct problem × depressive symptoms interaction and its association with adolescent alcohol, cigarette, and marijuana use in a national sample.
      ,
      • Gryczynski J.
      • Mitchell S.G.
      • Schwartz R.P.
      • et al.
      Computer- vs. nurse practitioner-delivered brief intervention for adolescent marijuana, alcohol, and sex risk behaviors in school-based health centers.
      ]. Accordingly, SBHCs may be conducive settings for engaging adolescents in SBIRT services, particularly in follow-up care after a positive SUD risk screen.
      The purpose of this study is to examine rates of screening and detection of SUD risk, as well as patterns and predictors of follow-up care in SBHCs implementing the SBIRT model. To date, empirical evaluations of SBIRT in SBHCs have focused on the efficacy of brief interventions for reducing substance use behavior [
      • Mitchell S.G.
      • Gryczynski J.
      • Gonzales A.
      • et al.
      Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based program: Services and outcomes.
      ,
      • McCarty C.A.
      • Gersh E.
      • Katzman K.
      • et al.
      Screening and brief intervention with adolescents with risky alcohol use in school-based health centers: A randomized clinical trial of the check yourself tool.
      ,
      • Gryczynski J.
      • Mitchell S.G.
      • Schwartz R.P.
      • et al.
      Computer- vs. nurse practitioner-delivered brief intervention for adolescent marijuana, alcohol, and sex risk behaviors in school-based health centers.
      ]. This study is the first to examine rates and patterns of follow-up care, as well as adolescent characteristics predicting engagement in follow-up care with SBHC providers.

      Methods

      All study procedures were approved by the University of New Mexico Health Sciences Center Human Research Review Committee (approval 19-213).

      Data

      The data were extracted from electronic health records (EHR) of adolescent patients who received services from one of three high school-based health centers implementing the SBIRT model. All three SBHCs were in the same school district and served student populations between 1,000 and 2,000 students. Each SBHC was staffed with one primary care provider (i.e., physician assistant, nurse practitioner), one licensed behavioral health therapist, one case manager, and two medical assistants.
      The EHR data set included adolescent demographics, health care service utilization records, as well as information from an annual comprehensive health screening instrument, which was administered on a wireless tablet device. Demographic information was obtained from all adolescents during routine patient intake procedures. Data were extracted for all adolescent patients screened during the following time periods: (a) August 1, 2019, through December 31, 2019, and (b) August 1, 2020, through March 26, 2021. The end dates of each period were pinpointed to allow for an 8-week follow-up period beginning the week following a positive SUD risk screen (excluding periods in which the SBHCs were closed, such as holiday breaks). The study sample did not include adolescents screened between January 1, 2020, and July 31, 2020, given SBHC service restrictions stemming from the onset of the COVID-19 pandemic. All three high schools were closed from March 27, 2020, through the remainder of the 2019–2020 school year. The schools reopened on a full-time remote basis in August 2020. SBHCs remained open and operational during periods of school closures and remote learning; however, most patient visits were conducted remotely using telehealth.

      Measures

      SUD risk

      Risk for SUD was assessed using the CRAFFT 2.0 (Car, Relax, Alone, Friends/Family, Forget, Trouble) screener, which is a well-validated measure of risk for SUD in SBIRT programs [
      • Bagley S.
      • Shrier L.
      • Levy S.
      Talking to adolescents about alcohol, drugs and sexuality.
      ,
      • Levy S.
      • Sherritt L.
      • Harris S.K.
      • et al.
      Test-retest reliability of adolescents’ self-report of substance use.
      ]. The CRAFFT contains six yes/no items assessing various types and aspects of substance use behavior indicating risk for SUD (e.g., using alcohol or drugs while alone, forgetting things while using alcohol or drugs). Scores of two or greater on the CRAFFT are considered positive for SUD risk and trigger more thorough assessment and intervention as needed.

      Risk for depression

      The risk for depression was measured using the Patient Health Questionnaire-2 (PHQ-2), which is a widely used and well-established depression screener [
      • Levis B.
      • Sun Y.
      • He C.
      • et al.
      Accuracy of the PHQ-2 alone and in combination with the PHQ-9 for screening to detect major depression: Systematic review and meta-analysis.
      ,
      • Staples L.G.
      • Dear B.F.
      • Gandy M.
      • et al.
      Psychometric properties and clinical utility of brief measures of depression, anxiety, and general distress: The PHQ-2, GAD-2, and K-6.
      ]. The PHQ-2 comprises two items with four-point ordinal response scales ranging from 0 to 3, indicating the degree to which patients experienced depressed mood and anhedonia over the past 2 weeks. A total score is obtained by summing the item scores. We used a PHQ-2 score of 2 as the cutoff for risk for depression based on recommendations by Bentley et al. [
      • Bentley K.H.
      • Sakurai H.
      • Lowman K.L.
      • et al.
      Validation of brief screening measures for depression and anxiety in young people with substance use disorders.
      ] in a study of depression among adolescents with SUD.

      Health behaviors

      The health screening questionnaire included a set of yes/no items assessing adolescent engagement in specific health-related behaviors in a typical day, including (a) at least 1 hour of physical activity, (b) more than 2 hours of screen time, (c) at least five servings of fruits and vegetables, and (d) at least 8 hours sleep.

      Psychosocial risk and protective factors

      The health screening questionnaire included a set of individual yes/no items measuring psychosocial risk and protective factors, including whether adolescents (a) ever had sex, (b) ever attempted suicide, (c) ever attempted nonsuicidal self-injury, (e) have problems with arguing or fighting at home, (f) have problems at school with grades, (g) have problems at school with attendance, (h) are able to talk with a parent, (i) are able to talk with a close friend, (j) are able to talk with a teacher, and (k) have no one with whom they can talk.

      Engagement in follow-up care

      For adolescents with a positive SUD risk screen, we documented whether a follow-up visit occurred with an SBHC provider during each week of the 8-week follow-up period, and if so, whether such contact was with a primary care (PC) provider, a behavioral health care (BHC) provider, or both. We assigned a follow-up contact score to each of the 8 weeks subsequent to the week of the positive screen, where 0 = no follow-up contact, 1 = follow-up solely with a PC provider only, 2 = follow-up with a BHC provider (either solely or in addition to a PC provider).

      Statistical approach

      To model patterns and predictors of engagement in follow-up care, we used an application of item response theory (IRT) modeling demonstrated by Hedeker et al. [
      • Hedeker D.
      • Mermelstein R.J.
      • Flay B.R.
      Application of item response theory models for intensive longitudinal data.
      ]. This approach entailed treating each of the 8 weeks subsequent to the week of a positive SUD risk screen as “items” measuring adolescents’ engagement in follow-up care. We assigned a “response” to each “item” using the three-point follow-up contact scale described previously. Patterns of follow-up contact over the 8-week follow-up period were estimated using the following IRT model for nominal responses.
      Pr(Ypw=i|θp,αiw,βiw)=exp(αiwθpβiw)1+ exp(αiwθpβiw)
      (1)


      Due to space limitations, a full description of and rationale for the IRT model is provided in section S.1 of the Supplemental Materials.

      Results

      As shown in Table 1, 1,327 adolescents received SBHC services on at least one occasion during the study. Of these, 1,077 (81.2%) completed the health screening questionnaire, and 191 (17.7%) produced a positive result on the CRAFFT. Of all screened patients, 91 (6.9%) were screened multiple times during the study period. For patients with multiple screens, only the initial screen was included in the analysis.
      Table 1Number of adolescents with one or more SBHC visits, number of screened adolescents, and number of CRAFFT-positive results pre-COVID-19 and post-COVID-19 onset
      Patient groupPre-COVID-19 onset
      The pre-COVID-19 study period is 8/1/2019 to 12/31/2019.
      Post-COVID-19 onset
      The post-COVID-19 study period is 8/1/2020 to 3/26/2021.
      Total
      Adolescents with one or more SBHC visits8724551,327
      Adolescents screened on the CRAFFT6883891,077
      Adolescents with positive CRAFFT screens13754191
      a The pre-COVID-19 study period is 8/1/2019 to 12/31/2019.
      b The post-COVID-19 study period is 8/1/2020 to 3/26/2021.
      After the onset of the COVID-19 pandemic, the number of adolescents with one or more SBHC visits declined by 47.8%, the number completing the health screening questionnaire declined by 43.4%, and the number with a positive result on the CRAFFT declined by 60.6% compared to the pre-COVID-19 period of the study. The percentage of patients completing the health screening questionnaire increased somewhat between the pre-COVID-19 and post-COVID-19 periods of the study (78.9% vs. 85.5%, respectively). The rate of positive CRAFFT screens declined from 19.9% to 13.9% over the same period.

      Missing data

      Across the 1,077 screened adolescents, the percentage of missing data values ranged from 0.0% to 8.5%. Missing data were addressed using multiple imputations [
      • Rubin D.B.
      Multiple imputation for nonresponse in surveys.
      ,
      • Little R.J.A.
      • Rubin D.B.
      Statistical analysis with missing data.
      ]. See section S.2 of the Supplemental Materials for further details.

      Characteristics of adolescent patients at risk for SUD

      The mean age of adolescents completing the health screening questionnaire was 15.7 years (SD = 1.2), which did not differ significantly between those with positive and negative results on the CRAFFT. Table 2 presents additional demographic and behavioral/clinical characteristics of screened adolescents stratified by risk for SUD based on the CRAFFT. Demographically, adolescents with a positive CRAFFT screen were more likely to identify as LGBQ (lesbian, gay, bisexual, or questioning) and were marginally more likely to be Medicaid recipients than those with a negative CRAFFT screen. On nearly all of the behavioral/clinical characteristics measured in this study, adolescents with a positive CRAFFT screen exhibited significantly poorer health status on average, including risk for depression, history of self-harm and suicidal behavior, conflictual home environments, and problems at school with grades and attendance than those with a negative CRAFFT screen. Moreover, adolescents with a positive CRAFFT screen were at greater physical risk stemming from less daily engagement in healthy behaviors such as proper nutrition, physical activity, and sleep and from having more than 2 hours of screen time per day. Finally, in terms of social support, adolescents with a positive CRAFFT screen were half as likely as their counterparts to report being able to talk with a parent and over twice as likely to report being able to talk to no one.
      Table 2Percentages of adolescents in demographic and behavioral/clinical categories by CRAFFT screening result
      CategoryPatient group
      All values in the patient group columns are percentages.
      Test of CRAFFT-positive versus CRAFFT-negative
      All adolescents (n = 1,077)CRAFFT-negative (n = 886)CRAFFT-positive (n = 191)χ2(1)p value
      Demographic
       Female66.765.771.22.15.142
       Hispanic (any race)59.159.656.50.60.437
       Native American10.310.310.50.01.934
       Black9.79.89.40.03.867
       White14.214.015.20.18.670
       Lesbian, gay, bisexual, or questioning22.219.236.126.11.000
       Medicaid recipient61.760.567.53.30.069
      Behavioral/Clinical
       At risk for depression40.933.773.8104.43.000
       Previous self-harm22.716.750.8103.85.000
       Previous suicide attempt(s)16.812.238.276.15.000
       Previous sexual activity46.739.878.594.50.000
       Usually at least five servings of fruits and vegetables every day38.342.817.343.25.000
       Usually at least 8 hours sleep every night56.762.231.460.63.000
       Usually at least 1 hour of physical activity every day76.778.966.513.52.000
       Usually more than 2 hours of screen time every day80.078.288.510.36.001
       Problems at home with arguing/fighting7.14.917.839.68.000
       Problems at school with grades21.017.039.346.81.000
       Problems at school with attendance7.95.220.450.12.000
       Can really talk with a friend72.773.569.11.51.219
       Can really talk with a parent47.452.325.146.37.000
       Can really talk with a teacher14.714.714.70.00.996
       Can really talk with no one5.14.010.513.79.000
      All race categories are non-Hispanic.
      a All values in the patient group columns are percentages.

      Rates of follow-up care for adolescents at risk for SUD

      Table 3 presents percentages of adolescents receiving any type of follow-up contact, those receiving follow-up contact solely from a PC provider, and those receiving follow-up contact from a BHC provider (possibly in addition to a PC provider) during the entire study and during the pre-COVID-19 and post-COVID-19 periods. During the first 4 weeks of the follow-up period, 56.0% of adolescents with a positive CRAFFT screen received at least one follow-up visit with an SBHC provider, increasing to 65.4% across the entire 8-week follow-up period. The percentage of adolescents receiving any type of follow-up contact was stable between the pre-COVID-19 and post-COVID-19 study periods. However, the percentage of adolescents having follow-up contact with a BHC provider was significantly higher, whereas those with follow-up contact solely with a PC provider were significantly lower during the post-COVID-19 period of the study compared to the pre-COVID-19 period.
      Table 3Percentage of adolescents receiving any type of follow-up care, follow-up care with a primary care provider only, and follow-up care with a behavioral health care provider stratified by study period
      Number of weeks after a positive CRAFFT screenType of follow-upStudy periodTest of pre-COVID-19 versus post-COVID-19
      Entire studyPre-COVID-19
      The pre-COVID-19 study period is 8/1/2019 to 12/31/2019.
      Post-COVID-19
      The post-COVID-19 study period is 8/1/2020 to 3/26/2021.
      χ2(1)p value
      4Any follow-up56.056.255.60.0.9352
      Primary care provider only19.424.85.69.2.0024
      Behavioral health care provider36.631.450.05.8.0162
      8Any follow-up65.466.463.00.2.6506
      Primary care provider only25.732.19.310.6.0011
      Behavioral health care provider39.834.353.76.1.0136
      a The pre-COVID-19 study period is 8/1/2019 to 12/31/2019.
      b The post-COVID-19 study period is 8/1/2020 to 3/26/2021.
      For all adolescents with a positive CRAFFT screen, the mean number of follow-up visits with SBHC providers was 1.2 (SD = 1.4; range: 0–7) during the first 4 weeks of the follow-up period and 2.3 (SD = 2.6; range: 0–12) across the entire 8-week follow-up period. Among adolescents with at least one follow-up visit, these mean values were 2.1 (SD = 1.3; range: 1–7) and 3.4 (SD = 2.5; range: 1–12), respectively.

      Patterns of follow-up care for adolescents at risk for SUD

      We used the IRT model shown in (1) to analyze patterns of follow-up contact over the 8-week follow-up period. Figure 1 displays plots of the item characteristic curves (ICC) for each week of the follow-up period. The y-axis in each plot displays the probability of follow-up contact during a given week w (w = 1,. . ., 8). The x-axis displays values of the θ dimension in the IRT model, which is a latent variable capturing levels of adolescent engagement in follow-up care (see section S.1), ranging from −2 to +2 standard deviations with the mean being equal to 0. ICCs with relatively high locations on θ (i.e., the value of θ at which probability of follow-up contact in category i during week w equals .5) indicate types of follow-up contact characterizing highly engaged adolescents and vice-versa. ICCs with relatively steep slopes indicate types of follow-up contact that distinguish between adolescents at high and low levels of engagement.
      Figure thumbnail gr1
      Figure 1Item characteristic curves for each type of follow-up contact during each week of the follow-up period.
      The plots in Figure 1 indicate that levels of adolescent engagement in follow-up care are strongly associated with follow-up contact with a BHC provider, as evidenced by the sharply positive slopes of the corresponding ICCs. Specifically, during all weeks of the follow-up period, the probability of follow-up contact with a BHC provider is approximately zero across all negative values of θ and begins to increase sharply around θ equal to 0, reaching its peak at approximately θ equal to 2. This ICC slope pattern suggests that adolescents at above-average levels of engagement (θ > 0) are more likely to have a follow-up visit with a BHC provider than adolescents at below-average levels of engagement (θ < 0) during any given week of the follow-up period. In contrast, the ICCs corresponding to follow-up contact solely with a PC provider have relatively flat slopes and no clear location on the θ dimension during any week of the follow-up period, indicating that follow-up contact solely with a PC provider is unlikely for adolescents across all levels of θ and provides little information about levels of adolescent engagement in follow-up care.
      To further illustrate the association between levels of adolescent engagement in follow-up care and patterns of follow-up contact, Figure 2 presents estimated probabilities of each type of follow-up contact across all weeks of the follow-up period for values of θ equal to −1, 0, and 1, representing low, average, and high levels of adolescent engagement, respectively. Consistent with information conveyed by the ICCs, high levels of adolescent engagement in follow-up care (the right-hand panel in Figure 2) are distinguished from low and average levels of engagement (the left-hand and middle panels in Figure 2, respectively) by markedly higher probabilities of follow-up contact with a BHC provider across all 8 weeks of the follow-up period. The highest probability of follow-up contact with a BHC provider at θ = 1 is during week 2 of the follow-up period (0.53), after which this probability decreases steadily, reaching its lowest value during week 8 (0.22). In contrast, during each week of the follow-up period, the probability of follow-up contact with a BHC provider is less than 0.10 at θ = 0 and is virtually zero at θ = −1. Conversely, the probability of having no follow-up contact ranges from 0.82 to 0.97 across all 8 weeks of the follow-up period at θ ≤ 0. This range is markedly lower at θ = 1, going from 0.45 to 0.70. The probability of follow-up contact solely with a PC provider does not exceed 0.15 during any week of the follow-up period for all three levels of adolescent engagement.
      Figure thumbnail gr2
      Figure 2Probabilities of follow-up contact for 8 weeks following the week of a positive screen for risk of substance use disorder associated with low, average, and high levels of engagement in follow-up care.

      Predictors of engagement in follow-up contact for adolescents at risk for SUD

      We expanded the IRT model in (1) to include adolescent age and all demographic and behavioral/clinical characteristics listed in Table 2 as predictors of levels of adolescent engagement in follow-up care. We tested each predictor separately in a preliminary set of IRT regression models. From these individual regressions, we retained statistically significant predictors (p < .10) exhibiting a moderately sized effect, defined as a standardized regression coefficient of |.30| or greater, and estimated their conjoint effects on adolescent engagement in a multiple regression expansion of the baseline IRT model.
      Given previously reported differences in follow-up rates before and after the onset of the COVID-19 pandemic (Table 3), the first predictor variable we tested was whether the positive CRAFFT screen occurred during the post-COVID-19 period of the study (coded 0 = no and 1 = yes). The estimate of this effect was statistically significant in a positive direction and large in magnitude (γˆ = 0.58, SE = 0.20, df = 190, t = 2.95, p = .0035), indicating that adolescents with a positive CRAFFT screen after the COVID-19 onset exhibited higher levels of engagement in follow-up care on average than those with a positive CRAFFT screen prior to the COVID-19 onset. Given this finding, we retained the COVID-19 indicator variable as a covariate in all subsequent regression analyses.
      The results of the individual regression analyses are shown in Table S.1 of the Supplemental Materials. The individual predictors with estimates meeting our requirements for inclusion in the multiple regression IRT model were as follows: previous suicide attempt(s), at risk for depression, previous sexual activity, previous self-harm, problems at home with arguing/fighting, problems at school with grades, female, and Hispanic ethnicity. These predictor variables were added sequentially to the multiple regression component of the IRT model in decreasing order of standardized regression coefficient (see Table S.1). In the final multiple regression model we retained predictors with a statistically significant (p < .05) standardized regression coefficient of |.30| or greater. The final set of results is presented in Table 4. Controlling for the effects of the COVID-19 onset, adolescents reporting previous suicide attempt(s) and those at risk for depression had higher estimated levels of engagement in follow-up care on average compared to adolescents without these characteristics. Furthermore, female adolescents and those reporting previous sexual activity had lower estimated levels of engagement in follow-up care on average compared to male adolescents and those reporting no previous sexual activity.
      Table 4Results of the final multiple regression IRT model
      PredictorEstimateSEt-value
      Degrees of freedom for all tests equal 190.
      p value
      Post-COVID-19 CRAFFT screen0.670.203.29.0012
      Previous suicide attempt(s)0.520.192.75.0066
      At risk for depression0.480.222.19.0297
      Previous sexual activity−0.580.23−2.54.0120
      Female−0.450.20−2.26.0249
      SE = standard error.
      a Degrees of freedom for all tests equal 190.

      Discussion

      The results of this study have several implications for the implementation of SBIRT in SBHCs. First, SBHCs are appropriate and effective settings for screening adolescents for SUD risk. In our study, 81.2% of adolescents utilizing SBHC services completed the health screening questionnaire, including the CRAFFT. This screening rate is comparable to that reported in PPC clinics implementing SBIRT [
      • Mitchell S.G.
      • Gryczynski J.
      • Schwartz R.P.
      • et al.
      Adolescent SBIRT implementation: Generalist vs. specialist models of service delivery in primary care.
      ,
      • Sterling S.
      • Kline-Simon A.H.
      • Satre D.D.
      • et al.
      Implementation of screening, brief intervention, and referral to treatment for adolescents in pediatric primary care: A cluster randomized trial.
      ,
      • Monico L.B.
      • Mitchell S.G.
      • Dusek K.
      • et al.
      A comparison of screening practices for adolescents in primary care after implementation of screening, brief intervention, and referral to treatment.
      ]. Second, a substantial proportion of adolescents at risk for SUD can be identified via self-report screening in SBHCs. In the current study, 17.6% of adolescent patients completing the health screener yielded a positive CRAFFT result, which is consistent with corresponding rates reported in other studies of SBIRT implementation in SBHCs [
      • Mitchell S.G.
      • Gryczynski J.
      • Gonzales A.
      • et al.
      Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based program: Services and outcomes.
      ,
      • Gryczynski J.
      • Mitchell S.G.
      • Schwartz R.P.
      • et al.
      Computer- vs. nurse practitioner-delivered brief intervention for adolescent marijuana, alcohol, and sex risk behaviors in school-based health centers.
      ] and higher than those reported in PPC clinics implementing SBIRT [
      • Sterling S.
      • Kline-Simon A.H.
      • Satre D.D.
      • et al.
      Implementation of screening, brief intervention, and referral to treatment for adolescents in pediatric primary care: A cluster randomized trial.
      ,
      • Monico L.B.
      • Mitchell S.G.
      • Dusek K.
      • et al.
      A comparison of screening practices for adolescents in primary care after implementation of screening, brief intervention, and referral to treatment.
      ].
      Third, adolescents identified as being at risk for SUD in SBHCs are a vulnerable population exhibiting an array of adversities and health challenges. Specifically, adolescents with a positive CRAFFT screen were more likely than those with a negative CRAFFT screen to identify as LGBQ, to be Medicaid recipients, to be at risk for depression, as well as to report previous self-harm and suicidal behavior, problems at home and school, less engagement in daily physically healthy behaviors, and fewer social supports. These results are consistent with research showing that the onset and progression of substance use during adolescence typically is preceded by psychosocial and behavioral risk factors, the presence of which increases the risk for SUD [
      • Maslowsky J.
      • Schulenberg J.E.
      Interaction matters: Quantifying conduct problem × depressive symptoms interaction and its association with adolescent alcohol, cigarette, and marijuana use in a national sample.
      ,
      • Farmer R.F.
      • Seeley J.R.
      • Kosty D.B.
      • et al.
      Internalizing and externalizing psychopathology as predictors of cannabis use disorder onset during adolescence and early adulthood.
      ,
      • Griffith-Lendering M.F.H.
      • Huijbregts S.C.J.
      • Mooijaart A.
      • et al.
      Cannabis use and development of externalizing and internalizing behaviour problems in early adolescence: A TRAILS study.
      ,
      • Hussong A.M.
      • Ennett S.T.
      • Cox M.J.
      • et al.
      A systematic review of the unique prospective association of negative affect symptoms and adolescent substance use controlling for externalizing symptoms.
      ,
      • Khoddam R.
      • Jackson N.J.
      • Leventhal A.M.
      Internalizing symptoms and conduct problems: Redundant, incremental, or interactive risk factors for adolescent substance use during the first year of high school?.
      ,
      • O’Neil K.A.
      • Conner B.T.
      • Kendall P.C.
      Internalizing disorders and substance use disorders in youth: Comorbidity, risk, temporal order, and implications for intervention.
      ]. Given the range of adverse health conditions characterizing adolescents at risk for SUD, engaging such adolescents in follow-up care should be standard practice in SBIRT programs.
      Our IRT analysis of patterns of follow-up contact suggests that adolescents at risk for SUD exhibiting relatively high levels of engagement in follow-up care are those with at least one visit with a BHC provider over the 8-week period following the week of a positive CRAFFT screen. For such adolescents, contact with a BHC provider is most likely to occur during the second week of the follow-up period, after which this probability decreases steadily. In contrast, adolescents at low and average levels of engagement are significantly more likely to have no follow-up contact during any given week, and therefore, across all weeks of the follow-up period. Follow-up contact solely with a PC provider does not appear to be a strong indicator of engagement in follow-up care. Therefore, in keeping with prevailing models of collaborative care in SBHCs [
      Council on School Health
      School-based health centers and pediatric practice.
      ,
      • Stephan S.
      • Mulloy M.
      • Brey L.
      Improving collaborative mental health care by school-based primary care and mental health providers.
      ], providing adolescents at risk for SUD with follow-up care from a BHC provider should be prioritized in SBIRT programs implemented in SBHCs.
      We found that the proportion of adolescents with a positive CRAFFT screen receiving follow-up care from a BHC provider increased significantly after the onset of the COVID-19 pandemic. Although this increase may be due to a number of factors, a primary contributor is likely to be the substantial drop in overall patient volumes after the COVID-19 onset, and consequently, the number of adolescents in need of follow-up care due to a positive CRAFFT screen. This finding suggests that increasing the number of BHC providers in SBHCs would facilitate behavioral health follow-up care for such adolescents as patient volumes gradually recover toward pre-COVID-19 levels. However, simply retaining, let alone increasing the number of BHC providers, may be quite challenging in many SBHC settings for numerous reasons (e.g., limited resources, lack of available BHCs). Therefore, in lieu of (or in addition to) increasing the number of BHC providers, more frequent and systematic use of telehealth, which was necessitated in SBHCs subsequent to the COVID-19 onset, may facilitate a larger proportion of adolescents at risk for SUD receiving behavioral health follow-up care [
      • Love H.
      • Panchal N.
      • Schlitt J.
      • et al.
      The use of telehealth in school-based health centers.
      ,
      • Goddard A.
      • Sullivan E.
      • Fields P.
      • et al.
      The future of telehealth in school-based health centers: Lessons from COVID-19-19.
      ,
      • Sullivan E.
      • Brey L.
      • Soleimanpour S.
      School-based health center operations during the COVID-19-19 pandemic: A preliminary study.
      ].
      Adolescents at risk for SUD at low to average levels of engagement in follow-up care may warrant enhanced outreach and engagement efforts. Although it is not possible to definitely identify such adolescents at the time of screening, our analyses provide some clues that may help to direct and focus such efforts. First, adolescents with a positive CRAFFT screen that have not had follow-up contact with a BHC provider by week 2 of the follow-up period are increasingly more likely to be at low to average levels of engagement as additional weeks progress without such follow-up contact. Therefore, SBHC personnel should attempt to schedule a follow-up visit with a BHC provider for all adolescents with a positive CRAFFT screen who have not had such contact by the end of second week of the follow-up period. As additional weeks pass without follow-up contact with a BHC provider, the less likely such follow-up contact becomes, barring enhanced outreach and engagement efforts.
      Additionally, our analysis of adolescent predictors of engagement in follow-up care suggests that adolescents that are female, have had previous sexual activity, are not at risk for depression, and have no previous suicide attempts are less likely to engage in follow-up care on average than those without these characteristics. On the one hand, it is encouraging that adolescents at risk for SUD, as well as depression and suicidality, exhibit relatively high levels of engagement in follow-up care. On the other, adolescents at risk for SUD without these risks appear relatively less likely to engage in follow-up care, thereby increasing the risk for progression of substance use behavior within this group. Ozechowski and Waldron [
      • Ozechowski T.J.
      • Waldron H.B.
      Assertive outreach strategies for narrowing the adolescent substance abuse treatment gap: Implications for research, practice, and policy.
      ] characterized adolescents with SUD absent of comorbid behavioral problems as a clinically elusive subpopulation in need of specialized outreach and engagement interventions. Our results suggest a similar approach may be warranted for adolescents screened as being at risk for SUD without elevated behavioral comorbidities such as depression and suicidality. The same conclusion may hold for females and adolescents with previous sexual activity.

      Limitations

      Because this study was an implementation evaluation and not a clinical trial, we were not able to experimentally test the impact of SBIRT services, particularly engagement in follow-up care, on substance use and associated behavioral health outcomes. Although engagement in follow-up care is a critically important service outcome in its own right, assessing the impact of SBIRT services on substance use behavior and associated behavioral health outcomes is a vital objective that would have strengthened this study. An additional limitation is that we did not measure or analyze provider characteristics or behaviors likely to be influential in the engagement of adolescent patients in follow-up care. Nor were organizational SBHC-level influences on adolescent engagement in follow-up care taken into account. The omission of provider and organizational influences leads to an incomplete view of a complex process involving multilevel influences beyond adolescent demographic and behavioral/clinical characteristics. An additional limitation is that the study included only three SBHCs from a single school district in a mid-sized urban city in the Western United States. Our results may not generalize well to SBHCs in other parts of the country and beyond, particularly those in either rural or large metropolitan areas with populations having different racial/ethnic and socioeconomic compositions.

      Directions for further research

      Future research on SBIRT in SBHCs should focus on strategies to increase the overall proportion of adolescents that receive health screening and intervention in schools with SBHCs [
      • Ozechowski T.J.
      • Waldron H.B.
      Assertive outreach strategies for narrowing the adolescent substance abuse treatment gap: Implications for research, practice, and policy.
      ]. In addition, experimental evaluations of the effects of SBIRT services in SBHCs on substance use and other behavioral health outcomes are needed. Finally, comparative evaluations are needed of SBIRT services delivered in different types of health care settings for adolescents, including SBHCs and PPC clinics.

      Acknowledgments

      We thank the participating school-based health centers and their staff, the study participants, and the research staff. We thank Grace McCauley and Rachel Sebastian for their contributions in conceptualizing and implementing the study.

      Funding Sources

      This project was funded by the Conrad N. Hilton Foundation (Grant #17608 ) and the New Mexico Human Services Department (Contract #14-630-8000-0008 A2) .

      Supplementary Data

      References

        • Bronstein L.R.
        • Mason S.E.
        School-linked services: Promoting equity for children, families, and communities.
        Columbia University Press, New York, NY2016
        • Kolbe L.J.
        School health as a strategy to improve both public health and education.
        Annu Rev Public Health. 2019; 40: 443-463
        • Lee R.M.
        • Gortmaker S.L.
        School interventions to support health behavior change.
        in: Hilliard M.E. Riekert K.A. Ockene J.K. The handbook of health behavior change. 5th ed. Springer Publishing Company, New York, NY2018
        • Substance Abuse and Mental Health Services Administration
        Ready, set, go review: Screening for behavioral health risk in school.
        Office of the Chief Medical Officer, Rockville, MD2019
        • Kjolhede C.
        • Lee A.C.
        • Duncan De Pinto C.
        • et al.
        School-based health centers and pediatric practice.
        Pediatrics. 2021; 148e2021053758
        • Dunfee M.N.
        School-based health centers in the United States: Roots, reality, and potential.
        J Sch Health. 2020; 90: 665-670
        • Keeton V.
        • Soleimanpour S.
        • Brindis C.D.
        School-based health centers in an era of health care reform: Building on history.
        Curr Probl Pediatr Adolesc Health Care. 2012; 42: 132-156
        • Love H.E.
        • Schlitt J.
        • Soleimanpour S.
        • et al.
        Twenty years of school-based health care growth and expansion.
        Health Aff. 2019; 38: 755-764
        • Weitzman C.
        • Wegner L.
        • The Section on Developmental and Behavioral Pediatrics
        • et al.
        Promoting optimal development: screening for behavioral and emotional problems.
        Pediatrics. 2015; 135: 384-395
        • Council on School Health
        School-based health centers and pediatric practice.
        Pediatrics. 2012; 129: 387-393
        • Allison M.A.
        • Crane L.A.
        • Beaty B.L.
        • et al.
        School-based health centers: Improving access and quality of care for low-income adolescents.
        Pediatrics. 2007; 120: e887-e894
        • Arenson M.
        • Hudson P.J.
        • Lee N.
        • et al.
        The evidence on school-based health centers: A review.
        Glob Pediatr Health. 2019; 6 (2333794X19828745)
        • Bersamin M.
        • Coulter R.W.S.
        • Gaarde J.
        • et al.
        School-based health centers and school connectedness.
        J Sch Health. 2019; 89: 11-19
        • Choi K.R.
        • Easterlin M.C.
        Intervention models for increasing access to behavioral health services among youth: A systematic review.
        J Dev Behav Pediatr. 2018; 39: 754-762
        • Gibson E.J.
        • Santelli J.S.
        • Minguez M.
        • et al.
        Measuring school health center impact on access to and quality of primary care.
        J Adolesc Health. 2013; 53: 699-705
        • Knopf J.A.
        • Finnie R.K.C.
        • Peng Y.
        • et al.
        School-based health centers to advance health equity.
        Am J Prev Med. 2016; 51: 114-126
        • Harris B.R.
        Communicating about screening, brief intervention, and referral to treatment: Messaging strategies to raise awareness and promote voluntary adoption and implementation among New York school-based health center providers.
        Subst Abus. 2016; 37: 511-515
        • Harris S.K.
        • Aalsma M.C.
        • Weitzman E.R.
        • et al.
        Research on clinical preventive services for adolescents and young adults: Where are we and where do we need to go?.
        J Adolesc Health. 2017; 60: 249-260
        • Mitchell S.G.
        • Gryczynski J.
        • Gonzales A.
        • et al.
        Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based program: Services and outcomes.
        Am J Addict. 2012; 21: 5-13
        • Levy S.J.L.
        • Williams J.F.
        • Committee on Substance Use and Prevention
        Substance use screening, brief intervention, and referral to treatment.
        Pediatrics. 2016; 138e20161211
        • Bagley S.
        • Shrier L.
        • Levy S.
        Talking to adolescents about alcohol, drugs and sexuality.
        Minerva Pediatr. 2014; 66: 77-87
        • Bains R.M.
        • Diallo A.F.
        Mental health services in school-based health centers: Systematic review.
        J Sch Nurs. 2016; 32: 8-19
        • Bains R.M.
        • Cusson R.
        • White-Frese J.
        • et al.
        Utilization of mental health services in school-based health centers.
        J Sch Health. 2017; 87: 584-592
        • Koenig K.T.
        • Ramos M.M.
        • Fowler T.T.
        • et al.
        A statewide profile of frequent users of school-based health centers: Implications for adolescent health care.
        J Sch Health. 2016; 86: 250-257
        • Chadi N.
        • Levy S.
        • Wisk L.E.
        • et al.
        Student experience of school screening, brief intervention, and referral to treatment.
        J Sch Health. 2020; 90: 431-438
        • Curtis B.L.
        • McLellan A.T.
        • Gabellini B.N.
        Translating SBIRT to public school settings: An initial test of feasibility.
        J Subst Abuse Treat. 2014; 46: 15-21
        • McCarty C.A.
        • Gersh E.
        • Katzman K.
        • et al.
        Screening and brief intervention with adolescents with risky alcohol use in school-based health centers: A randomized clinical trial of the check yourself tool.
        Subst Abus. 2019; 40: 510-518
        • Maslowsky J.
        • Schulenberg J.E.
        Interaction matters: Quantifying conduct problem × depressive symptoms interaction and its association with adolescent alcohol, cigarette, and marijuana use in a national sample.
        Dev Psychopathol. 2013; 25: 1029-1043
        • Gryczynski J.
        • Mitchell S.G.
        • Schwartz R.P.
        • et al.
        Computer- vs. nurse practitioner-delivered brief intervention for adolescent marijuana, alcohol, and sex risk behaviors in school-based health centers.
        Drug Alcohol Depend. 2021; 218: 108423
        • Levy S.
        • Sherritt L.
        • Harris S.K.
        • et al.
        Test-retest reliability of adolescents’ self-report of substance use.
        Alcohol Clin Exp Res. 2004; 28: 1236-1241
        • Levis B.
        • Sun Y.
        • He C.
        • et al.
        Accuracy of the PHQ-2 alone and in combination with the PHQ-9 for screening to detect major depression: Systematic review and meta-analysis.
        JAMA. 2020; 323: 2290-2300
        • Staples L.G.
        • Dear B.F.
        • Gandy M.
        • et al.
        Psychometric properties and clinical utility of brief measures of depression, anxiety, and general distress: The PHQ-2, GAD-2, and K-6.
        Gen Hosp Psychiatry. 2019; 56: 13-18
        • Bentley K.H.
        • Sakurai H.
        • Lowman K.L.
        • et al.
        Validation of brief screening measures for depression and anxiety in young people with substance use disorders.
        J Affect Disord. 2021; 282: 1021-1029
        • Hedeker D.
        • Mermelstein R.J.
        • Flay B.R.
        Application of item response theory models for intensive longitudinal data.
        in: Walls T.A. Schafer J.L. Models for intensive longitudinal data. Oxford University Press, New York2006: 84-108
        • Rubin D.B.
        Multiple imputation for nonresponse in surveys.
        Wiley-Interscience, Hoboken, NJ2004
        • Little R.J.A.
        • Rubin D.B.
        Statistical analysis with missing data.
        3rd ed. Wiley, Hoboken, NJ2020
        • Mitchell S.G.
        • Gryczynski J.
        • Schwartz R.P.
        • et al.
        Adolescent SBIRT implementation: Generalist vs. specialist models of service delivery in primary care.
        J Subst Abuse Treat. 2020; 111: 67-72
        • Sterling S.
        • Kline-Simon A.H.
        • Satre D.D.
        • et al.
        Implementation of screening, brief intervention, and referral to treatment for adolescents in pediatric primary care: A cluster randomized trial.
        JAMA Pediatr. 2015; 169e153145
        • Monico L.B.
        • Mitchell S.G.
        • Dusek K.
        • et al.
        A comparison of screening practices for adolescents in primary care after implementation of screening, brief intervention, and referral to treatment.
        J Adolesc Health. 2019; 65: 46-50
        • Farmer R.F.
        • Seeley J.R.
        • Kosty D.B.
        • et al.
        Internalizing and externalizing psychopathology as predictors of cannabis use disorder onset during adolescence and early adulthood.
        Psychol Addict Behav. 2015; 29: 541-551
        • Griffith-Lendering M.F.H.
        • Huijbregts S.C.J.
        • Mooijaart A.
        • et al.
        Cannabis use and development of externalizing and internalizing behaviour problems in early adolescence: A TRAILS study.
        Drug Alcohol Depend. 2011; 116: 11-17
        • Hussong A.M.
        • Ennett S.T.
        • Cox M.J.
        • et al.
        A systematic review of the unique prospective association of negative affect symptoms and adolescent substance use controlling for externalizing symptoms.
        Psychol Addict Behav. 2017; 31: 137-147
        • Khoddam R.
        • Jackson N.J.
        • Leventhal A.M.
        Internalizing symptoms and conduct problems: Redundant, incremental, or interactive risk factors for adolescent substance use during the first year of high school?.
        Drug Alcohol Depend. 2016; 169: 48-55
        • O’Neil K.A.
        • Conner B.T.
        • Kendall P.C.
        Internalizing disorders and substance use disorders in youth: Comorbidity, risk, temporal order, and implications for intervention.
        Clin Psychol Rev. 2011; 31: 104-112
        • Stephan S.
        • Mulloy M.
        • Brey L.
        Improving collaborative mental health care by school-based primary care and mental health providers.
        Sch Ment Health. 2011; 3: 70-80
        • Love H.
        • Panchal N.
        • Schlitt J.
        • et al.
        The use of telehealth in school-based health centers.
        Glob Pediatr Health. 2019; 62333794X19884194
        • Goddard A.
        • Sullivan E.
        • Fields P.
        • et al.
        The future of telehealth in school-based health centers: Lessons from COVID-19-19.
        J Pediatr Health Care. 2021; 35: 304-309
        • Sullivan E.
        • Brey L.
        • Soleimanpour S.
        School-based health center operations during the COVID-19-19 pandemic: A preliminary study.
        Health Promot Pract. 2021; 22: 616-621
        • Ozechowski T.J.
        • Waldron H.B.
        Assertive outreach strategies for narrowing the adolescent substance abuse treatment gap: Implications for research, practice, and policy.
        J Behav Health Serv Res. 2010; 37: 40-63