Advertisement

Youth-Reported School Connection and Experiences of a Middle School–Based Screening, Brief Intervention, and Referral to Treatment Initiative: Preliminary Results From a Program Evaluation

      Abstract

      Purpose

      This study aimed (1) to evaluate the feasibility of a school-based Screening, Brief Intervention, and Referral to Treatment (SBIRT) program that expands on traditional SBIRT to support the mental health and well-being of middle school students and (2) to assess its effects on students’ connection with adults at school.

      Methods

      Focus group discussions were conducted with 26 students in grades 6–8 to understand student perspectives about an innovative school-based SBIRT program. A subset of middle school students from the SBIRT program who received a brief intervention (BI) after screening (n = 116) were asked to rate their experience meeting with the interventionist in terms of feeling comfortable, feeling listened to, and talking about their goals. Additionally, these students’ ratings of connection to adults at school was compared from the time of screening (baseline) to following BI using two-sided paired t-tests.

      Results

      Students who participated in focus groups expressed favorable opinions about universal screening and this school-based SBIRT model and noted that relationship building with adults at school was an important factor for open communication and motivating behavior change for students. Nearly all students who completed the post-BI survey rated their experiences with interventionists during BI as “Excellent,” “Very Good,” or “Good” in all categories (98%). Students’ reported mean school connection scores significantly higher after participation in school-based SBIRT than at baseline (5.9/8 vs. 7.0/8, p < .001).

      Discussion

      Middle school students were satisfied with the school-based SBIRT model and participation in the program resulted in increased student connection with adults at school. These findings improve our understanding of the experience of SBIRT intervention with middle school students and on school connection in particular.

      Keywords

      Implications and Contribution
      This evaluation describes student experiences of a novel school-based SBIRT model. Results indicate that the SBIRT intervention was well-received by middle school students, and participating students had an increased connection with adults at school. This model is a promising approach to prevention in a school setting.
      Early prevention and intervention to address substance use and mental health is critical to support young people’s healthy development. Early initiation of substance use has been associated with poor cognitive functioning, mental health difficulties, and dropping out of high school [
      • Medina K.L.
      • Hanson K.L.
      • Schweinsburg A.D.
      • et al.
      Neuropsychological functioning in adolescent marijuana users: Subtle deficits detectable after a month of abstinence.
      ,
      • Zimmerman M.A.
      • Schmeelk-Cone K.H.
      A longitudinal analysis of adolescent substance use and school motivation among African American youth.
      ,
      • Gil A.G.
      • Wagner E.F.
      • Tubman J.G.
      Associations between early-adolescent substance use and subsequent young-adult substance use disorders and psychiatric disorders among a multiethnic male sample in South Florida.
      ]. Initiation of substance use before the age of 14 has also been shown to be a strong predictor of later problems including heavier substance use throughout adolescence and adulthood, more serious long-term substance use problems, and adult substance dependence [
      • Johnston L.D.
      • O’Maley P.M.
      • Miech R.A.
      • et al.
      Overview: Key findings on adolescent drug use.
      ]. A large body of evidence has shown that preventing or delaying the onset of emotional and behavioral disorders and substance use can have lifetime benefits and that focusing such prevention efforts on young people can yield the most impact [
      National Research Council and Institute of Medicine
      Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Committee on prevention of mental disorders and substance abuse among children, youth and young adults: Research advances and promising interventions.
      ]. A recent study showed that exposing early adolescents to prevention in seventh grade reduced the likelihood of alcohol use disorders 5 years later, showing the potential for long-term impacts of early intervention [
      • Gonzales N.A.
      • Jensen M.
      • Tein J.Y.
      • et al.
      Effect of middle school interventions on alcohol misuse and abuse in Mexican American high school adolescents: Five-year follow-up of a randomized clinical trial.
      ].
      Schools can be a critical partner in improving systems of support to promote youth well-being and school environments play an important role in student academic and behavioral outcomes [
      • Suldo S.M.
      • Gormley M.J.
      • DuPaul G.J.
      • Anderson-Butcher D.
      The impact of school mental health on student and school level academic outcomes: Current status of the research and future directions.
      ,
      • Doll B.
      • Spies R.
      • Champion A.
      Contributions of ecological school mental health services to students' academic success.
      ,
      • Haynes N.M.
      • Emmons C.
      • Ben-Avie M.
      School climate as a factor in student adjustment and achievement.
      ]. Since adolescents spend the majority of their time in school, this can be an ideal setting to implement early prevention and intervention programs. Many young people, especially those from low-income households and those who identify as Black, Indigenous, and people of color, access mental health services solely in educational settings [
      • Ali M.M.
      • West K.
      • Teich J.L.
      • et al.
      Utilization of mental health services in educational setting by adolescents in the United States.
      ]. In addition, a student’s perceived school connectedness is associated with less violence and substance use, as well as higher student academic achievement, better school attendance, and increased social and emotional well-being [
      • Bond L.
      • Butler H.
      • Thomas L.
      • et al.
      Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health, and academic outcomes.
      ,
      • Kuperminc G.P.
      • Leadbeater B.J.
      • Blatt S.J.
      School social climate and individual differences in vulnerability to psychopathology among middle school students.
      ,
      • McNeely C.
      • Nonnemaker J.
      • Blum J.
      Promoting school connectedness: Evidence from the National Longitudinal Study of Adolescent Health.
      ]. School connectedness refers to a feeling of belonging or the extent to which a student feels cared for at school [
      • McNeely C.
      • Nonnemaker J.
      • Blum J.
      Promoting school connectedness: Evidence from the National Longitudinal Study of Adolescent Health.
      ]. In addition to being a strong protective factor in adolescence, school connectedness is also protective in adulthood, reducing emotional distress, suicidal ideation, and sexual and substance use risk [
      • Steiner R.J.
      • Sheremenko G.
      • Lesesne C.
      • et al.
      Adolescent connectedness and adult health outcomes.
      ,
      • Seil K.S.
      • Desai M.M.
      • Smith M.V.
      Sexual orientation, adult connectedness, substance use, and mental health outcomes among adolescents: Findings from the 2009 New York City Youth Risk Behavior Survey.
      ]. Student connections with a trusted adult at school are one component of school connectedness. These connections with a trusted adult at school can have far ranging impacts on youth academic and social outcomes and health risk behaviors in adulthood [
      • Sieving R.E.
      • McRee A.L.
      • McMorris B.J.
      • et al.
      Youth-adult connectedness: A key protective factor for adolescent health.
      ,
      • Culyba A.J.
      • Gingburg K.R.
      • Fein J.A.
      • et al.
      Protective effects of adolescent-adult connection on male youth in Urban environments.
      ]. The transition to middle school, a stressful time for many adolescents, is an especially important time to promote connection and to increase protective factors [
      • Lester L.
      • Cross D.
      The relationship between school climate and mental and emotional wellbeing over the transition from primary to secondary school.
      ].
      Screening, Brief Intervention, and Referral to Treatment (SBIRT) is a public health model for identifying and addressing substance use and related risks [
      SAMHSA
      Screening, Brief intervention, and referral to treatment (SBIRT).
      ,
      SAHMSA
      Evidence supporting the effectiveness of SBIRT in behavioral healthcare [white paper].
      ]. SBIRT has been used to deliver early intervention and treatment to people with substance use disorders. The original SBIRT model consists of screening to assess risks and severity of substance use, delivery of a brief intervention (BI) focused on increasing insight and motivation toward behavioral change, and when needed a referral to treatment for those needing additional intervention or services. Motivational interviewing (MI) has commonly been used as part of SBIRT as a strategy to assess readiness to change, provide personalized feedback, and motivate behavior change [
      • Miller W.R.
      Motivational interviewing with problem drinkers.
      ,
      • Miller W.R.
      • Rollnick S.
      Motivational interviewing: Preparing people to change addictive behavior.
      ]. The SBIRT model originated with adults in healthcare settings but has more recently been adapted to address adolescent substance use and related risk factors. School-based SBIRT has been used to provide early intervention for students prior to the development of substance use disorders, including for those who are early initiators.
      When implemented in high school settings, SBIRT has been found to reduce adolescents’ use of marijuana and alcohol and to result in reduced binge drinking and other drug use [
      • Walker D.D.
      • Roffman R.A.
      • Stephens R.S.
      • et al.
      Motivational enhancement therapy for adolescent marijuana users: A preliminary randomized controlled trial.
      ,
      • Winters K.C.
      • Fahnhorst T.
      • Botzet A.
      • et al.
      Brief intervention for drug-abusing adolescents in a school setting: Outcomes and mediating factors.
      ,
      • Mitchell S.G.
      • Gryczynski J.
      • O’Grady K.E.
      • Schwartz R.P.
      SBIRT for adolescent drug and alcohol use: Current status and future directions.
      ,
      • Hamza D.M.
      • Bercov M.
      • Suen V.Y.M.
      • et al.
      School-based Screening, Brief Interventions and Referral to Treatment (SBIRT) significantly decreases long-term substance abuse in 6,227 students aged 11-18.
      ]. In addition, SBIRT implemented in school-based health centers has shown promising reductions in youth alcohol and marijuana use [
      • Gryczynski J.
      • Mitchell S.G.
      • Schwarts R.P.
      • et al.
      Computer- vs. nurse practitioner-delivered brief intervention for adolescent marijuana, alcohol, and sex risk behaviors in school-based health centers.
      ,
      • McCarty C.A.
      • Gersh E.
      • Katzman K.
      • et al.
      Screening and brief intervention with adolescents with risky alcohol use in school-based health centers: A randomized clinical trial of the check yourself tool.
      ]. Although school-based health centers are a convenient setting for intervention with young people, they are not available in most US schools [
      • Love H.
      • Soleimanpour S.
      • Panchal N.
      • et al.
      2016-17 National school-based health care census report.
      ]. Although SBIRT appears to be an effective tool for addressing and preventing youth substance use, little is known about the impact of school-based SBIRT on school connections with trusted adults and how SBIRT is perceived by younger adolescents, such as those in middle school. One recent study surveyed middle and high school students who had participated in school-based SBIRT focused on substance use and found that the majority of students felt positively about substance use screening in schools, and that speaking to an adult about drugs or alcohol was valuable regardless of their use of substances in the past year [
      • Chadi N.
      • Levy L.
      • Wisk E.
      • Weitzman E.R.
      Student experience of school screening, brief intervention and referral to treatment.
      ].
      In 2015, the Best Starts for Kids tax levy was approved by voters in King County, Washington to support the health and well-being of children, families, and communities within the county. This initiative, along with funding through a local behavioral health sales tax, Mental Illness and Drug Dependency, supported the implementation of a school-based SBIRT program in King County schools starting in September 2018. Although traditional SBIRT focuses specifically on substance use, this school-based SBIRT model is a novel approach which is intended to broadly address the mental health and well-being of middle school students by assessing and offering an intervention structure for a range of risk factors, such as anxiety and depressive symptoms, bullying, self-harm, and offering connection with a continuum of supports at school and in the community based on the student’s identified needs [
      • McCarty C.A.
      • Elkin E.
      • Soukup M.
      School-based screening, brief intervention and referral to treatment: An implementation playbook.
      ]. In addition to addressing risk factors, this SBIRT intervention was intended to build upon students’ strengths and connect students with a caring adult at school. This paper focuses on (1) the feasibility of this school-based SBIRT model among middle school students and (2) the effect of participation in school-based SBIRT on student connection with adults at school.

      Methods

      Description of screening, brief intervention, and referral to treatment program

      The SBIRT program, developed and overseen by King County Public Health to support students’ well-being within the schools, consisted of screening for substance use, mental health concerns, and bullying, alongside assessment of strengths; a BI based on MI principles that involved semi-structured 15- to 20-minute sessions with both the youth alone and together with their caregiver, as appropriate; and referral to assessment and/or other community-based services and supports, including counseling, mentoring, and youth leadership opportunities. Participating middle schools conducted universal screening of an entire grade level or the whole school based on capacity of the staff to implement the model. In addition to universal screening, 38% of schools also conducted indicated screening where students were referred to participate in school-based SBIRT based on academic or disciplinary indicators by teachers or other school staff.
      Interventionists were identified by each school district and their roles varied depending on each school district’s implementation plan, which could be tailored to their resources and needs. Interventionists included school counselors, staff from community-based organizations, or other student support staff. All interventionists participated in program trainings on the implementation of school-based SBIRT and received in-depth MI training. Students participated in multiple BI sessions when needed, but due to the brief nature of this intervention the majority (78.3%) participated in only one BI session. A small proportion of students (13.1%) received two BI sessions, and 8.0% received three or more sessions.
      Based on their screening results, students were prioritized for follow-up into one of three tiers defined by the risk factors they endorsed. Students who were prioritized into Tier 3 based on screening received a BI within 24 hours of screening and those who were prioritized into Tier 2 based on screening received a BI in the next few weeks after screening. Those students who did not endorse any risk factors were considered Tier 1 and participated in whole-school prevention activities but did not receive a BI. Tier 3 risk factors included suicidal ideation, self-harm, a recent suicide attempt, and a student request to speak with a counselor as soon as possible. Tier 2 risk factors included substance use, anxiety symptoms, depressive symptoms, experiencing frequent aches and pains, indicating that the student has been feeling angry, sad, or worried frequently, and a request to speak with a counselor in the next few weeks. Students were considered Tier 3 if they endorsed any of the Tier 3 risk factors, regardless of whether they also endorsed Tier 2 risk factors. Students were considered Tier 2 if they endorsed any of the Tier 2 risk factors and did not also endorse a Tier 3 risk factor. Interventionists conducted BI sessions with students in Tiers 2 and 3, using MI strategies to assess strengths, explore student goals, provide referrals, and follow-up as needed, with engagement of caregivers when appropriate.
      School districts were selected to participate in the evaluation component based on their readiness and progress in program implementation, their capacity to participate in the evaluation, and their geographic and demographic diversity to represent different areas of King County. Each school district obtained parent permission prior to administering screening and students provided assent before participating. Two of the school districts obtained passive parental permission and four obtained active parental permission. The type of parental permission was determined by each school district individually. A unique identifier was assigned to each student to protect student privacy. The program evaluation activities including the focus group and the post-BI surveys were reviewed by the Seattle Children’s Research Institute Institutional Review Board and determined to not be human subjects research because the purpose was to understand and provide feedback to King County about this particular program.

      Measures

      Focus group guide

      The focus groups were structured using the same guide for each group. Students were asked to react to two fictitious scenarios about the types of experiences young people may have while participating in SBIRT, and their opinions were solicited about screening settings, how school counselors can best support students, and barriers that young people may face in accessing resources or support.

      Baseline screening

      Baseline screening was conducted using school-based Check Yourself, a multirisk electronic screening tool that provides personalized feedback for the respondent based on their responses [
      • McCarty C.A.
      • Elkin E.
      • Soukup M.
      School-based screening, brief intervention and referral to treatment: An implementation playbook.
      ]. School-based Check Yourself screens for a variety of concerns including substance use, anxiety and depressive symptoms, bullying, self-harm, and suicidal ideation. It also identifies youth protective factors and student support systems. Demographic information and connection to adults at school was also collected at this time.

      Experiences of brief intervention

      To assess student experiences with BI, we used the 5-item Consultation and Relational Empathy measure, a validated measure of patient-rated experience [
      • Bikker A.P.
      • Fitzpatrick B.
      • Murphy D.
      • Mercer S.W.
      Measuring empathic, person-centred communication in primary care nurses: Validity and reliability of the Consultation and Relational Empathy (CARE) measure.
      ]. This measure includes five statements about the student’s experience meeting with the interventionist and asks students to rate their agreement with each statement on a 5-point scale from Poor to Excellent. We calculated the proportion of students who rated each of the five experiences as “Good,” “Very Good,” or “Excellent” as an indicator of satisfaction with BI.

      Connection to adults at school

      We assessed student connection with adults at school, a component of school connection [
      Centers for Disease Control and Prevention
      School connectedness: Strategies for increasing protective factors among youth.
      ], during screening and post-BI using a 4-item measure adapted from the Student Resilience Survey, a validated survey of student protective factors [
      • Lereya S.T.
      • Hymphrey N.
      • Patalay P.
      • et al.
      The student resilience survey: Psychometric validation and associations with mental health.
      ]. Each item begins with the statement “At school there is an adult who” and includes the following: cares about me, tells me when I do a good job, listens to me when I have something to say, and believes I will be a success. The original measure uses a 5-point response scale for each item; however, we simplified the responses to yes (scored 2), sometimes (scored 1), or no (scored 0). All items were summed to create an overall connection score, ranging from 0 to 8. This scale was administered at screening and again following BI.

      Participants and procedures

      Focus groups

      Three focus group discussions were conducted with 26 middle school students from 3 of the participating schools (1 group per school) across 3 school districts that participated in the program evaluation. Schools were selected based on interventionist capacity to participate in the evaluation activities and with consideration of demographic and geographic diversity. SBIRT interventionists invited students who had participated in the program during that school year to participate in the focus groups. All students who participated in the SBIRT program were eligible, regardless of whether or not they also received BI or a referral. Parental permission was obtained by the school prior to student participation in the school-based SBIRT program, and student assent was obtained verbally prior to participation in the focus group discussions.
      Student focus groups were conducted in June 2019. The purpose of these focus groups was to collect students’ perspectives on screening, BI, and referral components of the SBIRT program. After introductions, students were presented with a series of ground rules meant to guide the conversation and provide an inviting space for each student to share their perspective. To protect confidentiality, students were asked not to share personal experiences. A member of the program evaluation team took notes about key topics and students were asked to write responses on post-it notes during a brainstorming exercise to anonymously capture student feedback. Focus groups were not recorded to protect student privacy.

      Post–brief intervention survey

      A post-BI survey was conducted with 116 middle school students from 15 schools in 6 school districts as part of a program evaluation from 2019 to 2020. Interventionists implementing school-based SBIRT at the 15 middle schools that participated in the evaluation invited students who received BI to participate in the post-BI survey after their first BI meeting. Students with an identified safety concern or need for immediate support (such as when suicidal ideation was disclosed) were typically not invited to participate in the post-BI surveys unless the interventionist determined that administering a follow-up survey was appropriate. Interventionists asked students to complete post-BI surveys privately on a tablet device.

      Analysis

      Focus groups

      We analyzed focus group notes using inductive thematic analysis, which uses an iterative process of identifying themes within the data and grouping themes into categories for analysis [
      • Braun V.
      • Clarke V.
      Using thematic analysis in psychology.
      ]. Initially, focus group notes were reviewed for accuracy by three team members who conducted the groups. Each focus group was coded independently by two team members using an initial set of themes developed based on the focus group topics. The coders met to review and reconcile all codes and ensure that interpretation of codes was consistent. Differences in codes were resolved through consensus discussion and a final code book was generated. Once coding was complete, themes and subcodes were reviewed to accurately reflect the data and codes were grouped into broad themes.

      Post–brief intervention surveys

      Descriptive data were presented on students, from the six districts who participated in post-BI surveys, including how they compared to the broader sample of students receiving BI from participating districts and descriptive information on student ratings of experiences of BI using the Consultation and Relational Empathy measure. We conducted two-sided paired t-tests to assess whether mean school connection was significantly different after participating in school-based SBIRT than at baseline using an alpha level of 0.05. To assess whether the proportion of students responding “Yes” to each of the individual school connection measure items increased after participating in school-based SBIRT, we conducted two-sample tests of proportion for each of the separate items using an alpha level of 0.05.

      Results

      Study sample

      Demographic characteristics of the study samples are shown in Table 1. Students who participated in the post-BI survey represented 4% of students who received BI in the participating school districts. Most of the students (83%) who participated in post-BI surveys were in sixth or seventh grade while most focus group participants (66%) were in eighth grade. Slightly more students who identified as male participated in both the post-BI survey and in focus group discussions. About half (48%) of students who participated in the focus group discussions identified as White and about one third (35%) of post-BI survey participants identified as White. In order to understand how representative students who participated in the program evaluation were of the broader population, we compared demographic characteristics of students from these six districts who received BI (n = 3,253) to the post-BI Survey participants (n = 116). Students who participated in post-BI surveys were representative of the overall school-based SBIRT sample in terms of their gender and race/ethnicity; however, the study sample included fewer eighth graders and more sixth graders than the program overall. Students from certain school districts (D and E) were over-represented in the post-BI survey participant sample, whereas students from other districts (B, F) were under-represented. Students who took the post-BI survey did not differ from students in participating districts in their scores on connectedness to adults at school at baseline (M = 5.89, standard deviation = 2.19 vs. M = 5.84, standard deviation = 2.10; t(3,242) = 0.23, p = .82).
      Table 1Demographic characteristics of study samples
      VariableStudents who received BI in participating district (n = 3,253), % (n)Post-BI survey participants (n = 116), % (n)Focus group participants (n = 26), % (n)
      School district
       A12.7% (414)37.1% (43)-
       B20.7% (674)4.3% (5)31% (8)
       C20.0% (650)18.1% (21)-
       D11.4% (370)19.0% (22)38% (10)
       E18.6% (604)19.0% (22)31% (8)
       F16.6% (541)2.6% (3)-
      Grade
      Of the 3,253 students who participated in BI (column 1), 7 students were in grades higher than 8th grade.
       6th14.5% (471)40.5% (47)38% (10)
       7th48.4% (1,575)42.2% (49)-
       8th36.9% (1,200)17.2% (20)62% (16)
      Age
      Of the 3,253 students who participated in BI (column 1), 10 students were 15 years or older.
       10–1111.6% (375)31.0% (36)-
      Focus group participants were not asked to share their age.
       1234.9% (1,135)23.3% (27)-
       1340.9% (1,331)32.8% (38)-
       1412.4% (402)12.9% (15)-
      Sex
      Of the 3,253 students who participated in BI (column 1), 93 were gender diverse including nonbinary, transgender, questioning their gender identity, gender fluid, or writing in another response, and 56 students in the survey sample preferred not to share their gender identity, 4 students were not asked, and gender identity information was missing for 2 students. In the focus group sample, gender was missing for 1 participant.
      Information for cells containing <10 students were not included in this table to comply with the Family Educational rights and Privacy Act guidelines.
       Female48.1% (1,564)44.8% (52)48% (12)
       Male47.2% (1,534)47.4% (55)52% (13)
       Gender diverse2.9% (93)<10-
      Race/ethnicity
      Information for cells containing <10 students were not included in this table to comply with the Family Educational rights and Privacy Act guidelines.
      Categories with <10 participants have been suppressed in compliance with Family Educational rights and Privacy Act guidelines.
       Asian or Asian Indian14.0% (455)15.5% (18)17% (5)
       American Indian or Alaska Native1.7% (55)<10-
       Black or African American7.9% (256)10.3% (12)<10
       Middle Eastern or North African1.4% (44)<10
       Latinx or Hispanic22.1% (720)24.1% (28)<10
       Multiracial11.2% (365)<10<10
       Native Hawaiian or Pacific Islander2.8% (92)<10<10
       White36.4% (1,185)34.5% (40)48% (14)
       Unknown2.5% (81)<10-
      School connectedness, mean (SD), range = 0–85.84 (2.19)5.89 (2.10)-
      a Of the 3,253 students who participated in BI (column 1), 7 students were in grades higher than 8th grade.
      b Of the 3,253 students who participated in BI (column 1), 10 students were 15 years or older.
      c Focus group participants were not asked to share their age.
      d Of the 3,253 students who participated in BI (column 1), 93 were gender diverse including nonbinary, transgender, questioning their gender identity, gender fluid, or writing in another response, and 56 students in the survey sample preferred not to share their gender identity, 4 students were not asked, and gender identity information was missing for 2 students. In the focus group sample, gender was missing for 1 participant.
      e Information for cells containing <10 students were not included in this table to comply with the Family Educational rights and Privacy Act guidelines.
      f Categories with <10 participants have been suppressed in compliance with Family Educational rights and Privacy Act guidelines.

      Focus groups

      Preferred screening setting

      When asked how they felt about completing the screening component of school-based SBIRT in a classroom setting with other students, participants preferred a group setting to individual screening and were comfortable with the universal screening model used in school-based SBIRT. One student shared “I would be really stressed if it was just me and someone else, but if there were more people around I would feel comfortable.” However, participants also valued privacy and wanted a clear explanation of how their information would be used or shared, especially information that was shared during BI following screening.

      Comfort with school-based screening, brief intervention, and referral to treatment interventionists

      When asked about what helps students feel comfortable meeting with adults at school and having potentially sensitive conversations, participants felt that having a personal connection and relationship building with school counselors is key to feeling comfortable sharing personal information. One participant shared that when school staff form connections with students “…they’re sharing a piece of themselves with you, they’re not just asking questions. It gives you more of a personal relationship with them.” Participants also felt that forming closer connections with adults at school could help motivate behavior change. Participants expressed being more comfortable talking with counselors than teachers. One student shared that “counselors are more trustworthy because that’s what they do…their reputation is to help you with problems.”

      Barriers to accessing referrals

      We asked participants about barriers that young people may experience when accessing referrals provided at school. The most common reasons that students gave for being hesitant to access resources outside of school were concerns over sharing of personal information, parental permission or buy-in, and stigma around seeking help for mental health concerns. One group discussed gender differences in help-seeking behavior and stigma around seeking mental health care, and felt that discussing mental health care and seeking help was seen as more acceptable for female identifying students than male identifying students.

      Student experiences of brief intervention

      Nearly all students who completed the post-BI survey rated their experiences with interventionists during BI as “Excellent,” “Very Good,” or “Good” in all categories (98%) (Figure 1). Most students indicated that the interventionists let them tell their story, really listened to them, and made them feel comfortable. Responses about explaining the goal of the BI meetings and talking with the student about their goals were somewhat less favorable, but still generally positive. In both areas, less than half of respondents rated their experiences as “Excellent” (40%), about a third rated their experiences as “Very Good” (33% and 34% respectively), and a quarter rated their experiences as “Good” (24% and 21% respectively).
      Figure thumbnail gr1
      Figure 1Student-reported ratings of interactions during brief intervention (n = 116).

      Intervention effects: screening to post–brief intervention

      Among students who responded to the post-BI survey, mean school connection scores were significantly higher after participating in school-based SBIRT than at screening (5.9/8 vs. 7.0/8, p < .001) and the proportion of students responding “Yes” to each of the school connection items increased significantly at post-BI compared to baseline (Table 2).
      Table 2Proportion of students responding “Yes” to each of the four school connection measure questions at each time point (n = 116)
      At school there is an adult who…% “Yes” baseline% “Yes” post-BIp-value
      Values <.05 were considered significant.
      Cares about me57.8%75.9%<.001
      Tells me when I do a good job54.3%72.4%<.001
      Listens to me51.7%76.7%<.001
      Believes that I will be a success68.1%85.3%<.001
      a Values <.05 were considered significant.

      Discussion

      To our knowledge, this is one of the first papers to evaluate the implementation of an expanded school-based SBIRT model that focuses broadly on supporting students across an array of risk factors such as substance use, while also strengthening protective factors such as connection with a trusted adult for middle school students. Most studies of SBIRT used with adolescents in a school setting have included only substance users and have been implemented with high school students [
      • Mitchell S.G.
      • Gryczynski J.
      • O’Grady K.E.
      • Schwartz R.P.
      SBIRT for adolescent drug and alcohol use: Current status and future directions.
      ,
      • Maslowsky J.
      • Whelan Capell J.
      • Moberg D.P.
      • Brown R.L.
      Universal school-based implementation of screening brief intervention and referral to treatment to reduce and prevent alcohol, marijuana, Tobacco, and other drug use: Process and feasibility.
      ,
      • Mitchell S.G.
      • Gryczynski J.
      • Gonzalez A.
      • et al.
      Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based program: Services and outcomes.
      ], while this model included universal screening of middle school students. This school-based SBIRT program was acceptable to middle school students who participated in this evaluation. Students reported positive experiences with BI and preferred a universal screening method to an indicated one-on-one approach. The majority of students rated their interactions during BI positively, although there could have been more attention to explaining why interventionists were meeting with the student and discussing the student’s goals. While exploring goals is a part of the BI process included in this model, this may not have been covered with every student, resulting in lower ratings for that specific area.
      In this evaluation, building a connection with adults at school was important to students. Connection with a trusted adult is a key protective factor for young people and can be important to healthy youth development [
      • Viner R.M.
      • Ozer E.M.
      • Denny S.
      • et al.
      Adolescence and the social determinants of health.
      ,
      • Resnick M.D.
      • Bearman P.S.
      • Blum R.W.
      • et al.
      Protecting adolescents from harm: Findings from the national longitudinal study on adolescent health.
      ]. In the focus group discussions, students reported that having a relationship with school counselors or other adults made them feel more comfortable having sensitive conversations. They also felt that adults with whom they had a stronger connection would be more likely to help motivate behavior change than those with whom they did not have an existing relationship. Students who participated in the post-BI survey reported significantly higher connection with adults at school after participating in school-based SBIRT and receiving BI. These findings indicate that this school-based SBIRT program helped students to feel more positive connections to adults in their middle schools. Although we did not explore causal factors underlying school connection changes in this evaluation, studies have shown that the use of MI with adolescents as part of an SBIRT model has been successful in promoting a variety of positive health outcomes [
      • McCambridge J.
      • Strang J.
      The efficacy of single-session motivational interviewing in reducing drug consumption and perceptions of drug-related risk and harm among young people: Results from a multi-site cluster randomized trial.
      ,
      • Jensen C.D.
      • Cushing C.C.
      • Aylward B.S.
      • et al.
      Effectiveness of motivational interviewing interventions for adolescent substance use behavior change: A meta-analytic review.
      ,
      • Barnett E.
      • Sussman S.
      • Smith C.
      • et al.
      Motivational interviewing for adolescent substance use: A review of the literature.
      ,
      • Mutschler C.
      • Naccarato E.
      • Rouse J.
      • et al.
      Realist-informed review of motivational interviewing for adolescent health behaviors.
      ]. MI approaches are relational in nature, involve active listening by the interventionist and support adolescent autonomy which may contribute to the improvement of student connection with adults at school.
      In contrast to traditional SBIRT models, this program was also unique in its broad focus and goal to connect young people with supportive adults at school. The model focus was not exclusive to substance use as it also incorporated intervention for depression, anxiety, bullying, self-harm, and other behavioral concerns. Most young people in the United States begin using substances after middle school (after the age of 13), although use at age 11 or younger is associated with a higher likelihood of developing substance use disorders and co-occurring mental health disorders [
      Centers for Disease Control and Prevention
      Youth online: Youth behavior risk Surveillance survey data 2019.
      ,
      Substance Abuse and Mental Health Services Administration
      The TEDS report: Age of substance use initiation among treatment admissions aged 18 to 30.
      ]. Studies have shown that intervention programs implemented in middle school, when most young people either do not use substances or are not heavy users, have been successful in decreasing use, reducing the likelihood of substance use disorders, and increasing school engagement in later years [
      • Gonzales N.A.
      • Jensen M.
      • Tein J.Y.
      • et al.
      Effect of middle school interventions on alcohol misuse and abuse in Mexican American high school adolescents: Five-year follow-up of a randomized clinical trial.
      ,
      • Gonzales N.A.
      • Wong J.J.
      • Toomey R.B.
      • et al.
      School engagement mediates long term prevention effects for Mexican American adolescents.
      ,
      • Onrust S.A.
      • Otten R.
      • Lammers J.
      • Smit F.
      School-based programmes to reduce and prevent substance use in different age groups: What works for whom? Systematic review and meta-regression analysis.
      ,
      • Daily S.M.
      • Mann M.J.
      • Lilly C.L.
      • et al.
      School climate as a universal intervention to prevent substance use initiation in early adolescence: A longitudinal study.
      ]. In addition, school connectedness and school climate have been found to have a positive impact on student outcomes [
      • Michael S.L.
      • Merlo C.L.
      • Basch C.E.
      • et al.
      Critical connections: Health and academics.
      ,
      • Niehaus K.
      • Moritz Rudasill K.
      • Rakes C.R.
      A longitudinal study of school connectedness and academic outcomes across sixth grade.
      ]. Therefore, implementing preventive programs such as school-based SBIRT in middle school may be a strategy to positively impact young people’s health and well-being while preventing future risk factors, although research exploring such outcomes over time is necessary to confirm this.

      Addressing limitations of this evaluation

      This study has several limitations. This program evaluation was conducted at the same time as implementation and scale-up occurred; therefore, our sample size for post-BI surveys was limited by school district capacity to participate in program evaluation activities, the number of students screened during the data collection period at participating schools, and interventionist capacity to administer the post-BI surveys during program implementation. Several of the participating schools delayed screening or screened fewer students than expected during the evaluation timeframe resulting in fewer students completing the post-BI survey. Because this was program evaluation, students were not selected at random. However, the students who participated in surveys were generally representative of the overall sample of students who participated received BI on gender, race/ethnicity, and connectedness to adults at school baseline, although there were some differences in grade level and age. In addition, participating school districts were able to tailor implementation of some program components such as screening timing and process, who administered BI, and how parental permission was obtained to their unique context and needs, resulting in variation in how the intervention was delivered across school districts. Two of the six participating school districts had less than five students participate in the post-BI survey which limited our ability to conduct more in-depth analyses including controlling for school district or assessing differences in outcomes by school district.
      This evaluation was not powered to identify demographic differences in school connection changes after participating in school-based SBIRT; however, this is an area worthy of future focus. In addition, because of the low proportion of students who endorsed substance use in the program, we were not able to assess changes in student substance use or mental health symptoms after participating in school-based SBIRT. Although our sample includes a diverse group of students, this program was implemented only in King County, Washington; therefore, our findings are specific to this geographic and demographic context. Although more data are needed to further understand the full impact of SBIRT on outcomes in middle school students, this evaluation demonstrates the value of an expanded school-based SBIRT model as an approach for strengthening school connectedness.

      Acknowledgments

      This work was accepted for presentation at the Society of Adolescent Health and Medicine 2020 Annual Meeting (oral presentation) and the American Public Health Association Annual Meeting 2021 (oral presentation). Funding for school-based SBIRT comes from a tax levy-funded investment in the health of young people ages 5–24 called Best Starts for Kids, and with additional funding through a local behavioral health sales tax, Mental Illness and Drug Dependency.

      Funding Sources

      Grant funding for the program evaluation of school-based SBIRT was supported by the Conrad N. Hilton Foundation (Grant #18444 ) and a contract from Best Starts for Kids.

      References

        • Medina K.L.
        • Hanson K.L.
        • Schweinsburg A.D.
        • et al.
        Neuropsychological functioning in adolescent marijuana users: Subtle deficits detectable after a month of abstinence.
        J Int Neuropsychol Soc. 2007; 13: 807-820
        • Zimmerman M.A.
        • Schmeelk-Cone K.H.
        A longitudinal analysis of adolescent substance use and school motivation among African American youth.
        J Res Adolesc. 2003; 13: 185-210
        • Gil A.G.
        • Wagner E.F.
        • Tubman J.G.
        Associations between early-adolescent substance use and subsequent young-adult substance use disorders and psychiatric disorders among a multiethnic male sample in South Florida.
        Am J Public Health. 2004; 94: 1603-1609
        • Johnston L.D.
        • O’Maley P.M.
        • Miech R.A.
        • et al.
        Overview: Key findings on adolescent drug use.
        in: Monitoring the future national survey results on drug use 1975–2017. Institute for Social Research, University of Michigan, Ann Arbor, MI2017 (Available at:)
        • National Research Council and Institute of Medicine
        Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Committee on prevention of mental disorders and substance abuse among children, youth and young adults: Research advances and promising interventions.
        in: Board on children, youth, and families, division of behavioral and social sciences and education. The National Academies Press, Washington, DC2009
        • Gonzales N.A.
        • Jensen M.
        • Tein J.Y.
        • et al.
        Effect of middle school interventions on alcohol misuse and abuse in Mexican American high school adolescents: Five-year follow-up of a randomized clinical trial.
        JAMA Psychiatry. 2018; 75: 429-437
        • Suldo S.M.
        • Gormley M.J.
        • DuPaul G.J.
        • Anderson-Butcher D.
        The impact of school mental health on student and school level academic outcomes: Current status of the research and future directions.
        Sch Ment Health. 2003; 6: 84-98
        • Doll B.
        • Spies R.
        • Champion A.
        Contributions of ecological school mental health services to students' academic success.
        J Educ Psychol. 2012; 22: 44-61
        • Haynes N.M.
        • Emmons C.
        • Ben-Avie M.
        School climate as a factor in student adjustment and achievement.
        J Educ Psychol. 1997; 8: 321-329
        • Ali M.M.
        • West K.
        • Teich J.L.
        • et al.
        Utilization of mental health services in educational setting by adolescents in the United States.
        J Sch Health. 2019; 89: 393-401
        • Bond L.
        • Butler H.
        • Thomas L.
        • et al.
        Social and school connectedness in early secondary school as predictors of late teenage substance use, mental health, and academic outcomes.
        J Adolesc Health. 2007; 40: 9-18
        • Kuperminc G.P.
        • Leadbeater B.J.
        • Blatt S.J.
        School social climate and individual differences in vulnerability to psychopathology among middle school students.
        J Sch Psychol. 2001; 39: 141-159
        • McNeely C.
        • Nonnemaker J.
        • Blum J.
        Promoting school connectedness: Evidence from the National Longitudinal Study of Adolescent Health.
        J Sch Health. 2002; 72: 138-147
        • Steiner R.J.
        • Sheremenko G.
        • Lesesne C.
        • et al.
        Adolescent connectedness and adult health outcomes.
        Pediatrics. 2019; 144: e20183766
        • Seil K.S.
        • Desai M.M.
        • Smith M.V.
        Sexual orientation, adult connectedness, substance use, and mental health outcomes among adolescents: Findings from the 2009 New York City Youth Risk Behavior Survey.
        Am J Public Health. 2014; 104: 1950-1956
        • Sieving R.E.
        • McRee A.L.
        • McMorris B.J.
        • et al.
        Youth-adult connectedness: A key protective factor for adolescent health.
        Am J Prev Med. 2017; 52: S275-S278
        • Culyba A.J.
        • Gingburg K.R.
        • Fein J.A.
        • et al.
        Protective effects of adolescent-adult connection on male youth in Urban environments.
        J Adolesc Health. 2016; 58: 237-240
        • Lester L.
        • Cross D.
        The relationship between school climate and mental and emotional wellbeing over the transition from primary to secondary school.
        Appl Psychol Health Well-being. 2015; 5: 9
        • SAMHSA
        Screening, Brief intervention, and referral to treatment (SBIRT).
        (Available at:)
        https://www.samhsa.gov/sbirt
        Date accessed: November 14, 2021
        • SAHMSA
        Evidence supporting the effectiveness of SBIRT in behavioral healthcare [white paper].
        (Available at:)
        • Miller W.R.
        Motivational interviewing with problem drinkers.
        Behav Psychother. 1983; 11: 147-172
        • Miller W.R.
        • Rollnick S.
        Motivational interviewing: Preparing people to change addictive behavior.
        Guilford Press, New York, NY1991
        • Walker D.D.
        • Roffman R.A.
        • Stephens R.S.
        • et al.
        Motivational enhancement therapy for adolescent marijuana users: A preliminary randomized controlled trial.
        J Consult Clin Psychol. 2006; 74: 628-632
        • Winters K.C.
        • Fahnhorst T.
        • Botzet A.
        • et al.
        Brief intervention for drug-abusing adolescents in a school setting: Outcomes and mediating factors.
        J Subst Abuse Treat. 2012; 42: 279-288
        • Mitchell S.G.
        • Gryczynski J.
        • O’Grady K.E.
        • Schwartz R.P.
        SBIRT for adolescent drug and alcohol use: Current status and future directions.
        J Subst Abuse Treat. 2013; 44: 463-472
        • Hamza D.M.
        • Bercov M.
        • Suen V.Y.M.
        • et al.
        School-based Screening, Brief Interventions and Referral to Treatment (SBIRT) significantly decreases long-term substance abuse in 6,227 students aged 11-18.
        J Addict Behav Ther. 2018; 2
        • Gryczynski J.
        • Mitchell S.G.
        • Schwarts R.P.
        • et al.
        Computer- vs. nurse practitioner-delivered brief intervention for adolescent marijuana, alcohol, and sex risk behaviors in school-based health centers.
        Drug Alcohol Depend. 2021; 218: 108423
        • McCarty C.A.
        • Gersh E.
        • Katzman K.
        • et al.
        Screening and brief intervention with adolescents with risky alcohol use in school-based health centers: A randomized clinical trial of the check yourself tool.
        Subst Abus. 2019; 40: 510-518
        • Love H.
        • Soleimanpour S.
        • Panchal N.
        • et al.
        2016-17 National school-based health care census report.
        School-Based Health Alliance, Washington, DC2018 (Available at:)
        • Chadi N.
        • Levy L.
        • Wisk E.
        • Weitzman E.R.
        Student experience of school screening, brief intervention and referral to treatment.
        J Sch Health. 2020; 90: 431-438
        • McCarty C.A.
        • Elkin E.
        • Soukup M.
        School-based screening, brief intervention and referral to treatment: An implementation playbook.
        2021 (Seattle, WA: SeattleChildrens_SB-SBIRT_Playbook_Sept-1-DIGITAL.pdf. Available at:)
        reclaimingfutures.org
        Date accessed: June 16, 2022
        • Bikker A.P.
        • Fitzpatrick B.
        • Murphy D.
        • Mercer S.W.
        Measuring empathic, person-centred communication in primary care nurses: Validity and reliability of the Consultation and Relational Empathy (CARE) measure.
        BMC Fam Pract. 2015; 16: 149
        • Centers for Disease Control and Prevention
        School connectedness: Strategies for increasing protective factors among youth.
        U.S. Department of Health and Human Services, Atlanta, GA2009
        • Lereya S.T.
        • Hymphrey N.
        • Patalay P.
        • et al.
        The student resilience survey: Psychometric validation and associations with mental health.
        Child Adolesc Psychiatry Ment Health. 2016; 10: 44
        • Braun V.
        • Clarke V.
        Using thematic analysis in psychology.
        Qual Res Psychol. 2006; 3: 77-101
        • Maslowsky J.
        • Whelan Capell J.
        • Moberg D.P.
        • Brown R.L.
        Universal school-based implementation of screening brief intervention and referral to treatment to reduce and prevent alcohol, marijuana, Tobacco, and other drug use: Process and feasibility.
        Subst Abuse. 2017; 111178221817746668
        • Mitchell S.G.
        • Gryczynski J.
        • Gonzalez A.
        • et al.
        Screening, brief intervention, and referral to treatment (SBIRT) for substance use in a school-based program: Services and outcomes.
        Am J Addict. 2012; 21: S5-S13
        • Viner R.M.
        • Ozer E.M.
        • Denny S.
        • et al.
        Adolescence and the social determinants of health.
        Lancet. 2012; 379: 1641-1652
        • Resnick M.D.
        • Bearman P.S.
        • Blum R.W.
        • et al.
        Protecting adolescents from harm: Findings from the national longitudinal study on adolescent health.
        JAMA. 1997; 278: 823-832
        • McCambridge J.
        • Strang J.
        The efficacy of single-session motivational interviewing in reducing drug consumption and perceptions of drug-related risk and harm among young people: Results from a multi-site cluster randomized trial.
        Addiction. 2004; 99: 39-52
        • Jensen C.D.
        • Cushing C.C.
        • Aylward B.S.
        • et al.
        Effectiveness of motivational interviewing interventions for adolescent substance use behavior change: A meta-analytic review.
        J Consult Clin Psychol Aug. 2011; 79: 433-440
        • Barnett E.
        • Sussman S.
        • Smith C.
        • et al.
        Motivational interviewing for adolescent substance use: A review of the literature.
        Addict Behav. 2012; 37: 1325-1334
        • Mutschler C.
        • Naccarato E.
        • Rouse J.
        • et al.
        Realist-informed review of motivational interviewing for adolescent health behaviors.
        Syst Rev. 2018; 7: 109
        • Centers for Disease Control and Prevention
        Youth online: Youth behavior risk Surveillance survey data 2019.
        (Available at:)
        • Substance Abuse and Mental Health Services Administration
        The TEDS report: Age of substance use initiation among treatment admissions aged 18 to 30.
        Center for Behavioral Health Statistics, Rockville, MD2014 (Available at:)
        • Gonzales N.A.
        • Wong J.J.
        • Toomey R.B.
        • et al.
        School engagement mediates long term prevention effects for Mexican American adolescents.
        Prev Sci. 2014; 15: 929-939
        • Onrust S.A.
        • Otten R.
        • Lammers J.
        • Smit F.
        School-based programmes to reduce and prevent substance use in different age groups: What works for whom? Systematic review and meta-regression analysis.
        Clin Psychol Rev. 2016; 44: 45-59
        • Daily S.M.
        • Mann M.J.
        • Lilly C.L.
        • et al.
        School climate as a universal intervention to prevent substance use initiation in early adolescence: A longitudinal study.
        Health Educ Behav. 2020; 47: 402-411
        • Michael S.L.
        • Merlo C.L.
        • Basch C.E.
        • et al.
        Critical connections: Health and academics.
        J Sch Health. 2015; 85: 740-758
        • Niehaus K.
        • Moritz Rudasill K.
        • Rakes C.R.
        A longitudinal study of school connectedness and academic outcomes across sixth grade.
        J Sch Psychol. 2012; 50: 443-460