Advertisement

Promotion of Preconception Care Among Adolescents and Young Adults by Conversational Agent

      Abstract

      Purpose

      Preconception care is important for all women to improve infant and maternal health outcomes and may be especially important for adolescents and young adults. This study assesses the acceptance, usability, and use of an automated intervention to screen women on 108 preconception care risks and address them over the course of a year via a Web-based virtual animated health counselor and compares these measures for the adolescent and young adult users aged 18–25 years with those of users aged 26–34 years. We hypothesize that the younger cohort will have significantly greater use of and satisfaction with the online intervention.

      Methods

      A randomized controlled trial involving a national sample of 528 women was conducted. We present a secondary data analysis on the system use and self-reported usability and satisfaction of the 79 women aged 18–25 years randomized to the intervention group, compared with the 183 women aged 26–34 years in the intervention group. Participants were required to self-identify as female, black or African American, aged 18–34 years, not pregnant, and English-speaking and were recruited through a variety of advertisements and outreach activities.

      Results

      Of the adolescent and young adult participants (aged 18–25 years) enrolled and randomized to the intervention, 20.25% of participants accessed the system 0 times; 29.11%, 1–3 times; and 50.63%, >3 times over the course of a year. At the end of the year, almost all (96.4%) indicated they had either acted on recommendations made by the agent or planned to. Most (75.0%) said they would recommend the system to someone they knew. There were no significant differences between the two age groups on intervention use or satisfaction.

      Conclusions

      Web-based conversational agents are a viable medium for delivering longitudinal preconception care counseling to adolescents and young adults.

      Keywords

      Preconception care (PCC) is care provided before pregnancy to improve maternal and infant health outcomes [
      • Johnson K.
      • Posner S.F.
      • Biermann J.
      • et al.
      Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
      ]. PCC involves many aspects of health care, spanning 83 topics in 14 clinical areas, including diverse topics such as family planning, lifestyle behaviors including physical activity and weight management, immunizations, substance abuse, infectious disease treatment, and chronic disease management [
      • Jack B.W.
      • Atrash H.
      • Coonrod D.V.
      • et al.
      The clinical content of preconception care: An overview and preparation of this supplement.
      ].
      There is growing consensus and evidence that PCC should be provided to all women of reproductive age [
      • Johnson K.
      • Posner S.F.
      • Biermann J.
      • et al.
      Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
      ,
      • van der Zee B.
      • de Beaufort I.
      • Temel S.
      • et al.
      Preconception care: An essential preventive strategy to improve children's and women's health.
      ]. However, adolescence may be a particularly important time to begin PCC for a variety of reasons. First, because many aspects of PCC require longitudinal behavior change (e.g., lifestyle behaviors such as diet and exercise, chronic disease self-care management, long-term adherence to folic acid, etc.), months or years may be required for an intervention target to be reached, indicating these interventions should be started as early as possible. Second, adolescence is when many health-related behaviors are established for adulthood, indicating it as the ideal time to begin healthy PCC behaviors to be maintained throughout adulthood [
      • Sawyer S.M.
      • Afifi R.A.
      • Bearinger L.H.
      • et al.
      Adolescence: A foundation for future health.
      ,
      • Viner R.M.
      • Ozer E.M.
      • Denny S.
      • et al.
      Adolescence and the social determinants of health.
      ]. Third, many PCC-related disorders first appear during adolescence (e.g., 75% of mental health disorders [
      • Kessler R.C.
      • Berglund P.
      • Demler O.
      • et al.
      Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.
      ]), indicating this is an ideal time to conduct screening and early treatment.
      Practitioners do an imperfect job addressing preventive health issues for adolescents in general because of a range of barriers including lack of time, discomfort discussing sensitive topics, lack of skill, and lack of confidence [
      • M'Hamdi H.I.
      • van Voorst S.F.
      • Pinxten W.
      • et al.
      Barriers in the uptake and delivery of preconception care: Exploring the views of care providers.
      ]. The situation is no different for PCC: practitioners rarely discuss the availability of and need for PCC for all women of reproductive age [
      • M'Hamdi H.I.
      • van Voorst S.F.
      • Pinxten W.
      • et al.
      Barriers in the uptake and delivery of preconception care: Exploring the views of care providers.
      ,
      • Mazza D.
      • Chapman A.
      Improving the uptake of preconception care and periconceptional folate supplementation: What do women think?.
      ].
      Automated interventions that systematically screen for all known PCC risk factors and provide longitudinal support following best practices from behavioral medicine may represent an ideal mechanism to help adolescent and young adult (AYA) women address their unique set of PCC risks. Technology in general (computers, smartphones, etc.) is widely used by adolescents [
      • Lenhart A.
      Teens, social media & technology: Overview 2015.
      ], and there is growing evidence that technology-based interventions may be especially effective for adolescents, particularly regarding sexual and reproductive health issues and other sensitive topics [
      • Paperny D.M.
      Computerized health assessment and education for adolescent HIV and STD prevention in health care settings and schools.
      ,
      • Schwarz E.B.
      • Burch E.J.
      • Parisi S.M.
      • et al.
      Computer-assisted provision of hormonal contraception in acute care settings.
      ]. Adolescents may view advice from an automated system as less judgmental than advice from a health educator or clinician [
      • Lenhart A.
      Teens, social media & technology: Overview 2015.
      ].
      In this work, we report the development and evaluation of an automated intervention for PCC among AYA women (Figure 1). The intervention first screens women for 108 PCC risk factors using an online survey and then uses a Web-based conversational agent that plays the role of a health counselor that talks to women over the course of a year to address their PCC risks. Because PCC is particularly important for African American women, given that they have twice the risk of delivering a low-birth-weight infant and four times the risk of maternal mortality, compared with white women [
      • Martin J.
      • Hamilton B.
      • Osterman M.
      • et al.
      Births: Final data for 2012.
      ], the intervention was tailored for AYA African American women.
      Figure thumbnail gr1
      Figure 1The Gabby preconception care conversational agent.

      The “Gabby” preconception care conversational agent

      The intervention uses an animated, embodied conversational agent to deliver simulated face-to-face health counseling to women. Several studies have demonstrated that adult patients and consumers find this modality acceptable and effective, particularly among those with inadequate health literacy [
      • Bickmore T.
      • Pfeifer L.
      • Byron D.
      • et al.
      Usability of conversational agents by patients with inadequate health literacy: Evidence from two clinical trials.
      ,
      • Bickmore T.
      • Utami D.
      • Matsuyama R.
      • Paasche-Orlow M.
      Improving access to online health information with conversational agents: A randomized controlled experiment.
      ]. The agent uses both verbal and nonverbal conversational behavior to provide a natural and intuitive computer interface. It talks using synthetic speech and animated coverbal behavior, such as hand gestures and facial displays, and the user responds primarily by selecting what they want to say from a multiple-choice menu of options updated during each turn of dialog. The interface does not rely on unconstrained natural language understanding (allowing users to say anything they want), which can represent a safety issue in automated health counseling systems [
      • Bickmore T.
      • Trinh H.
      • Olafsson S.
      • et al.
      Patient and consumer safety risks when using conversational assistants for medical information: An observational study of Siri, Alexa, and Google Assistant.
      ]. Dialog is driven by hierarchical transition networks, modeling the layered structure of conversation [
      • Clark H.H.
      Using language.
      ], with template-based text generation [
      • Reiter E.
      • Dale R.
      Building natural language generation systems.
      ], allowing every agent utterance to be completely tailored to each user and the current discourse context (e.g., what was just discussed in the current or past conversations, as described previously in the studies by Bickmore and Picard [
      • Bickmore T.
      • Picard R.
      Establishing and maintaining long-term human-computer Relationships.
      ] and Bickmore et al. [
      • Bickmore T.
      • Gruber A.
      • Picard R.
      Establishing the computer-patient working alliance in automated health behavior change interventions.
      ]). Agent coverbal behavior is primarily generated using an automated text-to-embodied-speech generator [
      • Cassell J.
      • Vilhjálmsson H.
      • Bickmore T.
      BEAT: The behavior expression animation toolkit.
      ].
      There are several reasons why conversational agents provide an effective medium for health communication with patients, especially those with limited literacy. First, the interface relies only minimally on text and uses the universally understood format of face-to-face conversation, thus making it less intimidating and more accessible to patients with limited literacy skills. Second, agents can emphasize critical information using nonverbal behavior the same way people do in conversation (e.g., using hand gesture, eyebrow raise, and/or speech prosody). Third, the use of nonverbal conversational behaviors, such as hand gestures that convey specific information through pointing (“deictic” gestures) or through shape or motion (“iconic” and “metaphoric” gestures), can provide a redundant medium for conveying semantic content to enhance message comprehension. For example, the agent can point at a patient's hip when describing where to wear a pedometer to reinforce and clarify the instruction.

      Health behavior change framework

      Our overarching approach to addressing each individual PCC risk is the transtheoretical, or “stages of change” model of health behavior change. The transtheoretical model posits that individuals go through a series of five discrete stages, from “precontemplation” to “maintenance” in the process of changing their behavior, and that the behavior change techniques that should be used for an individual at a given time should be selected based on their stage of change [
      • Prochaska J.
      • Velicer W.
      The transtheoretical model of health behavior change.
      ]. For individuals in “precontemplation” (not yet intending to change), we use counseling techniques from motivational interviewing to help them resolve them ambivalence and take first steps in addressing a risk [
      • Miller W.
      • Rollnick S.
      Motivational interviewing: Preparing people for change.
      ]. Other behavior change techniques are used for individuals in later stages of change, for example, goal setting and positive reinforcement for behaviors requiring incremental, longitudinal change, such as exercise and diet.

      Multiple health behavior change

      Most automated health behavior change interventions developed to date focus on changing only one behavior at a time, and many do not provide ongoing support over the weeks or months that change may require. However, PCC almost always involves intervening on many health behaviors: in our pilot work, we found that women had 23.2 (standard deviation: 6.12) PCC risks on average, and of these, women chose to address an average of 6.3 (standard deviation: 7.2) risks with the conversational agent during a six-month intervention period in which they could access the agent at any time over the Web [
      • Ren J.
      • Bickmore T.
      • Hempstead M.
      • Jack B.
      Birth control, drug abuse, or domestic violence? What health risk topics are women willing to discuss with a virtual agent?.
      ].
      One approach to changing multiple behaviors is to address them sequentially; only addressing a new behavior once a prior one has been successfully changed. Individuals who have some success changing one target behavior have been found to be more likely to achieve success on a second behavior [
      • Johnson S.S.
      • Pavia A.L.
      • Cummins C.O.
      • et al.
      Transtheoretical model-based multiple behavior intervention for weight management: Effectiveness on a population basis.
      ], and any behavior change success may increase general self-efficacy that increases motivation to change many other health behaviors [
      • Prochaska J.
      Multiple health behavior research represents the future of preventive medicine.
      ]. However, some behaviors can take weeks or months to address, making an overall sequential intervention potentially years in duration. Some researchers have identified “bundles” of behaviors that can effectively be addressed at the same time [
      • Evers K.
      • Quintiliani L.
      Advances in multiple health behavior change research.
      ]. In addition, there may actually be benefits to changing multiple health behaviors simultaneously [
      • Evers K.
      • Quintiliani L.
      Advances in multiple health behavior change research.
      ]. Actions that address multiple behaviors can reduce the total perceived effort that an individual takes and increases her confidence to change. Multiple health behavior change has also been shown to be economically advantageous [
      • Evers K.
      • Quintiliani L.
      Advances in multiple health behavior change research.
      ].
      Our automated PCC intervention uses a visual customizable list of identified risks and recommendations to minimize or eliminate these risks, called a “My Health To-Do List” (Figure 2). This aid helps users keep track of their risks and progress over time and provides feedback on actions taken and risks addressed to provide positive reinforcement and motivation to continue. After the initial PCC risk survey, the agent introduces the “My Health To-Do List” and lets users select the identified risks they want to discuss. The agent then asks questions to identify the user's stage of change for addressing the risk [
      • Prochaska J.
      • Velicer W.
      The transtheoretical model of health behavior change.
      ], then describes why it is important, and offers users the opportunity to take action on it.
      All aspects of the agent's appearance, voice, the design of the “My Health To-Do List,” and the counseling dialogs were designed with significant feedback from focus groups that are comprised of women from the target population.

      Methods

      This is a secondary data analysis from an unpublished randomized controlled trial (ClinicalTrials.gov Identifier: NCT01827215). The parent trial compared African American women aged 18–34 years who used the Gabby PCC intervention (n = 262) with a control group who only received a report describing their PCC risks (n = 266), with primary outcomes based on aggregate change in “stage of change” for each of their identified PCC risks. After enrollment, all participants took an online risk assessment, and those in the intervention group were then provided with 12 months' access to the Web-based conversational agent, with outcomes collected from all participants at six and 12 months after randomization.
      Here, we report on the experiences of the intervention group with the intervention itself for the 168 AYA women aged 18–25 years in the study (of 528 total), focusing on the acceptance and use of the system. We hypothesize that the AYA women users will have significantly greater use of and satisfaction with the online intervention over the course of the one-year intervention period than users aged 26–34 years. The study was approved by the Boston Medical Center Institutional Review Board.

      Measures

      Personal characteristics factors

      We assessed participants' ethnic origin, primary use of language, education background, marital status, employment status, and household income. We also assessed participants' previous experience with computers and their attitudes toward using computers.

      Health literacy

      Health literacy is assessed using the Rapid Estimate of Adult Literacy in Medicine (REALM) instrument [
      • Davis T.C.
      • Long S.W.
      • Jackson R.H.
      • et al.
      Rapid estimate of adult literacy in medicine: A shortened screening instrument.
      ]. Scores range from 0 to 66, with higher scores indicating higher levels of health literacy.

      System usage

      System usage is tracked in log files and assessed using the number of sessions that participants had with the virtual counselor during the study period and the time (in minutes) that participants spent on interacting with the virtual counselor.

      Satisfaction

      Six single-item scale questions were asked to assess participants' satisfaction with the virtual counselor: [
      • Johnson K.
      • Posner S.F.
      • Biermann J.
      • et al.
      Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
      ] “How easy was it to talk with Gabby?” [
      • Jack B.W.
      • Atrash H.
      • Coonrod D.V.
      • et al.
      The clinical content of preconception care: An overview and preparation of this supplement.
      ] “How much do you trust Gabby?” [
      • van der Zee B.
      • de Beaufort I.
      • Temel S.
      • et al.
      Preconception care: An essential preventive strategy to improve children's and women's health.
      ] “Did you feel your sessions with Gabby were (too short, just right, too long)?” [
      • Sawyer S.M.
      • Afifi R.A.
      • Bearinger L.H.
      • et al.
      Adolescence: A foundation for future health.
      ] “How well did Gabby answer any questions you had?” [
      • Viner R.M.
      • Ozer E.M.
      • Denny S.
      • et al.
      Adolescence and the social determinants of health.
      ] “To what degree would you rather have talked to a health-care provider than Gabby?” and [
      • Kessler R.C.
      • Berglund P.
      • Demler O.
      • et al.
      Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.
      ] “If given opportunity, would you like to talk with Gabby again?”

      Results

      The current analysis consisted of 79 female participants aged 25 years or younger, compared with the 183 participants aged 26–34 years. Their baseline demographics and personal characteristics are shown in Table 1. The two cohorts were primarily African American (80% of the AYA women, 83% of those aged 26–34 years), primarily non-Hispanic (92%, 96%), college educated (42%, 33%), single (82%, 58%), and had full-time employment (43%, 63%). Both groups were also self-reported “expert” computer users (58%, 50%) and had adequate health literacy (REALM scores: 63.6, 63.8). These characteristics do differ from those of the general US population, given our focus on African American women aged 18–34 years.
      Table 1Descriptive statistics of intervention participant characteristics by age group
      Participant characteristicsAYA women, age 18–25 years (N = 79)AYA women, age 26–34 years (N = 183)
      Age, mean (SD)22.75 (1.93)29.58 (2.42)
      Ethnic origin, n (%)
       African American63 (80)151 (83)
       Cape Verdean1 (1)4 (2)
       Caribbean12 (15)21 (11)
       Ethiopian1 (1)2 (1)
       Somalian1 (1)1 (1)
       Nigerian8 (10)8 (4)
       Kenyan0 (0)0 (0)
       Other13 (16)41 (22)
      Hispanic, n (%)
       No, not of Hispanic, Latino, or Spanish origin73 (92)176 (96)
       Yes, Mexican, Mexican American, Chicano1 (1)0 (0)
       Yes, Puerto Rican1 (1)2 (1)
       Yes, Cuban0 (0)0 (0)
       Yes, another Hispanic, Latino, or Spanish origin4 (5)5 (3)
      Language, n (%)
       English77 (97)172 (94)
       Other language2 (3)11 (6)
      Education, n (%)
       Less than high school (0 through 8)0 (0)0 (0)
       Some high school (9 through <12)2 (3)3 (2)
       GED or high school equivalency0 (0)5 (3)
       High-school graduate9 (11)7 (4)
       Attended vocational, trade, or business school after high school1 (1)1 (1)
       College, <2 years11 (14)12 (7)
       College, associate degree2 (3)11 (6)
       College 2 or more years, no degree10 (13)17 (9)
       College graduate (B.S. or B.A.)33 (42)61 (33)
       Postgraduate/no degree5 (6)10 (5)
       Postgraduate/degree (M.S., M.A., Ph.D., M.D., etc.)6 (8)56 (31)
      Marital status, n (%)
       Single, never married65 (82)107 (58)
       Single with partner, never married10 (13)34 (19)
       Married4 (5)34 (19)
       Divorced0 (0)2 (1)
       Separated0 (0)5 (3)
       Widowed0 (0)1 (1)
      Employment status, n (%)
       Full-time34 (43)116 (63)
       Part-time22 (28)31 (17)
       Unemployed or laid off13 (16)24 (13)
       Retired0 (0)0 (0)
       Home maker2 (3)2 (1)
       Student26 (33)33 (18)
       Disabled0 (0)0 (0)
       Active military status0 (0)0 (0)
       Other0 (0)3 (2)
      Income, n (%)
       Less than $5,0000 (0)1 (1)
       $5,000–$9,9992 (3)4 (2)
       $10,000–$14,9991 (1)1 (1)
       $15,000–$19,9993 (4)2 (1)
       $20,000–$29,9991 (1)6 (3)
       $30,000–$39,9995 (6)11 (6)
       $40,000–$49,99913 (16)19 (10)
       $50,000–$74,9997 (9)18 (10)
       $75,000–$99,99914 (18)48 (26)
       $100,000 or more6 (8)18 (1)
       Refused to answer10 (13)20 (11)
       Do not know0 (0)5 (3)
       No income17 (22)30 (16)
      Computer experience, n (%)
       I've never used one0 (0)1 (1)
       I've tried a few times0 (0)0 (0)
       I use one regularly33 (42)90 (49)
       I am an expert46 (58)92 (50)
      Computer access, n (%)
       Home46 (58)114 (62)
       Library4 (5)4 (2)
       Friends'/family's computer2 (3)1 (1)
       Work25 (32)63 (34)
       Other2 (3)1 (1)
      Computer access personal, n (%)
       Home50 (63)118 (64)
       Library1 (1)1 (1)
       Friends'/family's computer0 (0)0 (0)
       Work2 (3)7 (4)
       Mobile device23 (29)55 (30)
       Other2 (3)2 (1)
      Computer attitudes, n (%)
       I don’t like them0 (0)3 (2)
       They are okay17 (22)41 (22)
       I love playing with them62 (78)139 (76)
      Health literacy, REALM score, mean (SD)63.63 (2.80)63.77 (4.04)
      General Education Development (GED) is a High School Equivalency Certificate).
      AYA = adolescent and young adult; REALM, Rapid Estimate of Adult Literacy in Medicine; SD = standard deviation.

      System usage

      A total of 63 AYA participants (79.7% of those enrolled and randomized to intervention) used the system during the study period, compared with 136 aged 26–34 years (74.32% of those enrolled, Table 2). AYA users held a median of 6 (interquartile range: 9) sessions with the virtual counselor during the 12-month intervention period, compared with a median of 6 (interquartile range: 7) sessions for the older cohort. Overall, 20.25% of the AYA users interacted zero times with Gabby, 29.11% interacted 1–3 times, and 50.63% interacted more than 3 times over the course of the year, compared with 25.68%, 28.96%, and 45.36% for the older cohort, respectively. There were no significant differences across any of these metrics between the two age groups.
      Table 2Use of virtual counselor
      Number of Gabby interactions over a yearAge 18–25 years (n = 79)Age 26–34 years (n = 183)p
      Median interactions (IQR)6 (9)6 (7).761
      n (%) with 0 interactions16 (20.25)47 (25.68)χ2 (2) = 1.00, p = .61
      n (%) with 1–3 interactions23 (29.11)53 (28.96)
      n (%) with >3 interactions40 (50.63)83 (45.36)
      IQR = interquartile range.
      We conducted an analysis of the 16 AYA and 47 older users who did not use Gabby (Table 3). We did not find any significant systematic differences across usage patterns for participants based on education (high-school graduate vs. not), employment (full-time vs. not), computer literacy (self-proclaimed “computer expert” vs. not), or health literacy (adequate vs. inadequate) based on a REALM score of 61 or greater (ninth grade and above), as has been done in several prior studies [
      • Bickmore T.
      • Utami D.
      • Matsuyama R.
      • Paasche-Orlow M.
      Improving access to online health information with conversational agents: A randomized controlled experiment.
      ][
      • Lincoln A.
      • Paasche-Orlow M.
      • Cheng D.
      • et al.
      Impact of health literacy on depressive symptoms and mental health-related: Quality of life among adults with addiction.
      ][
      • Mancuso C.
      • Rincon M.
      Impact of health literacy on longitudinal asthma outcomes.
      ].
      Table 3Characteristics of AYA participants who did not use Gabby (cohort with participants aged 18–25 years)
      Personal characteristic0 interactions1–3 interaction>3 interactionsp
      High-school graduate or equivalent, n (%)16 (20.78%)21 (27.27%)40 (51.94%).082
      Not high-school graduate or equivalent, n (%)0 (.00%)2 (100.00%)0 (.00%)
      Full-time employment, n (%)4 (11.76%)10 (29.41%)20 (58.82%).233
      Not employed full-time, n (%)12 (26.67%)13 (28.89%)20 (44.44%)
      Expert computer user, n (%)8 (17.39%)14 (30.43%)24 (52.17%).755
      Not expert computer user, n (%)8 (24.24%)9 (27.27%)16 (48.48%)
      Adequate health literacy, n (%)11 (16.67%)20 (30.30%)35 (53.03%).202
      Inadequate health literacy, n (%)5 (38.46%)3 (23.08%)5 (38.46%)
      AYA = adolescent and young adult.
      Table 4Satisfaction with a virtual counselor
      Single-item questionsAnchor 1Anchor 4Anchor 7Median (IQR), (6 month)Median (IQR), (12 month)
      AYA women aged 18–25 yearsAYA women aged 26–34 yearspAYA women aged 18–25 yearsAYA women aged 26–34 yearsp
      How easy was it to talk with Gabby?EasyNeutralDifficult2 (2)2 (2).7982 (2)2 (2).584
      How much do you trust Gabby?Very muchNeutralNot at all2 (3)2 (3).7092 (3)2 (3).318
      Did you feel your sessions with Gabby were?Too shortJust rightToo long4 (1)4 (1).0544 (1)4 (1).544
      How well did Gabby answer any questions you had?Definitely yesNeutralDefinitely no2 (1)2 (1).5772 (1)2 (1).451
      To what degree would you rather have talked to a healthcare provider than Gabby?Definitely prefer my providerNeutralDefinitely prefer Gabby4 (2)4 (2).4063 (2.75)3 (2.75).904
      If given opportunity, would you like to talk with Gabby again?Definitely yesNeutralDefinitely no2 (3)2 (3).7603 (3)3 (3).227
      AYA = adolescent and young adult.

      Usability

      When asked about ease of use, 62.9% of AYA respondents rated ease of use a one or two on a scale of 1 (“Easy”) to 7 (“Difficult”) at six months, increasing slightly to 67.9% at 12 months (Table 4). Only 6.1% of AYA users rated Gabby as difficult to use (greater than neutral rating). Participants indicated that the length of their conversations with Gabby were “about right” (median 4) on a scale of 1 (“too short”) to 7 (“too long”) at both six and 12 months. There were no significant differences between AYA users and those in the older cohort on usability measures.

      Satisfaction

      Participants trusted Gabby, scoring a median of two on a scale of 1 (“very much”) to 7 (“not at all”) at both six and 12 months, and 80.0% indicated that Gabby answered their questions about PCC at six months, increasing to 85.7% at 12 months (Table 4). When asked if there were things about their health they were not comfortable telling Gabby, 88.6% of respondents said “no” at six months, changing to 78.6% at 12 months.
      When asked if they would recommend Gabby to someone they knew, 71.4% of respondents said they would at six months, increasing to 75.0% at 12 months. When asked if they would like to talk to Gabby again, 70.6% indicated they would at six months, increasing to 71.4% at 12 months.
      Participants did not feel that Gabby was a replacement for a health-care provider. When asked if they would have rather talked to a provider, 57.1% indicated they would have, with another 22.9% being neutral at six months, changing to 66.7% and 22.2% at 12 months, respectively.
      There were no significant differences between AYA users and those in the older cohort on satisfaction measures.

      Use of information provided by Gabby

      When asked whether they had already followed the recommendations provided by Gabby, 60.0% of respondents said they had at six months (increasing to 71.4% at 12 months), while another 34.3% at six months said they planned to in the future (25.0% at 12 months).

      Discussion

      Overall, the AYA cohort in our study found that interacting with the conversational agent about topics related to PCC was acceptable, with many using the system frequently over the 12 months of the intervention. At both six and 12 months, they reported high levels of trust in the agent and satisfaction with the intervention and reported that they followed (or would follow) Gabby's recommendations. Although they would rather talk to their provider about PCC, they indicated that they would use the system again if they could and would recommend it to someone else, indicating its utility in providing PCC information and counseling when women are unable to see their provider.
      We did not find any significant differences between AYA users and older users (aged 26–34 years) randomized to receive the Gabby intervention on any measure.

      Limitations

      The study has several limitations, including the small sample used and the lack of a true control group for usability measures (e.g., comparing with a non–agent-based automated intervention, as in the study by Bickmore et al. [
      • Bickmore T.
      • Utami D.
      • Matsuyama R.
      • Paasche-Orlow M.
      Improving access to online health information with conversational agents: A randomized controlled experiment.
      ]). Although the intervention was a year in length, it also misses true postintervention follow-up measures to see whether users maintained behavior change or relapsed. This study also misses true health outcome measures, such as actual birth outcomes and maternal mortality rates.

      Conclusion

      Conversational agents represent a promising medium for AYA women's health education and counseling, especially for sensitive topics such as sexual and reproductive health. Our cohort of younger, African American women used the system over the course of a year and reported high levels of satisfaction with the experience. Comparison with other online interventions is very difficult, given the heterogeneity of intervention designs. In a recent systematic review, Clarke, et al. [
      • Clarke A.M.
      • Kuosmanen T.
      • Barry M.M.
      A systematic review of online youth mental health promotion and prevention interventions.
      ] reported on 28 studies of online AYA mental health interventions. While the median dropout rate was 19.6% across the studies, the typical intervention focused on only a single health behavior and had a median intervention period of only six weeks, compared with our PCC intervention addressing an average of 23.2 behaviors per women over an intervention period of one year.
      Future research should further explore the acceptability and efficacy of conversational agents on smartphones (as in the study by Magnani et al. [
      • Magnani J.
      • Schlusser C.
      • Kimani E.
      • et al.
      The atrial fibrillation health literacy information technology system: Pilot assessment.
      ]) and other technology platforms that AYA women use regularly. Future work should also validate whether conversational agents are actually cost-effective compared with obtaining similar advice from human health providers, given that most of our users indicated they had a slight preference for receiving PCC information from a health-care provider at the end of the 12-month intervention (at 6 months, they were ambivalent).

      Funding Sources

      This work was supported by U.S. National Institutes of Health grant R01MD006213. The first author wrote the first draft of the manuscript. No payment was received by any author for production of the manuscript.

      References

        • Johnson K.
        • Posner S.F.
        • Biermann J.
        • et al.
        Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
        MMWR Recomm Rep. 2006; 55: 1-23
        • Jack B.W.
        • Atrash H.
        • Coonrod D.V.
        • et al.
        The clinical content of preconception care: An overview and preparation of this supplement.
        Am J Obstet Gynecol. 2008; 199: S266-S279
        • van der Zee B.
        • de Beaufort I.
        • Temel S.
        • et al.
        Preconception care: An essential preventive strategy to improve children's and women's health.
        J Public Health Policy. 2011; 32: 367-379
        • Sawyer S.M.
        • Afifi R.A.
        • Bearinger L.H.
        • et al.
        Adolescence: A foundation for future health.
        Lancet. 2012; 379: 1630-1640
        • Viner R.M.
        • Ozer E.M.
        • Denny S.
        • et al.
        Adolescence and the social determinants of health.
        Lancet. 2012; 379: 1641-1652
        • Kessler R.C.
        • Berglund P.
        • Demler O.
        • et al.
        Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication.
        Arch Gen Psychiatry. 2005; 62: 593-602
        • M'Hamdi H.I.
        • van Voorst S.F.
        • Pinxten W.
        • et al.
        Barriers in the uptake and delivery of preconception care: Exploring the views of care providers.
        Matern Child Health J. 2017; 21: 21-28
        • Mazza D.
        • Chapman A.
        Improving the uptake of preconception care and periconceptional folate supplementation: What do women think?.
        BMC Public Health. 2010; 10: 786
        • Lenhart A.
        Teens, social media & technology: Overview 2015.
        Pew Research Center, 2015 (Available at:)
        • Paperny D.M.
        Computerized health assessment and education for adolescent HIV and STD prevention in health care settings and schools.
        Health Educ Behav. 1997; 24: 54-70
        • Schwarz E.B.
        • Burch E.J.
        • Parisi S.M.
        • et al.
        Computer-assisted provision of hormonal contraception in acute care settings.
        Contraception. 2013; 87: 242-250
        • Martin J.
        • Hamilton B.
        • Osterman M.
        • et al.
        Births: Final data for 2012.
        Natl Vital Stat Rep. 2013; 62: 1-68
        • Bickmore T.
        • Pfeifer L.
        • Byron D.
        • et al.
        Usability of conversational agents by patients with inadequate health literacy: Evidence from two clinical trials.
        J Health Commun. 2010; 15: 197-210
        • Bickmore T.
        • Utami D.
        • Matsuyama R.
        • Paasche-Orlow M.
        Improving access to online health information with conversational agents: A randomized controlled experiment.
        J Med Internet Res. 2016; 18
        • Bickmore T.
        • Trinh H.
        • Olafsson S.
        • et al.
        Patient and consumer safety risks when using conversational assistants for medical information: An observational study of Siri, Alexa, and Google Assistant.
        J Med Internet Res. 2018; 20
        • Clark H.H.
        Using language.
        Cambridge University Press, Cambridge1996
        • Reiter E.
        • Dale R.
        Building natural language generation systems.
        Cambridge University Press, Cambridge2000
        • Bickmore T.
        • Picard R.
        Establishing and maintaining long-term human-computer Relationships.
        ACM T Comput Hum Int. 2005; 12: 293-327
        • Bickmore T.
        • Gruber A.
        • Picard R.
        Establishing the computer-patient working alliance in automated health behavior change interventions.
        Patient Educ Couns. 2005; 59: 21-30
        • Cassell J.
        • Vilhjálmsson H.
        • Bickmore T.
        BEAT: The behavior expression animation toolkit.
        SIGGRAPH '01, Los Angeles, CA2001: 477-486
        • Prochaska J.
        • Velicer W.
        The transtheoretical model of health behavior change.
        Am J Health Promot. 1997; 12: 38-48
        • Miller W.
        • Rollnick S.
        Motivational interviewing: Preparing people for change.
        3rd ed. Guilford Press, New York2012
        • Ren J.
        • Bickmore T.
        • Hempstead M.
        • Jack B.
        Birth control, drug abuse, or domestic violence? What health risk topics are women willing to discuss with a virtual agent?.
        in: Bickmore T. Marsella S. Sidner C. Intelligent virtual agents. IVA 2014. Lecture notes in computer science. Vol. 8637. Springer, Cham2014
        • Johnson S.S.
        • Pavia A.L.
        • Cummins C.O.
        • et al.
        Transtheoretical model-based multiple behavior intervention for weight management: Effectiveness on a population basis.
        Prev Med. 2008; 46: 238-246
        • Prochaska J.
        Multiple health behavior research represents the future of preventive medicine.
        Prev Med. 2008; 46: 281-285
        • Evers K.
        • Quintiliani L.
        Advances in multiple health behavior change research.
        Transl Behav Med. 2013; 3: 59-61
        • Davis T.C.
        • Long S.W.
        • Jackson R.H.
        • et al.
        Rapid estimate of adult literacy in medicine: A shortened screening instrument.
        Fam Med. 1993; 25: 391-395
        • Lincoln A.
        • Paasche-Orlow M.
        • Cheng D.
        • et al.
        Impact of health literacy on depressive symptoms and mental health-related: Quality of life among adults with addiction.
        J Gen Intern Med. 2006; 21: 818-822
        • Mancuso C.
        • Rincon M.
        Impact of health literacy on longitudinal asthma outcomes.
        J Gen Intern Med. 2006; 21: 813-817
        • Clarke A.M.
        • Kuosmanen T.
        • Barry M.M.
        A systematic review of online youth mental health promotion and prevention interventions.
        J Youth Adolesc. 2015; 44: 90-113
        • Magnani J.
        • Schlusser C.
        • Kimani E.
        • et al.
        The atrial fibrillation health literacy information technology system: Pilot assessment.
        JMIR Cardio. 2017; 1: e7