If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Preconception care is important for all women to improve infant and maternal health outcomes and may be especially important for adolescents and young adults. This study assesses the acceptance, usability, and use of an automated intervention to screen women on 108 preconception care risks and address them over the course of a year via a Web-based virtual animated health counselor and compares these measures for the adolescent and young adult users aged 18–25 years with those of users aged 26–34 years. We hypothesize that the younger cohort will have significantly greater use of and satisfaction with the online intervention.
Methods
A randomized controlled trial involving a national sample of 528 women was conducted. We present a secondary data analysis on the system use and self-reported usability and satisfaction of the 79 women aged 18–25 years randomized to the intervention group, compared with the 183 women aged 26–34 years in the intervention group. Participants were required to self-identify as female, black or African American, aged 18–34 years, not pregnant, and English-speaking and were recruited through a variety of advertisements and outreach activities.
Results
Of the adolescent and young adult participants (aged 18–25 years) enrolled and randomized to the intervention, 20.25% of participants accessed the system 0 times; 29.11%, 1–3 times; and 50.63%, >3 times over the course of a year. At the end of the year, almost all (96.4%) indicated they had either acted on recommendations made by the agent or planned to. Most (75.0%) said they would recommend the system to someone they knew. There were no significant differences between the two age groups on intervention use or satisfaction.
Conclusions
Web-based conversational agents are a viable medium for delivering longitudinal preconception care counseling to adolescents and young adults.
Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
]. PCC involves many aspects of health care, spanning 83 topics in 14 clinical areas, including diverse topics such as family planning, lifestyle behaviors including physical activity and weight management, immunizations, substance abuse, infectious disease treatment, and chronic disease management [
Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
]. However, adolescence may be a particularly important time to begin PCC for a variety of reasons. First, because many aspects of PCC require longitudinal behavior change (e.g., lifestyle behaviors such as diet and exercise, chronic disease self-care management, long-term adherence to folic acid, etc.), months or years may be required for an intervention target to be reached, indicating these interventions should be started as early as possible. Second, adolescence is when many health-related behaviors are established for adulthood, indicating it as the ideal time to begin healthy PCC behaviors to be maintained throughout adulthood [
]), indicating this is an ideal time to conduct screening and early treatment.
Practitioners do an imperfect job addressing preventive health issues for adolescents in general because of a range of barriers including lack of time, discomfort discussing sensitive topics, lack of skill, and lack of confidence [
Automated interventions that systematically screen for all known PCC risk factors and provide longitudinal support following best practices from behavioral medicine may represent an ideal mechanism to help adolescent and young adult (AYA) women address their unique set of PCC risks. Technology in general (computers, smartphones, etc.) is widely used by adolescents [
], and there is growing evidence that technology-based interventions may be especially effective for adolescents, particularly regarding sexual and reproductive health issues and other sensitive topics [
In this work, we report the development and evaluation of an automated intervention for PCC among AYA women (Figure 1). The intervention first screens women for 108 PCC risk factors using an online survey and then uses a Web-based conversational agent that plays the role of a health counselor that talks to women over the course of a year to address their PCC risks. Because PCC is particularly important for African American women, given that they have twice the risk of delivering a low-birth-weight infant and four times the risk of maternal mortality, compared with white women [
The “Gabby” preconception care conversational agent
The intervention uses an animated, embodied conversational agent to deliver simulated face-to-face health counseling to women. Several studies have demonstrated that adult patients and consumers find this modality acceptable and effective, particularly among those with inadequate health literacy [
]. The agent uses both verbal and nonverbal conversational behavior to provide a natural and intuitive computer interface. It talks using synthetic speech and animated coverbal behavior, such as hand gestures and facial displays, and the user responds primarily by selecting what they want to say from a multiple-choice menu of options updated during each turn of dialog. The interface does not rely on unconstrained natural language understanding (allowing users to say anything they want), which can represent a safety issue in automated health counseling systems [
Patient and consumer safety risks when using conversational assistants for medical information: An observational study of Siri, Alexa, and Google Assistant.
], allowing every agent utterance to be completely tailored to each user and the current discourse context (e.g., what was just discussed in the current or past conversations, as described previously in the studies by Bickmore and Picard [
There are several reasons why conversational agents provide an effective medium for health communication with patients, especially those with limited literacy. First, the interface relies only minimally on text and uses the universally understood format of face-to-face conversation, thus making it less intimidating and more accessible to patients with limited literacy skills. Second, agents can emphasize critical information using nonverbal behavior the same way people do in conversation (e.g., using hand gesture, eyebrow raise, and/or speech prosody). Third, the use of nonverbal conversational behaviors, such as hand gestures that convey specific information through pointing (“deictic” gestures) or through shape or motion (“iconic” and “metaphoric” gestures), can provide a redundant medium for conveying semantic content to enhance message comprehension. For example, the agent can point at a patient's hip when describing where to wear a pedometer to reinforce and clarify the instruction.
Health behavior change framework
Our overarching approach to addressing each individual PCC risk is the transtheoretical, or “stages of change” model of health behavior change. The transtheoretical model posits that individuals go through a series of five discrete stages, from “precontemplation” to “maintenance” in the process of changing their behavior, and that the behavior change techniques that should be used for an individual at a given time should be selected based on their stage of change [
]. For individuals in “precontemplation” (not yet intending to change), we use counseling techniques from motivational interviewing to help them resolve them ambivalence and take first steps in addressing a risk [
]. Other behavior change techniques are used for individuals in later stages of change, for example, goal setting and positive reinforcement for behaviors requiring incremental, longitudinal change, such as exercise and diet.
Multiple health behavior change
Most automated health behavior change interventions developed to date focus on changing only one behavior at a time, and many do not provide ongoing support over the weeks or months that change may require. However, PCC almost always involves intervening on many health behaviors: in our pilot work, we found that women had 23.2 (standard deviation: 6.12) PCC risks on average, and of these, women chose to address an average of 6.3 (standard deviation: 7.2) risks with the conversational agent during a six-month intervention period in which they could access the agent at any time over the Web [
One approach to changing multiple behaviors is to address them sequentially; only addressing a new behavior once a prior one has been successfully changed. Individuals who have some success changing one target behavior have been found to be more likely to achieve success on a second behavior [
]. However, some behaviors can take weeks or months to address, making an overall sequential intervention potentially years in duration. Some researchers have identified “bundles” of behaviors that can effectively be addressed at the same time [
]. Actions that address multiple behaviors can reduce the total perceived effort that an individual takes and increases her confidence to change. Multiple health behavior change has also been shown to be economically advantageous [
Our automated PCC intervention uses a visual customizable list of identified risks and recommendations to minimize or eliminate these risks, called a “My Health To-Do List” (Figure 2). This aid helps users keep track of their risks and progress over time and provides feedback on actions taken and risks addressed to provide positive reinforcement and motivation to continue. After the initial PCC risk survey, the agent introduces the “My Health To-Do List” and lets users select the identified risks they want to discuss. The agent then asks questions to identify the user's stage of change for addressing the risk [
All aspects of the agent's appearance, voice, the design of the “My Health To-Do List,” and the counseling dialogs were designed with significant feedback from focus groups that are comprised of women from the target population.
Methods
This is a secondary data analysis from an unpublished randomized controlled trial (ClinicalTrials.gov Identifier: NCT01827215). The parent trial compared African American women aged 18–34 years who used the Gabby PCC intervention (n = 262) with a control group who only received a report describing their PCC risks (n = 266), with primary outcomes based on aggregate change in “stage of change” for each of their identified PCC risks. After enrollment, all participants took an online risk assessment, and those in the intervention group were then provided with 12 months' access to the Web-based conversational agent, with outcomes collected from all participants at six and 12 months after randomization.
Here, we report on the experiences of the intervention group with the intervention itself for the 168 AYA women aged 18–25 years in the study (of 528 total), focusing on the acceptance and use of the system. We hypothesize that the AYA women users will have significantly greater use of and satisfaction with the online intervention over the course of the one-year intervention period than users aged 26–34 years. The study was approved by the Boston Medical Center Institutional Review Board.
Measures
Personal characteristics factors
We assessed participants' ethnic origin, primary use of language, education background, marital status, employment status, and household income. We also assessed participants' previous experience with computers and their attitudes toward using computers.
Health literacy
Health literacy is assessed using the Rapid Estimate of Adult Literacy in Medicine (REALM) instrument [
]. Scores range from 0 to 66, with higher scores indicating higher levels of health literacy.
System usage
System usage is tracked in log files and assessed using the number of sessions that participants had with the virtual counselor during the study period and the time (in minutes) that participants spent on interacting with the virtual counselor.
Satisfaction
Six single-item scale questions were asked to assess participants' satisfaction with the virtual counselor: [
Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
] “If given opportunity, would you like to talk with Gabby again?”
Results
The current analysis consisted of 79 female participants aged 25 years or younger, compared with the 183 participants aged 26–34 years. Their baseline demographics and personal characteristics are shown in Table 1. The two cohorts were primarily African American (80% of the AYA women, 83% of those aged 26–34 years), primarily non-Hispanic (92%, 96%), college educated (42%, 33%), single (82%, 58%), and had full-time employment (43%, 63%). Both groups were also self-reported “expert” computer users (58%, 50%) and had adequate health literacy (REALM scores: 63.6, 63.8). These characteristics do differ from those of the general US population, given our focus on African American women aged 18–34 years.
Table 1Descriptive statistics of intervention participant characteristics by age group
Participant characteristics
AYA women, age 18–25 years (N = 79)
AYA women, age 26–34 years (N = 183)
Age, mean (SD)
22.75 (1.93)
29.58 (2.42)
Ethnic origin, n (%)
African American
63 (80)
151 (83)
Cape Verdean
1 (1)
4 (2)
Caribbean
12 (15)
21 (11)
Ethiopian
1 (1)
2 (1)
Somalian
1 (1)
1 (1)
Nigerian
8 (10)
8 (4)
Kenyan
0 (0)
0 (0)
Other
13 (16)
41 (22)
Hispanic, n (%)
No, not of Hispanic, Latino, or Spanish origin
73 (92)
176 (96)
Yes, Mexican, Mexican American, Chicano
1 (1)
0 (0)
Yes, Puerto Rican
1 (1)
2 (1)
Yes, Cuban
0 (0)
0 (0)
Yes, another Hispanic, Latino, or Spanish origin
4 (5)
5 (3)
Language, n (%)
English
77 (97)
172 (94)
Other language
2 (3)
11 (6)
Education, n (%)
Less than high school (0 through 8)
0 (0)
0 (0)
Some high school (9 through <12)
2 (3)
3 (2)
GED or high school equivalency
0 (0)
5 (3)
High-school graduate
9 (11)
7 (4)
Attended vocational, trade, or business school after high school
A total of 63 AYA participants (79.7% of those enrolled and randomized to intervention) used the system during the study period, compared with 136 aged 26–34 years (74.32% of those enrolled, Table 2). AYA users held a median of 6 (interquartile range: 9) sessions with the virtual counselor during the 12-month intervention period, compared with a median of 6 (interquartile range: 7) sessions for the older cohort. Overall, 20.25% of the AYA users interacted zero times with Gabby, 29.11% interacted 1–3 times, and 50.63% interacted more than 3 times over the course of the year, compared with 25.68%, 28.96%, and 45.36% for the older cohort, respectively. There were no significant differences across any of these metrics between the two age groups.
We conducted an analysis of the 16 AYA and 47 older users who did not use Gabby (Table 3). We did not find any significant systematic differences across usage patterns for participants based on education (high-school graduate vs. not), employment (full-time vs. not), computer literacy (self-proclaimed “computer expert” vs. not), or health literacy (adequate vs. inadequate) based on a REALM score of 61 or greater (ninth grade and above), as has been done in several prior studies [
When asked about ease of use, 62.9% of AYA respondents rated ease of use a one or two on a scale of 1 (“Easy”) to 7 (“Difficult”) at six months, increasing slightly to 67.9% at 12 months (Table 4). Only 6.1% of AYA users rated Gabby as difficult to use (greater than neutral rating). Participants indicated that the length of their conversations with Gabby were “about right” (median 4) on a scale of 1 (“too short”) to 7 (“too long”) at both six and 12 months. There were no significant differences between AYA users and those in the older cohort on usability measures.
Satisfaction
Participants trusted Gabby, scoring a median of two on a scale of 1 (“very much”) to 7 (“not at all”) at both six and 12 months, and 80.0% indicated that Gabby answered their questions about PCC at six months, increasing to 85.7% at 12 months (Table 4). When asked if there were things about their health they were not comfortable telling Gabby, 88.6% of respondents said “no” at six months, changing to 78.6% at 12 months.
When asked if they would recommend Gabby to someone they knew, 71.4% of respondents said they would at six months, increasing to 75.0% at 12 months. When asked if they would like to talk to Gabby again, 70.6% indicated they would at six months, increasing to 71.4% at 12 months.
Participants did not feel that Gabby was a replacement for a health-care provider. When asked if they would have rather talked to a provider, 57.1% indicated they would have, with another 22.9% being neutral at six months, changing to 66.7% and 22.2% at 12 months, respectively.
There were no significant differences between AYA users and those in the older cohort on satisfaction measures.
Use of information provided by Gabby
When asked whether they had already followed the recommendations provided by Gabby, 60.0% of respondents said they had at six months (increasing to 71.4% at 12 months), while another 34.3% at six months said they planned to in the future (25.0% at 12 months).
Discussion
Overall, the AYA cohort in our study found that interacting with the conversational agent about topics related to PCC was acceptable, with many using the system frequently over the 12 months of the intervention. At both six and 12 months, they reported high levels of trust in the agent and satisfaction with the intervention and reported that they followed (or would follow) Gabby's recommendations. Although they would rather talk to their provider about PCC, they indicated that they would use the system again if they could and would recommend it to someone else, indicating its utility in providing PCC information and counseling when women are unable to see their provider.
We did not find any significant differences between AYA users and older users (aged 26–34 years) randomized to receive the Gabby intervention on any measure.
Limitations
The study has several limitations, including the small sample used and the lack of a true control group for usability measures (e.g., comparing with a non–agent-based automated intervention, as in the study by Bickmore et al. [
]). Although the intervention was a year in length, it also misses true postintervention follow-up measures to see whether users maintained behavior change or relapsed. This study also misses true health outcome measures, such as actual birth outcomes and maternal mortality rates.
Conclusion
Conversational agents represent a promising medium for AYA women's health education and counseling, especially for sensitive topics such as sexual and reproductive health. Our cohort of younger, African American women used the system over the course of a year and reported high levels of satisfaction with the experience. Comparison with other online interventions is very difficult, given the heterogeneity of intervention designs. In a recent systematic review, Clarke, et al. [
] reported on 28 studies of online AYA mental health interventions. While the median dropout rate was 19.6% across the studies, the typical intervention focused on only a single health behavior and had a median intervention period of only six weeks, compared with our PCC intervention addressing an average of 23.2 behaviors per women over an intervention period of one year.
Future research should further explore the acceptability and efficacy of conversational agents on smartphones (as in the study by Magnani et al. [
]) and other technology platforms that AYA women use regularly. Future work should also validate whether conversational agents are actually cost-effective compared with obtaining similar advice from human health providers, given that most of our users indicated they had a slight preference for receiving PCC information from a health-care provider at the end of the 12-month intervention (at 6 months, they were ambivalent).
Funding Sources
This work was supported by U.S. National Institutes of Health grant R01MD006213. The first author wrote the first draft of the manuscript. No payment was received by any author for production of the manuscript.
References
Johnson K.
Posner S.F.
Biermann J.
et al.
Recommendations to improve preconception health and health care--United States. A report of the CDC/ATSDR Preconception Care Work Group and the Select Panel on Preconception Care.
Patient and consumer safety risks when using conversational assistants for medical information: An observational study of Siri, Alexa, and Google Assistant.
Conflicts of interest: The authors have no conflicts of interest to disclose.
Disclaimer: This article was published in a Supplement supported by a National Science Foundation Smart and Connected Health Grant (SCH) to Principal Investigators Elizabeth Ozer, Ph.D. and James Lester, Ph.D. (IIS-1344670 & IIS-1344803). Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views of the National Science Foundation. The study sponsors were not involved in any aspects of the study design; the collection, analysis, and interpretation of data; the writing of the report; or the decision to submit the manuscript for publication.
30449-5&id=gr1.jpg){kind=link}
30449-5&id=gr2.jpg){kind=link}