Advertisement

Artificial Intelligence for Personalized Preventive Adolescent Healthcare

      Abstract

      Recent advances in artificial intelligence (AI) are creating new opportunities for personalizing technology-based health interventions to adolescents. This article provides a computer science perspective on how emerging AI technologies—intelligent learning environments, interactive narrative generation, user modeling, and adaptive coaching—can be utilized to model adolescent learning and engagement and deliver personalized support in adaptive health technologies. Many of these technologies have emerged from human-centered applications of AI in education, training, and entertainment. However, their application to improving healthcare, to date, has been comparatively limited. We illustrate the opportunities provided by AI-driven adaptive technologies for adolescent preventive healthcare by describing a vision of how future adolescent preventive health interventions might be delivered both inside and outside of the clinic. Key challenges posed by AI-driven health technologies are also presented, including issues of privacy, ethics, encoded bias, and integration into clinical workflows and adolescent lives. Examples of empirical findings about the effectiveness of AI technologies for user modeling and adaptive coaching are presented, which underscore their promise for application toward adolescent health. The article concludes with a brief discussion of future research directions for the field, which is well positioned to leverage AI to improve adolescent health and well-being.

      Keywords

      Implications and Contributions
      This article describes a computer science perspective on the opportunities and challenges in utilizing artificial intelligence (AI) to drive adaptive technologies in adolescent preventive healthcare. Several AI technologies are discussed, including intelligent learning environments, interactive narrative generation, user modeling, and adaptive coaching. AI applications raise important issues related to privacy, ethics, encoded bias, and integration within clinic workflows and adolescent lives, but the potential of personalized health interventions enabled by AI-driven adaptive technologies is significant and holds considerable promise.
      A key promise of health information technologies is their capacity to personalize health interventions to individual adolescents [

      Bickmore T, Zhang Z, Reichert M, et al. Promotion of preconception care among adolescents and young adults by conversational agent. J Adolesc Health. In press (Suppl).

      ,
      • Hekler E.B.
      • Michie S.
      • Pavel M.
      • et al.
      Advancing models and theories for digital behavior change interventions.
      ,
      • Orji R.
      • Vassileva J.
      • Mandryk R.
      Modeling the efficacy of persuasive strategies for different gamer types in serious games for health.
      ]. Many technology-based preventive health interventions are one-size-fits-all, or their support for personalization is limited. However, the potential of personalization within technology-based health interventions is broad and significant. Personalization promises to support a wide range of adolescent health interventions through responding to individual differences in age, gender, and background, predicting key health indicators, and supporting motivation, engagement, learning, and ultimately, well-being.
      We are in the midst of a dramatic technological transformation brought about by major advances in artificial intelligence (AI). AI encompasses a range of computational tools and methods for creating intelligent systems that can perceive, reason, learn, and act, and which can be applied toward solving problems in many domains, including education [
      • Kulik J.A.
      • Fletcher J.D.
      Effectiveness of intelligent tutoring systems: A meta-analytic review.
      ], training [
      • Henderson N.
      • Rowe J.
      • Mott B.
      • et al.
      4D affect detection: Improving frustration detection in game-based learning with posture-based temporal data fusion.
      ], and increasingly, healthcare [
      • Topol E.
      Deep medicine: How artificial intelligence can make healthcare human again.
      ]. For example, advances in computer vision are creating new opportunities for diagnosing disease from image data, such as MRI scans and retinal fundus photographs. Improvements in automated pattern recognition are driving progress in drug discovery and genetic analysis. Machine learning is being utilized to analyze patterns in smartwatch sensor data to detect cardiac issues such as atrial fibrillation.
      An emerging opportunity for leveraging AI to advance adolescent health is the design, development, and evaluation of adaptive technologies for personalized preventive healthcare. AI-driven personalized health technologies have several prospective benefits. AI can be used to model how adolescents engage with adaptive technologies to develop a data-rich understanding of adolescent traits and knowledge [
      • Pelánek R.
      The details matter: Methodological nuances in the evaluation of student models.
      ]. AI can also be used to dynamically generate or tailor interventions in response to adolescent interactions with technology [
      • Wang P.
      • Rowe J.P.
      • Min W.
      • et al.
      High-fidelity simulated players for interactive narrative planning.
      ]. Furthermore, AI can be used to devise predictive models that measure key health indicators to inform adolescent behavior and provide analytics to care providers. In short, AI can serve as the foundation for personalized health technologies that engage adolescents in their own health in ways that are personally meaningful, relevant, and effective.
      In this article, we describe emerging opportunities to leverage AI for enabling adaptive health interventions that are deeply personalized to adolescents. We provide a computer science perspective on the opportunities and challenges raised by AI, and we examine specific AI technologies that show particular promise, including intelligent learning environments, interactive narrative generation, user modeling, and adaptive coaching. Research on adaptive health technologies calls for the development of multidisciplinary partnerships that feature close collaboration between health researchers, computer scientists, and health-care providers. By integrating theoretically grounded health interventions with state-of-the-art AI, we anticipate the creation of tools and methods that extend the reach of clinicians and empower young people toward achieving significant improvements in health.

      Illustrative scenario

      To illustrate the vision of adaptive health technologies for personalized preventive adolescent healthcare, consider the following scenario. (The scenario is based on the work by Ozer et al. [
      • Ozer E.
      • Rowe J.
      • Tebb K.
      • et al.
      Fostering engagement in health behavior change: Iterative development of an interactive narrative environment to enhance adolescent preventive health services.
      ] on the design and development of Inspire, a narrative-centered behavior change environment for adolescent preventive healthcare focused on alcohol use.) Camila is a 14-year-old teenager who has arrived at the primary care clinic for her annual checkup. Camila recently started high school, and she is struggling to orient herself in the new social environment that she finds herself in at school. Although she may not be fully aware of it, Camila is at a developmental stage that is formative for her in establishing positive health behaviors.
      As Camila awaits her appointment in the clinic waiting room, she uses a clinic-provided tablet to complete an online screening tool to report her recent history related to various types of health behaviors, including risky behavior, such as smoking and consuming alcohol. Based on her responses, Camila is provided the opportunity to watch a 90-second video trailer about an interactive narrative video game that is freely available through the digital app store on her smartphone. The video trailer introduces a storyline about a group of teenage friends planning a social get-together that goes awry. Camila is intrigued to learn what will happen to the characters, and she looks forward to downloading the game to explore the story further when she returns home.
      After Camila returns home following her visit to see her primary care provider, she uses a link she received from the online screening tool to access the personalized interactive narrative game on her smartphone. Behind the scenes, the game is powered by an adaptive health behavior change system, including a machine learning-based interactive narrative generator that utilizes anonymized interaction data from prior adolescents, as well as an online, private user model that was earlier updated based on Camila's responses to the online screening tool, to drive a set of personalized behavior change narrative generation policies. These policies determine how the game dynamically adapts narrative events, character behaviors, and scaffolding in the storyline based on Camila's individual characteristics and gameplay choices. Camila begins to explore the interactive narrative, which presents a series of story-centric problem scenarios involving the potential for engagement in risky health behaviors, including peer pressure, social norms, and the consequences of alcohol use. Throughout these scenarios, Camila engages with a group of virtual teenage characters who respond to Camila's conversational decisions through a combination of speech, gestures, and facial expressions that are automatically generated by the game's AI-based animation engine.
      As Camila explores the interactive narrative, her choices drive how events unfold in the storyline. The system provides adaptive coaching to support Camila's self-regulatory process, prompting her to select goals for handling the social gathering and reflect on how different decisions might impact her goals. In effect, Camila is engaged with an AI-driven preventive health intervention, gaining practical experience with problem-solving strategies that are critical for building self-efficacy to navigate real-world situations that she may encounter in her own life.
      Midway through the narrative, Camila's avatar receives pressure from a virtual adolescent character to accept an alcoholic drink. Based on Camila's risk profile, the system adapts the narrative to provide just-in-time information about the effects of alcohol on an adolescent's brain, and it directs another virtual character to model a positive behavior strategy for handling peer pressure. Camila observes how the virtual teenager believably handles the situation, and she chooses a similar response for her avatar, successfully navigating the difficult social decision while adopting a strategy that promotes prosocial behavior and minimizes health risks.
      Camila's interactions with the intelligent learning environment continue in this manner as she engages with a series of dynamically generated narrative episodes over the course of four successive weeks. Throughout Camila's interactions with the game, her software interaction logs are anonymized and recorded by the system to serve as additional training data for the machine learning–based narrative generation system. In addition, Camila's responses to subsequent alcohol screens and questionnaires, which are administered between narrative episodes as well as after the final episode, are provided to the system to assess the impacts of the personalized behavior change interactive narratives on Camila's alcohol use and self-efficacy.
      Following Camila's usage of the game, her care provider obtains a summary report relaying how many times Camila logged into the system, how many episodes have been completed, how long she interacted with the system, and her self-reported alcohol use. The report provides a launching point for future communication with Camila about alcohol use and health behavior change.

      AI-driven adaptive technologies for adolescent preventive healthcare

      In recent years, major advances in AI can be largely attributed to improvements in machine learning, and in particular, in deep learning. Machine learning is a subfield of AI that centers on developing computer programs that learn to solve tasks by recognizing complex patterns in data rather than by being explicitly programmed [
      • Russell S.J.
      • Norvig P.
      Artificial intelligence: A modern approach.
      ]. Deep learning is a family of machine learning techniques that leverage artificial neural networks, a type of computational approach for representing complex, nonlinear mathematical functions that model patterns in large multidimensional data sets [
      • LeCun Y.
      • Bengio Y.
      • Hinton G.
      Deep learning.
      ]. Deep learning techniques have yielded dramatic advances in foundational AI tasks such as computer vision, speech recognition, and text understanding.
      The breadth of AI as it applies to adaptive health technologies, however, extends far beyond the machine learning techniques and applications described in the previous paragraph. A major frontier in AI is human-centered AI, which is situated at the intersection of AI and human-computer interaction, and it focuses on identifying how to design, develop, and investigate intelligent systems that support and enhance human experiences augmented with technology [
      • Ford K.M.
      • Hayes P.J.
      • Glymour C.
      • Allen J.
      Cognitive orthoses: Toward human-centered AI.
      ]. Research and development of adaptive technologies for preventive healthcare is an instantiation of human-centered AI, and it points toward the opportunity for investigating how AI techniques can be utilized to solve critical challenges of preventive health, and conversely, how open questions in preventive health can motivate algorithmic advances in AI techniques and methodologies.
      Within prevention, adaptive technologies have significant promise both as a health intervention tool and a health screening tool. Health information technologies, such as digital games and mobile devices, provide a natural vehicle for delivering personalized interventions that operationalize evidence-based strategies for affecting health behavior change. They extend the reach of primary care interventions, complementing adolescents' face-to-face interactions with their health-care provider, as well as promoting follow-through care. Conversely, adolescents' interactions with these technologies can also serve as a “dependent variable,” unobtrusively generating evidence by which to assess adolescents' health risks and attributes.
      A key capability of technology-based health interventions is their capacity to produce rich data streams about users' interactions with the technology. Detailed records of user interactions can be securely logged, timestamped, stored, and analyzed. These trace data logs include records of interaction events, such as button presses, text entries, and menu selections, as well as low-level user inputs, such as moment-to-moment mouse movements, keystrokes, touchscreen taps, and gestures. Increasingly, information technologies can also capture multimodal sensor data, such as speech, facial expression, gaze, location, and biometric data. Taken together, these multichannel data streams can serve as the raw input to machine learning systems for processing, analyzing, and triangulating key variables, such as measures of adolescent health and behavior [
      • Baltrušaitis T.
      • Ahuja C.
      • Morency L.P.
      Multimodal machine learning: A survey and taxonomy.
      ].
      Research and development on adaptive technologies for adolescent preventive healthcare has several parallels with applications of human-centered AI in related fields, such as education and training [
      • Kulik J.A.
      • Fletcher J.D.
      Effectiveness of intelligent tutoring systems: A meta-analytic review.
      ,
      • Pelánek R.
      The details matter: Methodological nuances in the evaluation of student models.
      ]. Research on AI in education spans several decades, and it has produced a rich literature on intelligent tutoring systems [
      • VanLehn K.
      The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems.
      ], student modeling [
      • Pelánek R.
      The details matter: Methodological nuances in the evaluation of student models.
      ], educational data mining [
      • Baker R.S.
      Educational data mining: An advance for intelligent systems in education.
      ], and computational models of affect and engagement [

      Graesser A. Emotions are the experiential glue of learning environments in the 21st century. Learn Instr. In press, https://doi.org/10.1016/j.learninstruc.2019.05.009

      ]. Intelligent tutoring systems have been shown to produce effect sizes that are comparable to one-on-one human tutoring with regard to student learning outcomes or approximately 0.7 standard deviations over conventional methods such as classroom instruction [
      • Kulik J.A.
      • Fletcher J.D.
      Effectiveness of intelligent tutoring systems: A meta-analytic review.
      ]. By leveraging run-time models of student knowledge, misconceptions, and related cognitive-affective states, intelligent learning environments can deliver carefully timed feedback, scaffolding, and problem scenarios to guide students toward highly effective learning experiences [
      • VanLehn K.
      The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems.
      ].
      Adaptive technologies for adolescent preventive healthcare share several key similarities with intelligent learning environments. Both use AI-driven personalization to enrich adolescent interactions with technology; both rely upon run-time models of assessment to inform how personalization is enacted for individual users; and both are designed to support users toward achieving, in the short term, positive outcomes, and in the longer term, effective self-regulation. The opportunity for cross-pollination between the two areas is significant and bidirectional.
      Another related area is interactive narrative generation, which integrates AI and commercial game engine technologies to generate immersive, interactive story experiences that are dynamically tailored to individual users [
      • Cavazza M.
      • Young R.
      Introduction to interactive storytelling.
      ]. In contrast to traditional narrative media such as books, animation, and film, interactive narrative generation enables the creation of rich, story-centric experiences in which users are active participants who shape the events and outcomes of an unfolding narrative. A range of computational approaches have been investigated for interactive narrative generation, most recently machine learning techniques such as collaborative filtering [
      • Yu H.
      • Riedl M.
      Personalized interactive narratives via sequential recommendation of plot points.
      ], dynamic Bayesian networks [
      • Lee S.
      • Rowe J.
      • Mott B.
      • Lester J.
      A supervised learning framework for modeling director agent strategies in educational interactive narrative.
      ], and reinforcement learning [
      • Wang P.
      • Rowe J.P.
      • Min W.
      • et al.
      High-fidelity simulated players for interactive narrative planning.
      ,
      • Rowe J.
      • Lester J.
      Improving student problem solving in narrative centered learning environments: A modular reinforcement learning framework.
      ]. These latter techniques comprise a family of data-driven systems that dynamically personalize interactive narratives by training generative models “bottom-up” from corpora of example story data. In healthcare, interactive narrative technologies are a natural fit for creating technology-based health interventions that are highly engaging and personalized to adolescents. Adolescents may identify with their virtual avatars, as well as other characters, and vicariously experience a range of health behavior scenarios with different consequences. Interactive narrative technologies have shown promise in several health applications, including narrative-centered behavior change environments for reducing adolescent risky behavior [
      • Ozer E.
      • Rowe J.
      • Tebb K.
      • et al.
      Fostering engagement in health behavior change: Iterative development of an interactive narrative environment to enhance adolescent preventive health services.
      ], interactive pedagogical dramas to support development of social problem-solving skills of relatives of pediatric cancer patients [
      • Marsella S.
      • Johnson W.L.
      • LaBore C.
      Interactive pedagogical drama for health interventions.
      ], and interactive visual novels designed to support patient empowerment in managing hospital stays [
      • Yin L.
      • Ring L.
      • Bickmore T.
      Using an interactive visual novel to promote patient empowerment through engagement.
      ].
      There is considerable potential to leverage human-centered AI techniques that have proven effective in intelligent learning environments and interactive narrative generation systems—these include user modeling and adaptive coaching technologies—toward supporting improvements in adolescent knowledge, behavior, and engagement in health (Table 1).
      Table 1Examples of AI-driven adaptive technologies for adolescent preventive healthcare
      AI-driven adaptive technologyExample applications toward adolescent preventive healthcare
      Intelligent learning environments
      • Create technology-rich health interventions that provide personalized learning experiences to enhance adolescent knowledge, self-efficacy, and problem-solving skills
      • Deliver engaging health technology experiences that serve as a screening tool for detecting potential health risks and attributes
      Interactive narrative generation
      • Generate interactive narrative vignettes that emulate real-world problem scenarios that adolescents are likely to encounter in their own lives
      • Tailor story events and character behaviors within interactive health problem scenarios to respond dynamically to adolescents' choices within the health intervention technology
      User modelling
      • Stealth assessment
      • Affective modeling
      • Goal recognition
      • Assess adolescent knowledge about relevant health concepts to inform personalized coaching in support of mastery learning
      • Recognize occurrences of negative, deactivating emotions during adolescent interactions with health technologies to predict potential disengagement
      • Model adolescents' goals during interactions with technology-based health interventions to support effective self-regulatory processes
      Adaptive coaching
      • Prompt adolescents to set goals related to their health, and follow up with guided reflection about how well they achieved their goals
      • Deliver personalized hints and feedback within technology-based health interventions to support learning

      Modeling adolescent learning and engagement

      To enable adaptive health technologies to recognize the individual characteristics of adolescent users, devising computational models of adolescent learning and engagement is essential. We briefly describe three applications of machine learning for creating user models in intelligent learning environments: (1) stealth assessment, (2) affective modeling, and (3) user goal recognition. These applications exemplify critical components of adaptive technologies designed to personalize support for individual learners. In the context of healthcare, these techniques promise to enable health technologies that model and predict adolescent knowledge, engagement, and behavior, respectively, and subsequently enable tailoring health technology interactions to adolescents.

      Stealth assessment

      Stealth assessment is the task of unobtrusively measuring student knowledge in an educational game using logs of user gameplay data as input [
      • Kim Y.J.
      • Almond R.G.
      • Shute V.J.
      Applying evidence-centered design for the development of game-based assessments in Physics Playground.
      ]. A key attribute of stealth assessment is that it is invisible: it occurs in the background as an individual interacts with a game-based learning environment, completing tasks and engaging with the software, yielding a gradually improving assessment of the student's knowledge as more evidence is collected from their interactions with the game. Stealth assessment is a framework that involves consumption of user game trace logs and pretest data as input and predicts user performance on post-test knowledge assessments as output.
      Stealth assessment in educational games has typically involved the creation of Bayesian networks, a type of probabilistic model, to capture how students' in-game behaviors provide evidence about student competencies, such as science knowledge or persistence [
      • Kim Y.J.
      • Almond R.G.
      • Shute V.J.
      Applying evidence-centered design for the development of game-based assessments in Physics Playground.
      ]. However, the creation of Bayesian network models requires a labor-intensive knowledge engineering process that involves encoding large numbers of variables and intervariable relationships that are explicitly represented in the model. Min et al. [

      Min W, Frankosky M, Mott B, et al. DeepStealth: Game-based learning stealth assessment with deep neural networks. IEEE Trans Learn Tech. In press, https://doi.org/10.1109/TLT.2019.2922356

      ] devised a deep learning–based framework for stealth assessment that utilizes long short-term memory (LSTM) networks to automate major portions of the model engineering process. In a comparison of several machine learning–based models, they found that the best performing model configuration, an LSTM network with 140 hidden units, significantly outperformed competing machine learning–based models in predicting student post-test assessment scores with respect to predictive accuracy and early prediction capacity.
      The effectiveness of stealth assessment in intelligent learning environments underscores its potential as a user modeling technique within health intervention technologies. Assessing adolescents' knowledge of health behavior and risks provides a direct mechanism for driving personalized presentation of information to impart knowledge and lay the foundation for improvements in adolescent health. However, leveraging stealth assessment within technology-based health interventions also raises important questions about the correspondence between adolescent choices within an interactive narrative game and their real-life choices that occur offline.

      Affective modeling

      Computational models of affect are integral to modeling and understanding user engagement with adaptive technologies. Recent years have witnessed significant interest in leveraging machine learning to create run-time models for automatically recognizing learner emotions [
      • D’Mello S.
      • Kappas A.
      • Gratch J.
      The affective computing approach to affect measurement.
      ]. This includes work on automatically recognizing expressions of learner emotion, modeling temporal changes in emotion over time, and interpreting learner emotions to inform delivery of adaptive support [

      Graesser A. Emotions are the experiential glue of learning environments in the 21st century. Learn Instr. In press, https://doi.org/10.1016/j.learninstruc.2019.05.009

      ]. A key objective of affective models is enabling adaptive technologies to respond to negative, deactivating emotions, such as boredom and frustration [
      • Henderson N.
      • Rowe J.
      • Mott B.
      • et al.
      4D affect detection: Improving frustration detection in game-based learning with posture-based temporal data fusion.
      ]; detect and measure positive affective states, such as flow [
      • Esteban-Millat I.
      • Martínez-López F.J.
      • Huertas-García R.
      • et al.
      Modelling students' flow experiences in an online learning environment.
      ]; and provide support to learners experiencing confusion or anxiety, which can have a significant impact on student learning outcomes [
      • D'Mello S.
      • Lehman B.
      • Pekrun R.
      • Graesser A.
      Confusion can be beneficial for learning.
      ].
      Affective models are often grouped into two categories: sensor-free models and sensor-based models. Sensor-free affective models utilize interaction trace log data as input for machine learning–based classifiers of human emotion [
      • Botelho A.
      • Baker R.
      • Ocumpaugh J.
      • Heffernan N.
      Studying affect dynamics and chronometry using sensor-free detectors.
      ]. Sensor-based affect recognition utilizes physical hardware sensors, such as webcams for tracking facial expression, eye trackers for measuring gaze, motion-tracking cameras for tracking gesture and posture, microphones for recording conversation, and biometric sensors for tracking heart rate and other physiological measures [
      • D’Mello S.
      • Kory J.
      A review and meta-analysis of multimodal affect detection systems.
      ]. Multimodal affect recognition, which combines multiple concurrent data streams using data fusion techniques, produces run-time classifications of learner emotions, and has shown promise in several contexts [
      • D’Mello S.
      • Kory J.
      A review and meta-analysis of multimodal affect detection systems.
      ].
      Obtaining “ground truth” measurements of emotion for training machine learning–based affective models, however, is a key challenge [
      • Barrett L.F.
      • Adolphs R.
      • Marsella S.
      • et al.
      Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements.
      ]. Research on affective modeling in education has largely relied upon self-report methods, observational methods, and elicitation methods to measure emotion [
      • D’Mello S.
      • Kappas A.
      • Gratch J.
      The affective computing approach to affect measurement.
      ]. However, each of these approaches has shortcomings. Emotional expressions are context sensitive, multivariate, and have complex temporal dynamics [
      • Barrett L.F.
      • Adolphs R.
      • Marsella S.
      • et al.
      Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements.
      ]. There is still a significant need for enriched methodologies for measuring learner emotion in naturalistic contexts and with finer temporal resolutions.
      Affective modeling shows significant promise for understanding adolescent engagement with technology-based health interventions. Even if a technology-based intervention is effective in the laboratory, if adolescents do not engage with it in the real-world, it is unlikely to have a meaningful impact on health outcomes. Affective modeling provides tools and methodologies for measuring key aspects of adolescent engagement, which can inform personalized support for maintaining healthy behavior, improve adherence in prevention programs, and help health-care providers understand the effectiveness of technology-based preventive interventions.

      Goal recognition

      Goal recognition in intelligent learning environments involves identifying the specific objectives that a user is attempting to achieve, where the user's goals are often hidden from the system and must be inferred from actions taken by the user. Automated goal recognition is a classic challenge in AI, and it is a special case of a more general problem known as plan, activity, and intent recognition [
      • Sukthankar G.
      • Geib C.
      • Bui H.H.
      • et al.
      Plan, activity, and intent recognition: Theory and practice.
      ]. Automatically inferring users' goals is critical for modeling patterns in users' goal-directed behavior, which is important to informing personalized support of self-regulatory processes during learning, and by extension, adolescents' ownership over their own health.
      Machine learning-based approaches to goal recognition have shown considerable promise in intelligent learning environments and interactive narrative technologies. For example, Min et al. [
      • Min W.
      • Mott B.
      • Rowe J.
      • et al.
      Player goal recognition in open-world digital games with long short-term memory networks.
      ] formulated goal recognition in intelligent learning environments as a sequence labeling task and utilized LSTM networks as a solution approach. The LSTM goal recognition framework was evaluated with trace interaction logs from over 100 players interacting with a game-based learning environment for middle school science education. Results showed that an LSTM model that used distributed action embeddings as input significantly outperformed several deep and non-deep learning baselines with respect to predictive accuracy and convergence rate.

      Delivering personalized support in adaptive health technologies

      Machine learning, and in particular, reinforcement learning (RL) techniques, introduce the potential for user-adaptive technologies that automatically induce models of adaptive coaching and feedback from observations of adolescent interactions in a virtual environment. Reinforcement learning is a subarea of machine learning that focuses on training models for sequential decision-making under uncertainty [
      • Sutton R.S.
      • Barto A.G.
      Reinforcement learning: An introduction.
      ]. It is an algorithmic process for learning by experience, rather than demonstration, where the agent attempts alternative courses of action through a form of trial-and-error and, over time, induces a model for performing actions that maximize its accumulated reward.
      Over the past several years, RL has been the subject of growing interest in applications of human-centered AI, especially in education [
      • Wang P.
      • Rowe J.P.
      • Min W.
      • et al.
      High-fidelity simulated players for interactive narrative planning.
      ,
      • Rowe J.
      • Lester J.
      Improving student problem solving in narrative centered learning environments: A modular reinforcement learning framework.
      ,
      • Ausin M.S.
      • Azizsoltani H.
      • Barnes T.
      • Chi M.
      Leveraging deep reinforcement learning for pedagogical policy induction in an intelligent tutoring system.
      ,
      • Chi M.
      • VanLehn K.
      • Litman D.
      • Jordan P.
      Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies.
      ]. This work has emphasized probabilistic models of behavior, as opposed to explicit models of cognitive states, to drive models for supporting student learning. For example, Chi et al. [
      • Chi M.
      • VanLehn K.
      • Litman D.
      • Jordan P.
      Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies.
      ] used RL to model tutorial dialogue interactions, devising pedagogical tactics directly from student data in an intelligent tutoring system for physics education. Rowe and Lester [
      • Rowe J.
      • Lester J.
      Improving student problem solving in narrative centered learning environments: A modular reinforcement learning framework.
      ] utilized modular RL to induce policies for narrative-centered tutorial planning in an educational game for middle school microbiology education. More recently, Ausin et al. [
      • Ausin M.S.
      • Azizsoltani H.
      • Barnes T.
      • Chi M.
      Leveraging deep reinforcement learning for pedagogical policy induction in an intelligent tutoring system.
      ] investigated a combination of deep learning and RL techniques to model tutorial decision-making in an intelligent tutor for undergraduate logic proofs.
      Reinforcement learning is naturally suited to planning adaptive coaching and feedback in intelligent learning environments and adaptive health technologies: the planner is tasked with making a series of decisions about how coaching and feedback should unfold at runtime to optimize adolescent learning and performance on behavior change objectives. Wang et al. [
      • Wang P.
      • Rowe J.P.
      • Min W.
      • et al.
      High-fidelity simulated players for interactive narrative planning.
      ] devised a deep RL-based adaptive coaching framework that was based upon asynchronous advantage actor-critic (A3C), a type of RL algorithm that has proven effective in benchmark RL tasks such as Atari game playing and racecar simulations [
      • Mnih V.
      • Badia A.P.
      • Mirza M.
      • et al.
      Asynchronous methods for deep reinforcement learning.
      ]. Wang et al. trained and evaluated the adaptive coaching model using trace log data from over 400 students who used an educational game in their science classrooms. Results from a simulation analysis suggested that the deep RL framework yielded more effective learning experiences than baseline RL architectures in terms of fostering improved student learning gains.
      In addition to RL, there are several alternative machine learning–based frameworks for providing personalized coaching and support to enhance learning outcomes. Lee et al. [
      • Lee S.
      • Rowe J.
      • Mott B.
      • Lester J.
      A supervised learning framework for modeling director agent strategies in educational interactive narrative.
      ] introduced a learning-by-demonstration framework in which a human tutor “teaches” the machine learning system how to provide adaptive coaching in an educational game for middle school science. Their framework involved conducting user studies under a Wizard of Oz paradigm, where a group of students interact with the learning environment, and behind the scenes, coaching and feedback interventions are selected by a human but are delivered entirely through the intelligent learning environment's interface. Orji et al. [
      • Orji R.
      • Vassileva J.
      • Mandryk R.
      Modeling the efficacy of persuasive strategies for different gamer types in serious games for health.
      ] utilized data-driven techniques from persuasive technologies to personalize game-based interventions toward individual user personalities in a serious game for health behavior change. These methods, among others, underscore the significant promise of AI-driven adaptive coaching technologies for supporting personalized interventions in preventive health.

      Challenges

      There are several important challenges raised by AI-driven adaptive technologies for adolescent preventive healthcare. Looking beyond AI's prospective benefits, it is important to also recognize challenges and potential risks, identify strategies for addressing or managing them, and assess areas where caution may be necessary to ensure that AI's impacts are beneficial for adolescent health rather than counterproductive.
      Because machine learning–based systems rely heavily upon access to detailed data about user attributes and behavior to drive intelligent support and personalization, privacy is a major issue in health applications of AI [
      • Stahl B.
      • Wright D.
      Ethics and privacy in AI and big data: Implementing responsible research and innovation.
      ]. Increasingly, it will be essential for technology developers and researchers to develop a deep understanding of the ethical and regulatory issues related to privacy of patient health data. By accepting responsibility for how AI technologies are used, the developers of these systems will be better positioned to ensure that adaptive health technologies are designed and developed to maximize benefits toward health. This includes careful attention toward ethics in human research, such as issues of user consent, data security, user autonomy, justice, beneficence, and data ownership, which are critically important for ensuring that the benefits of AI-driven adaptive health technologies are not outweighed by their potential risks and hazards. For example, user modeling techniques, such as stealth assessment, show significant promise for informing the provision of personalized support within adaptive health technologies. However, these same techniques raise questions about what data are collected and how else the data will be used; the prospect of invisibly collecting adolescent health data, especially for the purpose of informing health interventions, raises important considerations of user consent and patient rights.
      Another important challenge is the potential introduction of implicit bias into machine learning–based models in adaptive health technologies [
      • O'Neil C.
      Weapons of math destruction: How big data increases inequality and threatens democracy.
      ]. In general, machine learning–based models reflect the data they have been trained on; if systematic biases exist in the training data, then similar biases are likely to arise in the machine learning models too. To avoid codifying and entrenching harmful bias into machine learning–based models for adolescent preventive healthcare, it is imperative to recruit diverse adolescents to participate in studies for collecting data to train machine learning systems, and where possible, participate in discussions about the design and development of health information technologies themselves. Furthermore, there are emerging computational techniques that explicitly address issues of fairness, accountability, transparency, and generalizability of machine learning–based models that merit attention for their applicability toward adaptive health technologies [
      • Pedreshi D.
      • Ruggieri S.
      • Turini F.
      Discrimination-aware data mining.
      ,
      • Ananny M.
      • Crawford K.
      Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability.
      ].
      A third challenge is integrating AI-driven adaptive health technologies into existing workflows for delivering care, and relatedly, integrating into the real-world lives of adolescents. Within clinical settings, there are a broad range of questions about effective models for integration with electronic health record systems, linking adolescents to the technology, giving care providers effective interfaces, and minimizing issues related to maintenance and troubleshooting. Outside of the clinic, it is imperative that the design and development of adaptive health technologies account for where, when, and how adolescents are likely to use adaptive health intervention technologies. Adolescents have access to an enormous range of activities, entertainment technologies, and other resources, both technology-based and technology-free, that call upon their time and attention throughout the day. Developing tools and methods that enable adaptive health technologies to fit within the everyday lives of adolescents is important for adherence and engagement. Furthermore, the specific context in which adaptive technologies are used can have a significant impact on the design and effectiveness of machine learning–based models of adolescent learning and engagement. Ultimately, developing frameworks for better understanding how AI-driven adaptive health technologies are most effective in practice will be a critical challenge for the field.
      In conclusion, recent advances in AI show significant promise for enabling adaptive technologies for personalized adolescent preventive healthcare. Increasingly, new opportunities for personalization are being introduced by dramatic improvements in machine learning. Drawing upon related work on intelligent learning environments and interactive narrative generation, we have described significant opportunities in modeling adolescent learning and engagement—these including applying techniques such as stealth assessment, affective modeling, and goal recognition—and devising adaptive coaching to support improved adolescent health outcomes. There are several important challenges raised by AI-driven health technologies, which require solutions and active management, including issues of privacy, ethics, encoded bias, and effective integration into clinical workflows and adolescent lives. However, we see enormous potential for applying AI toward the creation of adaptive technologies for personalized preventive health, creating new possibilities for personalized technology-based health screening and intervention that are both effective and engaging for adolescents.
      There are many promising future directions for research on the design, development, and investigation of adaptive health technologies for personalized preventive adolescent healthcare. The human-centered AI technologies described in this article—intelligent learning environments, interactive narrative generation, stealth assessment, affective modeling, goal recognition, and adaptive coaching—have been widely investigated in education and entertainment settings, but there has been comparatively little work examining their application within adolescent health. Investigating how human-centered AI technologies translate to health-care settings, both inside and outside of the clinic, is an important future direction for the field. This will necessitate the creation of new models and methodologies for AI research and development within preventive healthcare, as well as validation of AI techniques within health applications, such as prevention, behavior change, and disease management. To conduct this research, it will be important to develop collaborations between health researchers, computer scientists, and health-care providers to ensure that the research is grounded in health behavior change theory, clinically relevant, and leverages the complementary perspectives that are uniquely available in multidisciplinary teams. Finally, investigating AI-driven personalization within adolescent preventive health promises to surface new and distinctive problems that require novel algorithmic approaches and methodologies, pushing forward the frontiers of both health and computer science alike.

      Acknowledgments

      The authors appreciate helpful conversations on earlier versions of this manuscript with Prof. Elizabeth Ozer, Prof. Kathleen Tebb, Dr. Carlos Penilla, Dr. Alison Giovanelli, and Mr. Mark Berna.

      Funding Sources

      The material described in this article is based upon work supported by the National Science Foundation under grant IIS-1344803.

      References

      1. Bickmore T, Zhang Z, Reichert M, et al. Promotion of preconception care among adolescents and young adults by conversational agent. J Adolesc Health. In press (Suppl).

        • Hekler E.B.
        • Michie S.
        • Pavel M.
        • et al.
        Advancing models and theories for digital behavior change interventions.
        Am J Prev Med. 2016; 51: 825-832
        • Orji R.
        • Vassileva J.
        • Mandryk R.
        Modeling the efficacy of persuasive strategies for different gamer types in serious games for health.
        User Model User-Adaptive Interaction. 2014; 24: 453-498
        • Kulik J.A.
        • Fletcher J.D.
        Effectiveness of intelligent tutoring systems: A meta-analytic review.
        Rev Educ Res. 2016; 86: 42-78
        • Henderson N.
        • Rowe J.
        • Mott B.
        • et al.
        4D affect detection: Improving frustration detection in game-based learning with posture-based temporal data fusion.
        in: Proceedings of the Twentieth International Conference on artificial intelligence in education. 2019: 144-156
        • Topol E.
        Deep medicine: How artificial intelligence can make healthcare human again.
        Basic Books, New York2019
        • Pelánek R.
        The details matter: Methodological nuances in the evaluation of student models.
        User Model User-Adaptive Interaction. 2018; 28: 207-235
        • Wang P.
        • Rowe J.P.
        • Min W.
        • et al.
        High-fidelity simulated players for interactive narrative planning.
        in: Proceedings of the Twenty-Seventh International Joint Conference on artificial intelligence. 2018: 3884-3890
        • Ozer E.
        • Rowe J.
        • Tebb K.
        • et al.
        Fostering engagement in health behavior change: Iterative development of an interactive narrative environment to enhance adolescent preventive health services.
        J Adolesc Health. 2020; 67: S34-S44
        • Russell S.J.
        • Norvig P.
        Artificial intelligence: A modern approach.
        Prentice Hall, Upper Saddle River, NJ2009
        • LeCun Y.
        • Bengio Y.
        • Hinton G.
        Deep learning.
        Nature. 2015; 521: 436-444
        • Ford K.M.
        • Hayes P.J.
        • Glymour C.
        • Allen J.
        Cognitive orthoses: Toward human-centered AI.
        AI Mag. 2015; 36: 5-8
        • Baltrušaitis T.
        • Ahuja C.
        • Morency L.P.
        Multimodal machine learning: A survey and taxonomy.
        IEEE Trans Pattern Anal Mach Intell. 2018; 41: 423-443
        • VanLehn K.
        The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems.
        Educ Psychol. 2011; 46: 197-221
        • Baker R.S.
        Educational data mining: An advance for intelligent systems in education.
        IEEE Intell Syst. 2014; 29: 78-82
      2. Graesser A. Emotions are the experiential glue of learning environments in the 21st century. Learn Instr. In press, https://doi.org/10.1016/j.learninstruc.2019.05.009

        • Cavazza M.
        • Young R.
        Introduction to interactive storytelling.
        in: Nakatsu R. Rauterberg M. Ciancarini P. Handbook of Digital Games and Entertainment Technologies. Springer, Singapore2017: 377-392
        • Yu H.
        • Riedl M.
        Personalized interactive narratives via sequential recommendation of plot points.
        IEEE Trans Comput Intell AI Games. 2014; 6: 174-187
        • Lee S.
        • Rowe J.
        • Mott B.
        • Lester J.
        A supervised learning framework for modeling director agent strategies in educational interactive narrative.
        IEEE Trans Comput Intell AI Games. 2014; 6: 203-215
        • Rowe J.
        • Lester J.
        Improving student problem solving in narrative centered learning environments: A modular reinforcement learning framework.
        Proceedings of the Seventeenth International Conference on artificial intelligence in education. 2015: 419-428
        • Marsella S.
        • Johnson W.L.
        • LaBore C.
        Interactive pedagogical drama for health interventions.
        Proceedings of the Eleventh International Conference on artificial intelligence in education. 2003: 341-348
        • Yin L.
        • Ring L.
        • Bickmore T.
        Using an interactive visual novel to promote patient empowerment through engagement.
        Proceedings of the International Conference on the foundations of digital Games. 2012: 41-48
        • Kim Y.J.
        • Almond R.G.
        • Shute V.J.
        Applying evidence-centered design for the development of game-based assessments in Physics Playground.
        Int J Test. 2016; 16: 142-163
      3. Min W, Frankosky M, Mott B, et al. DeepStealth: Game-based learning stealth assessment with deep neural networks. IEEE Trans Learn Tech. In press, https://doi.org/10.1109/TLT.2019.2922356

        • D’Mello S.
        • Kappas A.
        • Gratch J.
        The affective computing approach to affect measurement.
        Emot Rev. 2018; 10: 174-183
        • Esteban-Millat I.
        • Martínez-López F.J.
        • Huertas-García R.
        • et al.
        Modelling students' flow experiences in an online learning environment.
        Comput Educ. 2014; 71: 111-123
        • D'Mello S.
        • Lehman B.
        • Pekrun R.
        • Graesser A.
        Confusion can be beneficial for learning.
        Learn Instr. 2014; 29: 153-170
        • Botelho A.
        • Baker R.
        • Ocumpaugh J.
        • Heffernan N.
        Studying affect dynamics and chronometry using sensor-free detectors.
        Proceedings of the Eleventh International Conference on educational data mining. 2018: 157-166
        • D’Mello S.
        • Kory J.
        A review and meta-analysis of multimodal affect detection systems.
        ACM Comput Surv. 2015; 47: 43-79
        • Barrett L.F.
        • Adolphs R.
        • Marsella S.
        • et al.
        Emotional expressions reconsidered: Challenges to inferring emotion from human facial movements.
        Psychol Sci Publ Interest. 2019; 20: 1-68
        • Sukthankar G.
        • Geib C.
        • Bui H.H.
        • et al.
        Plan, activity, and intent recognition: Theory and practice.
        Morgan Kaufmann, Waltham, MA2014
        • Min W.
        • Mott B.
        • Rowe J.
        • et al.
        Player goal recognition in open-world digital games with long short-term memory networks.
        Proceedings of the Twenty-Fifth International Joint Conference on artificial intelligence. 2016: 2590-2596
        • Sutton R.S.
        • Barto A.G.
        Reinforcement learning: An introduction.
        MIT Press, Cambridge, MA2018
        • Ausin M.S.
        • Azizsoltani H.
        • Barnes T.
        • Chi M.
        Leveraging deep reinforcement learning for pedagogical policy induction in an intelligent tutoring system.
        Proceedings of the Twelfth International Conference on educational data mining. 2019: 168-177
        • Chi M.
        • VanLehn K.
        • Litman D.
        • Jordan P.
        Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies.
        User Model User-Adaptive Interaction. 2011; 21: 137-180
        • Mnih V.
        • Badia A.P.
        • Mirza M.
        • et al.
        Asynchronous methods for deep reinforcement learning.
        Proceedings of the Thirty-third International Conference on machine learning. 2016: 1928-1937
        • Stahl B.
        • Wright D.
        Ethics and privacy in AI and big data: Implementing responsible research and innovation.
        IEEE Secur Priv. 2018; 16: 26-33
        • O'Neil C.
        Weapons of math destruction: How big data increases inequality and threatens democracy.
        Broadway Books, New York2016
        • Pedreshi D.
        • Ruggieri S.
        • Turini F.
        Discrimination-aware data mining.
        in: Proceedings of the Fourteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2008: 560-568
        • Ananny M.
        • Crawford K.
        Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability.
        New Media Soc. 2018; 20: 973-989