If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to: Robert G. LaChausse, Ph.D., Department of Health Science & Human Ecology, California State University, San Bernardino, 5500 University Parkway, San Bernardino, CA 92407.
To examine how teacher characteristics affected program fidelity in an impact evaluation study of the Positive Prevention PLUS program, and to propose a comprehensive teacher training and professional development structure to increase program fidelity.
Methods
Curriculum fidelity logs, lesson observations, and teacher surveys were used to measure teacher characteristics and implementation fidelity including adherence, adaptation, and lesson quality.
Results
Compared with non–health credentialed teachers, credential health education teachers had greater comfort and self-efficacy regarding sex-related instruction. Teacher self-efficacy and comfort were significant predictors of adherence.
Conclusions
Implementation fidelity may be linked to teacher characteristics that can be enhanced during curriculum training. A 2-day teacher training may not adequately address teacher facilitation skills or the maintenance of institutional supports for implementing a program with fidelity and quality. A new model of comprehensive teacher training and support is offered. This new training infrastructure is intended to contribute to the school district's institutionalization of higher-quality comprehensive sexual health education and increase program fidelity.
This article discusses the importance of both teacher and organizational-level support in promoting implementation fidelity in teen pregnancy prevention programs. A comprehensive model of teacher training and program support is offered.
School-based programs are the primary means by which prevention curricula are delivered to young people and are now required in more than 85% of the schools in the United States [
Centers for Disease Control and Prevention. State-level school health policies and practices: A state-by-state summary from the school health policies and programs study 2006. http://www.cdc.gov/healthyyouth/shi/. Accessed June 15, 2013.
]. There is a large evidence base indicating that a number of school-based prevention programs are effective in preventing or reducing substance use, unintended pregnancies, and other health risk behaviors among youth [
]. Prevention researchers have long asserted that the effectiveness of prevention programs depends on implementation fidelity: the degree to which teachers and other program providers implement programs as intended by the program developers [
]. However, many school-based prevention programs are seldom implemented perfectly, and several studies have revealed the extent to which program fidelity occurs and how various factors of implementation fidelity affect program outcomes [
]. What is less known is the type and extent of training needed by teachers to prepare them to implement school-based prevention curricula with fidelity [
]. There is a growing awareness that teachers may be freely adapting classroom curricula, either by truncating the number of lessons taught or by modifying or adding to curricular content or instructional strategies [
] found that 84% of teachers omitted at least one module in the Teenage Health Teaching Modules curriculum, and they were less likely to use critical elements of the curriculum such as role-playing and other key curricular components.
Several studies have demonstrated the relationship between teacher training and greater implementation fidelity [
]. Factors associated with implementation fidelity include in-depth training for teachers, strong support from administration, the characteristics of the curriculum itself, and the provision of ongoing technical assistance [
]. Teacher workshops are critical for success because they provide the background justification, knowledge, and skills needed to implement the program, foster support and commitment to the program, and communicate the importance of program fidelity [
]. At the top level, district officials must champion the program replication from its inception and throughout implementation. School administrators must back the program and agree to adopt the initiative; make needed resources available; garner initial staff buy-in to the purpose, values, and ideals of the program; and exert strong, continuous pressure for implementation.
To support a well-implemented program that uses valuable class time, all program teachers must believe the program is worthwhile, have a sense of ownership for it, encourage implementation by others, and feel supported by school administrators. For example, implementation fidelity of the Towards No Tobacco Use program was positively associated with teachers' beliefs about the value of the program and these beliefs were positively associated with the district and school site support for the program [
]. Furthermore, comprehensive teacher training significantly increased teachers' self-efficacy, which resulted in an increase in implementation fidelity [
]. However, some research suggests that certain types of teacher training for school-based prevention programs can affect both fidelity and program impacts. For instance, Rohrbach et al. [
] examined the effects of three types of teacher training approaches of the Towards No Drug Abuse program: comprehensive implementation support (1-day teacher training workshop, onsite coaching, Web-based support, and technical assistance), 1-day workshop training only, and a control. Higher implementation fidelity was found for the comprehensive implementation support group and resulted in greater program impacts on intention to use drugs and knowledge regarding drug abuse compared with the 1-day training group. As a result, ongoing comprehensive training and support for teachers of school-based prevention programs may help ensure continued program involvement, rekindle commitment where needed, and ensure that teachers are continuing to deliver high-quality prevention education programs [
Other factors, including teacher preparation, may also affect implementation fidelity. In most states, health teachers have completed undergraduate training in health education or health promotion as well as courses in education and pedagogy. If no health education course is offered, this instruction is usually assigned to a physical education or life science teacher with little or no professional preparation in health education. Past research suggests that credentialed health education teachers are more supportive of school-based sex education programs and more comfortable teaching sex-related topics, and deliver a greater proportion of evidence-based health education program components than non-credentialed health education teachers [
]. Hence by depending on non-credentialed health education teachers to present the teen pregnancy prevention lessons, curriculum developers face challenges in terms of the teachers' familiarity, comfort, and skills in sex-related instruction. The purposes of this article were to examine how teacher characteristics affected program fidelity in an impact evaluation study of the Positive Prevention PLUS program and to propose a comprehensive teacher training and support model to increase program fidelity and reach program outcomes.
Background
The Positive Prevention PLUS Sexual Health Education curriculum [
] is a comprehensive teen pregnancy and sexually transmitted disease (STD)/human immunodeficiency virus (HIV) prevention program aimed at students in grades 9–12. The 11-lesson curriculum was developed after a review of the literature of effective teen pregnancy prevention programs, and uses social learning and cognitive behavior theories to increase a student's ability to use refusal skills, use condoms, and resist peer pressure. The curriculum is delivered in the classroom by classroom teachers. The curriculum includes lectures, discussions, demonstrations, and role-plays regarding healthy relationships, reasons to remain sexually abstinent, contraceptive methods, a condom demonstration, steps in condom use, negotiating condom use with a partner, refusal skills, reproductive health services, and future planning.
The current data come from a clustered randomized control trial funded by the United States Department of Health and Human Services, Office of Adolescent Health involving students in grade 9 and their teachers in 22 public high schools in five school districts in Southern California. Of these participating school districts, two have no required health education course, which resulted in life science teachers being assigned to implement the curriculum. As with other school districts adopting our curriculum in prior years, a 2-day teacher training was conducted for program teachers. This training included warm-up activities, a review of the state education code and history of curriculum development, an overview of the research on effective sexual health education, the logic model informing the curriculum and its learning activities, instruction on how to answer difficult questions, and actual practice implementing the lesson activities. Research protocols were also reviewed, including the importance of implementation fidelity.
After parental consent was obtained, school sites were randomly assigned to either a treatment group or control group. The treatment group included 11 school sites that either offered a health course (n = 7) or a ninth-grade life science course (n = 4); the control group included 11 school sites that had either a health course (n = 4) or a ninth-grade life science course (n = 7). The treatment group sites agreed to implement the Positive Prevention Curriculum and withhold any other school-based teen pregnancy prevention or STD/HIV prevention education. Schools randomly assigned to the control group were asked not to provide any teen pregnancy prevention/STD/HIV instruction during the study period, as stipulated in the memorandum of understanding with each school district.
Students completed a self-administered survey as a group during their regular class period, which included measures of sexual behavior (e.g., condom use in the past 30 days) and psychosocial factors (e.g., self-efficacy to use condoms) 2 weeks before the beginning of the program, 30 days after the last day of program implementation, and 6 months after the end of the program. The curriculum was delivered in the classroom over a 3-week period in the fall of 2012. Measures of implementation fidelity (adherence, adaptations, etc.) included a teacher curriculum fidelity log and video observations of teachers. The curriculum fidelity log is a teacher self-report of fidelity including adherence to curriculum components and lesson activities, adaptation to lessons, lesson completion, and participant responsiveness. The curriculum log is organized by lesson and asks program teachers to report whether each portion of the lesson was taught completely or with changes, or not taught at all. Teachers were asked to complete the fidelity log at the end of each lesson for each class period. In addition, each program teacher had one lesson videotaped by the Principal Investigator (PI). The lesson date and class period were randomly selected, and program teachers had no prior knowledge as to the date or time when the observation would occur. These lesson videos were rated to assess the overall quality of the program session and delivery of lesson activities, including the teacher's knowledge of the lesson, poise and confidence, and level of enthusiasm, and student responsiveness. Each area is scored from 1 = “poor” to 5 = “excellent.” A scale score was obtained by summing across the items so that a higher score represents greater lesson quality. All data were collected by the PI.
Before a 2-day training on the program, each intervention group teacher completed a brief survey that included measures pertaining to teacher self-efficacy, teacher comfort, and teacher characteristics (i.e., type of teaching credential held, prior training in comprehensive sex education). Teacher comfort was measured using a 15-item scale developed by the PI. This scale asks teachers to respond to various topics and pedagogical strategies in comprehensive sex education and teen pregnancy prevention, such as “how to use a condom” and “leading a role-play on how to get someone to use condoms.” The scale employs a 5-point response scale ranging from 1 = “not at all comfortable” to 5 = “completely comfortable.” This scale demonstrated satisfactory reliability (Cronbach α = .88). Teacher self-efficacy was measured using a modified version of the Health Teacher Self-Efficacy Scale [
]. This scale asks teachers to respond to 15 self-referential statements such as “I believe that if I do a good job teaching, the students I teach will be able to get out of a situation that can lead to unprotected sex.” The response format is a 5-point Likert-type scale ranging from 1 = “strongly disagree” to 5 = “strongly agree.” This scale demonstrated good reliability (Cronbach α = .92). The Institutional Review Board at California State University, San Bernardino, reviewed and approved this study.
Results
The average number of lesson components completed by each intervention site ranged from 59% to 94%. Overall, the average number of lesson components completed by teachers was 82%. Health teachers had higher levels of comfort with teaching comprehensive sex education topics (F[1,35] = 4.58; p = .03, eta = .11) and had greater self-efficacy to teach comprehensive sex education than non-health teachers (F[1,33] = 11.09; p = .001; eta = .50).
Table 1 shows the type and percentage of adaptations that program teachers made to four key lesson activities: refusals skills role-play, steps in condom use, condom demonstration, and condom use negotiation role-play. For example, over 25% of program teachers did not conduct the refusal skills practice, 23% did not show the steps in condom use, 30% of teachers did not demonstrate how to use condoms, and 25% did not conduct a role-play of negotiating condom use. In addition, 16.6% of the teachers made significant adaptations to the lesson on steps in condom use and condom use negotiation practice. Credentialed health education teachers were more likely than non-credentialed health teachers to conduct the lesson activity on steps in condoms use (F[1,29] = 4.92; p = .03; eta = .38) and were more likely to conduct the role-play on condom use negotiation (F[1,29] = 4.41; p = .04; eta = .36). No significant differences were found for lesson quality, average number of adaptations made, or total number of lesson activities taught by the two groups of teachers.
Table 1Type and percent of adaptations of key lesson activities
Normative professional development experiences related to cultural, socioeconomic, and gender-related (including lesbian, gay, bisexual, transgender, queer or questioning, and intersex) issues
Skill-based practice of learning activities including introducing, directing, facilitating, and debriefing the activity; directing and facilitating discussions; provision of ongoing technical assistance
A series of separate logistic regression analyses were conducted predicting whether the four key lesson activities (condom demonstration, steps in condom use, condom use negotiation role-play, and refusal skills role-play) were taught using teacher self-efficacy and teacher comfort as predictors. Both teacher self-efficacy (β = .25; Wald = 5.75; p = .02; odds ratio [OR], 1.28) and teacher comfort (β = .11; Wald = 6.86; p = .01; OR, 1.12) were significant predictors of whether program teachers taught the condom use negotiation role-play. Similarly, teacher self-efficacy (β = .15; Wald = 4.52; p = .03; OR, 1.16) and teacher comfort (β = .06; Wald = 3.57; p = .05; OR, 1.06) were significant predictors of whether program teachers demonstrated how to use a condom. On the other hand, teacher self-efficacy (β = .09; Wald = 2.98; p = .08; OR, 1.08) and teacher comfort (β = .04; Wald = 2.16; p = .14; OR, 1.04) were not significant predictors of whether program teachers conducted the role-play activity on refusal skills. Finally, teacher comfort was a significant predictor of lesson quality (β = .25; t = 2.51; p = .02) but teacher self-efficacy was not a significant predictor of lesson quality (β = .10; t = .41; p = .65).
Discussion
Low fidelity to the curriculum jeopardized our project's ability to demonstrate the efficacy of the curriculum in reducing a number of adolescent sexual risk behaviors and related predispositions. Although it is widely recognized that implementation fidelity matters, little is known about the most effective approaches for improving fidelity among teachers [
]. Data from this current study indicate that teacher characteristics affected program adherence, adaptions, and lesson quality. At the same time, recent research points to the need for a comprehensive teacher training and support model to increase implementation fidelity [
]. Based on these findings and more recent information regarding enhancing program fidelity, program staff proposed the following changes that will be implemented in the 2013–2014 project year.
Based on our current implementation fidelity and teacher data, it is evident that a one-shot, 2-day model of teacher training (Figure 1) fell short of the staff development needed by our classroom teachers to implement this teen pregnancy prevention curriculum with fidelity. The literature on teacher training suggests that teachers can learn about the program theory, key content, and mechanics of delivery during face-to-face workshops, but mastery of the skills and comfort required for teaching is achieved only when teachers have opportunities for practice, followed by feedback on the practice [
Implementation research: a synthesis of the literature.
University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network. (FMHI Publication 231),
Tampa (FL)2005
]. Because teacher comfort was positively related to lesson completion, we believe that teacher self-efficacy, comfort, and buy-in can be increased with a sequence of training opportunities. We propose a sequence of teacher training opportunities, beginning with an initial 2-day training session, the addition of an online teacher training component, a follow-up skill-building session, and technical assistance to improve teacher comfort and self-efficacy and result in improved implementation fidelity (Figure 1). We believe that this is critical because many of the teachers involved in this project lack a background and training in school health education, and specifically teen pregnancy prevention.
Figure 1One shot 2-day curriculum training versus enhanced comprehensive teacher training sequence.
Because most of our teachers experienced this curriculum training previously, more advanced and focused trainings are planned to include: (1) pre-assessment/surveying of teachers attending the training to determine levels of comfort and experience; (2) the differentiation between those teachers who are more or less experienced to establish a site-based “buddy system”; (3) less emphasis on the history of the project and curriculum development; and (4) a discussion on the purpose of the curriculum, the importance of implementation fidelity, and how lesson adaptations or deletions can affect key program outcomes. Each of these modifications customizes the training experience in support of greater teacher engagement, comfort, and self-efficacy.
An online teacher training component
The Positive Prevention Online Teacher Training Program is a series of self-directed online modules that teachers can review before each lesson's implementation. Each module is based on the teacher's lesson materials and provides a brief 10-minute reminder of the lesson content and objectives, the sequence of instruction, and guidance for facilitating the lesson's learning activity. Brief video clips of classroom teachers also model the activities. In addition, the teachers presented in the video clips discuss their initial discomfort with teaching sex-related instruction and reinforce the idea that implementation fidelity is a key feature in reaching program outcomes. This online training is a required follow-up to the 2-day training, to be completed by each teacher before returning for a 1-day skills practice before actual curriculum implementation. Hence, the online training serves as both a supplemental training platform and a resource for teachers before they implement each lesson.
Follow-up skill-building session
Because of the lag time that sometimes occurs between initial curriculum trainings and curriculum implementation, we are also piloting a 1-day skills-focused training. This is a fast-paced, fun, and interactive training in which key lesson activities (i.e., condom use) are repeatedly presented, practiced, and perfected with feedback from the program developers. The purpose of this follow-up skill-building session is to enhance teacher comfort and self-efficacy in support of greater implementation fidelity.
Onsite technical assistance during curriculum implementation
One of the key factors associated with implementation fidelity is the provision of technical assistance [
]. Such assistance will take the form of lesson observations, collaborative coaching, ensuring that all lesson materials are prepared and ready for implementation, ensuring lesson completion, examining possible teacher discomfort, helping with lesson pacing and classroom management strategies, and creating a social norm in which implementation fidelity is crucial to reaching program outcomes. Each of our experienced trainers will be assigned to a small group of teachers implementing the curriculum within the same school district, to perform site visits, convene site-based discussions among teachers, and provide constant communication through telephone and e-mail.
Logic model for teachers' target behaviors
Our Office of Adolescent Health funding requires that we make explicit a logic model upon which our Positive Prevention PLUS program is based. A logic model identifies the determinants and mediating variables that contribute to student outcome behaviors. Determinants and mediating variables can also lead to teacher behaviors in support of curriculum fidelity and student engagement. However, development of a logic model for teacher training is seldom required of professional development activities. A behavior-determinants intervention logic model, based on the work of Kirby [
] is proposed in Table 2. An assumption made explicit by creating this logic model is that there are six determinants of the teacher outcome behavior (lesson fidelity). Of those, three contribute to perceived social norms (community–school district support, administrative support, and peer support for Comprehensive Sexual Health Education [CSHE]), one contributes to perceived benefits of the target behavior (positive attitudes toward the curriculum and its intended impact), and two contribute to teacher comfort and self-efficacy (lesson management and activities facilitation). In addition, the logic model assumes that student engagement is a result of respect for student diversity and also the skillful facilitation of lesson activities and discussions.
In contrast to one-shot teacher trainings, which do not adequately address teacher facilitation skills or the maintenance of institutional supports for ongoing comprehensive sexual health education, we propose an enhanced, comprehensive teacher training sequence aligned with a professional development logic model and consistent with newer literature on implementation fidelity and professional development. The goal of the new teacher training sequence is to improve teachers’ lesson presentation skills, comfort, and self-efficacy, to increase implementation fidelity and result in stronger program outcomes.
Acknowledgments
The authors acknowledge the contributions of the following persons to the content of this article: Deborah Christopher, Boulder Learning; Jeff Gould, ETR Associates; and Jody Oliver, San Bernardino County Superintendent of Schools Office.
Funding Source
This publication was made possible by Grant TP2AH000007 from the Office of Adolescent Health.
References
Centers for Disease Control and Prevention. State-level school health policies and practices: A state-by-state summary from the school health policies and programs study 2006. http://www.cdc.gov/healthyyouth/shi/. Accessed June 15, 2013.
Implementation research: a synthesis of the literature.
University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network. (FMHI Publication 231),
Tampa (FL)2005
Conflicts of Interest: Dr. LaChausse and Sabrina Chapple have no conflict of interest to disclose. Dr. Clark is the co-author of the Positive Prevention program.
Disclaimer: Publication of this article was supported by the Office of Adolescent Health, U.S. Department of Health and Human Services. The opinions or views expressed in this paper are those of the authors and do not necessarily represent the official position of the Office of Adolescent Health, U.S. Department of Health and Human Services.
00857-4&id=gr1.jpg){kind=link}