Advertisement

Planning for a Group-Randomized Trial With American Indian Youth

      Abstract

      Purpose

      To describe lessons learned working with tribal communities in the Northern Plains to plan and implement a group randomized trial of multimedia Circle of Life (mCOL), a sexual risk reduction program designed for American Indian (AI) youth.

      Methods

      Project records including emails, travel reports, and meeting minutes were reviewed and synthesized to describe participatory development of the project.

      Results

      Several challenges were identified including: discussing sexual health interventions for preteens with communities; developing a culturally appropriate research design; managing costs of conducting research in remote and culturally distinct tribal communities; and building research infrastructure of partner organizations. Opportunities for strengthening research partnerships included transparency, openness to bi-directional learning, planning for change, flexibility, and strategic use of technology.

      Conclusions

      Findings suggest that meaningful AI community participation in research trials is achievable and a critical step towards generating evidence for interventions in settings where they are most needed. Substantial investments in time, resources, and relationship-building are necessary.

      Keywords

      Implications and Contribution
      Little is known about developing randomized trial research designs with tribal communities. This paper describes our experiences navigating between the worlds of research and tribally based organizations serving American Indian youth to assess a culturally appropriate sexual risk reduction program.
      American Indian and Alaska Native (AIAN) youth 15–19 years of age experience high birth rates. Nationally in 2010, AIANs had the third highest teen birth rates (38.7 births per 1,000), after African-Americans (51.5 births) and Hispanics (55.7 births) [

      National Center for Health Statistics. Births: Final data for 2010. Available at: http://www.cdc.gov/nchs/data/nvsr/nvsr61/nvsr61_01.pdf. Accessed June 12, 2013.

      ]. However, national statistics mask especially high teen birth rates for AIANs in certain areas of the country, such as North Dakota and South Dakota, the location of the project. In these states, AIAN teen levels are 96.4 and 99.3 per 1,000, respectively, rates almost five times higher than their White counterparts of the same state [

      National Center for Health Statistics. Births: Final data for 2010. Available at: http://www.cdc.gov/nchs/data/nvsr/nvsr61/nvsr61_01.pdf. Accessed June 12, 2013.

      ]. In contrast, birth rates for younger adolescents,10–14 years of age, are comparable across race groups and substantially lower than those of 15–19 years of age. Clearly, effective prevention messages for AIAN youth should be aimed at this younger group. Yet, to date, no evidence-based intervention exists for this population.
      Circle of Life (COL) is a classroom-based curriculum developed for AIAN youth to address sexual risk [

      Orbis Associates. Circle of life HIV/AIDS and STD prevention curriculum: Wellness education for American Indian and Alaska Native middle school students. Washington, DC, 2002.

      ,
      • Kaufman C.E.
      • Litchfield A.
      • Schupman E.
      • et al.
      Circle of life HIV/AIDS-prevention intervention for American Indian and Alaska Native youth.
      ]. The original curriculum was an age-appropriate comprehensive HIV prevention program, developed with extensive community review and input from AIAN parents, educators, and health experts across the country. In a school-based group-randomized trial (GRT) with AI seventh and eighth graders of a Northern Plains community, COL was shown to be effective in delaying sexual initiation among the youngest of this sample—those who received COL at 13 years of age or younger [

      Kaufman CE, Whitesell NR, Keane EM, et al. Effectiveness of circle of life, an HIV-preventive intervention for American Indian middle-school youth: A group-randomized trial in a Northern Plains tribe. Am J Public Health. In press.

      ].
      In 2010, we received funding from the Office of Adolescent Health as a part of the Teen Pregnancy Prevention Program innovative strategies grants to adapt the COL curriculum to an online format appropriate for 10- to 12-year-olds in after-school settings and to test the modified curriculum's effectiveness in a GRT. We partnered with the Office of Minority Health Resource Center and the Indian Health Service to revise COL in a multimedia format (mCOL). The adapted curriculum comprises seven online chapters with supplemental group activities. The new version also contains more information on teen pregnancy prevention and Hepatitis B and C, a growing health concern in AIAN communities. We employed community-based participatory research (CBPR) approaches in this research effort, partnering with Native Boys and Girls Clubs (NBGCs) in tribal communities of the Northern Plains. CBPR is recognized as key to success in research with minorities [
      • Wallerstein N.
      • Duran B.
      Community-based participatory research contributions to intervention research: The intersection of science and practice to improve health equity.
      ,
      • Goins R.T.
      • Garroutte E.M.
      • Fox S.L.
      • et al.
      Theory and practice in participatory research: Lessons from the Native Elder Care Study.
      ]. However, little is known about the interplay of partnership and the implementation of randomized trials in AIAN communities [
      • Walkup J.T.
      • Barlow A.
      • Mullany B.C.
      • et al.
      Randomized controlled trial of a paraprofessional-delivered in-home intervention for young reservation-based American Indian mothers.
      ]. In randomized trials, researchers strive for a controlled environment; in many AIAN settings, scarce resources, traumatic events, weather, long distances, and distrust of outsiders may combine in various ways to disrupt the research process. Yet, evaluating interventions in challenging settings is vital; youth in such settings are often those most in need of effective programs. In any setting, a successful trial requires planning. The literature is replete with guidance for conventional clinical trials. However, the paucity of community-based trials in tribal settings has resulted in a virtual absence of analogous guidance. Here, focusing on preparations for a randomized trial, we share our experiences in navigating between the worlds of research and tribally based organizations serving AIAN youth to generate evidence of effectiveness for a culturally appropriate sexual risk reduction program.

      The Setting

      Our partners, the NBGCs, were located in highly disbursed tribal communities spanning an area of more than 6,250 square miles—the combined tribal populations totaled about 100,000 or approximately 16 persons per square mile. To serve youth across such wide expanses, several NBGCs had multiple sites, called “units.” In total, the project worked with 16 different units located in distinct tribal communities. These communities were located in some of the poorest counties of the nation, where educational and employment opportunities lagged substantially behind the nation's average [

      U.S. Census Bureau. United States Census 2010. 2012; http://www.census.gov. Accessed October 15, 2012.

      ]. They have withstood a long history of damaging federal policies resulting in the loss of land and forced relocation, educational policies emphasizing deculturation, and a prohibition on spiritual expression that extended into the 1970s [
      • Thornton R.
      American Indian holocaust and survival: A population history since 1492.
      ,
      • Deloria V.
      • Wildcat D.R.
      Power and place: Indian education in America.
      ,
      • Harrod H.L.
      Becoming and remaining a people: Native American religions on the northern plains.
      ]. Despite the difficult history, these tribes took deep pride in their legacy of activism and resilience, marked by a resurgence in traditional teachings, practices, and language [

      Nagel J. American Indian ethnic renewal: Red power and the resurgence of identity and culture. New York City, NY: Oxford University Press. USA; 1997.

      ]. (Due to an agreement with our tribal partners, we do not name specific communities.)
      NBGCs in these small, rural communities faced unique challenges. They operated as not-for-profit organizations, legally independent of tribal government. However, the relationship of each NBGC with its respective tribe varied considerably. Some tribes were highly supportive (financially and organizationally), whereas others were uninvolved and separate. To remain viable, each NBGC had to obtain its own funding, but securing stable financial support was often challenging. In contrast to urban Boys and Girls Clubs, which often receive corporate sponsorships or community funding, these rural NBGCs had few, if any, profitable local businesses for sponsorship. Consequently, without tribal support, the NBCGs struggled for funding.

      Community Views on Rigorous Research With Young AIAN Adolescents

      The design for the project was a longitudinal GRT with young adolescents to evaluate a sexual risk reduction intervention. Many components of the project were potentially problematic for the communities. First, research with AIAN communities has had a turbulent history punctuated by violations in confidentiality, research protocol, and ethics [
      • Norton I.M.
      • Manson S.M.
      Research in American Indian and Alaska Native communities: Navigating the cultural universe of values and process.
      ,
      • Kaufman C.E.
      • Ramarao S.
      Community confidentiality, consent, and the individual research process: Implications for demographic research.
      ]. Second, the concept of a control group has been objectionable to many underserved communities, since needs are so great. Excluding half of the participants from services has been simply unacceptable. Finally, while some in the communities were familiar with evaluation, few had experience with randomized trials. The requirements of a highly controlled setting were often incompatible with the practicalities of everyday life in NBGCs of reservation communities.
      Acknowledging the sensitive nature of this project, we worked to provide transparency and flexibility in anticipation of community concerns. Several examples illustrate this approach. (1) We had three community-based tribal members on our project team, all with extensive experience conducting research in AIAN Northern Plains communities. These team members were vital to initiating and sustaining community relationships. (2) We established a community advisory board (CAB) of representatives from each community. We held open discussions with the NBGCs, CAB, and respective tribes to present the project design including the risks and benefits afforded to participants and communities by collaboration. As those relationships strengthened, these entities in turn fostered support among community families. (3) Our design was originally a wait-listed design—so all units would receive the intervention but at staggered intervals. However, the CAB preferred a design with an alternative treatment. We thus modified our design to include a science intervention for the comparison group. This program, called After-School Science Plus [

      Colon L, Stimmer M, Froschl M, Sprung B. After-school science PLUS activity guide. 2012; Education Equity Center at FHI360, http://www.edequity.org/programs/science-and-math-programs#23.

      ], had no on-line section, but was age appropriate, included approximately the same number of chapters and time commitment as COL, involved simple hands-on activities, and had no content overlap with COL. (4) Random assignment was also a concept not readily accepted. Parents (and NBGC staff) thought it odd that programs would be assigned based not on need or preference, but randomly, and by someone other than a community member. However, they allowed for this requirement of the project since youth in both arms would receive beneficial programs.
      Teen pregnancy prevention, one goal of the project, was also a concern to some community members, especially when outsiders proclaim the value of delaying childbirth. AIAN communities have experienced traumatic historical events, including a history of genocide, assimilation, and forced sterilization, the effects of which endure [
      • Thornton R.
      American Indian holocaust and survival: A population history since 1492.
      ,
      • Lawrence J.
      The Indian health service and the sterilization of Native American women.
      ,
      • Yellow Horse Brave
      • Heart M.
      • Chase J.
      • Elkins J.
      • Altschul D.B.
      Historical trauma among indigenous peoples of the Americas: Concepts, research, and clinical considerations.
      ]. Community members, parents, and grandparents with whom we talked were by no means united on the appropriate age for the birth of a first child. They were also not united in approaches to sexual education—who should teach it, when, or how. Whatever be their position, many we spoke with were moved when they saw Figure 1, which presents the gap between the birth rates of 15- to 19-year-olds and 10- to 14-year-olds as a prevention opportunity.
      Figure thumbnail gr1
      Figure 1Teen birth rates, American Indian/Alaska Native girls 10- to 14-year-olds and 15- to 19-year-olds (per 1,000)
      [
      • Menacker F.
      • Martin J.A.
      • MacDorman M.F.
      • Ventura S.J.
      Births to 10–14 year-old mothers, 1990–2002: Trends and health outcomes. National Vital Statistics Reports; vol 53 no 7.
      ]
      .
      Supporting prevention messages at young ages, however, is distinct from providing an after-school program targeting sexual risk taking. Although mCOL provides facilitators latitude in curriculum content, some chapters include sensitive information about sex, condom use, and contraception. Similarly, some sensitive questions were required for data collection (e.g., “Have you ever had sexual intercourse?”). We made sure that parents could review mCOL content and that they knew questions about sexual activity would be asked of their children. We found that transparency fostered open dialogue and allowed parents to make informed decisions.

      Partnership to Plan for Design and Implementation

      Collaboration in evaluation design and redesign

      An integral component of program evaluation with tribal communities is adaptation and modification. Grant applications that adhere to CBPR principles will already reflect community preferences. However, adjustment of program implementation and evaluation are inevitable. The realities of short deadlines of funding announcements often compromise ideal collaboration with partners in project preparation. In this project, we crafted a memo of understanding (MOU) with each NBGC. MOUs with collaborating partners were a requirement of the funding agency; we used the MOUs to demonstrate flexibility and openness with the NBGCs. Specifically, we included language making it explicit that if funded, revisions to the MOU could be made. Similarly, no NBGC was obligated to enter into a final agreement with the University project team (UPT) if funded. We outlined the general plans for project design, roles and responsibilities of each party, and project-related financial support to the NBGCs. Once the project was funded, we worked with partners to revise the MOUs appropriately. This process reaffirmed our commitment to the NBGCs and the youth they serve and enabled our partners to learn more about the research process.
      We also maximized the funder's opportunity for an extended planning period to strengthen partner relationships and elicit input into the project. A 1-year planning period was a requirement of the project. We used this time in several key ways. First, we developed a resource profile for each NBGC, including personnel and their skills; computer equipment and Internet connectivity; the number of youth served in our age group; Club program schedules; and other concerns that might affect project implementation. While some of this occurred by phone, much was done in person at site visits, again, providing opportunities for relationship building and implementation planning. Second, NBGC staff assisted in developing questions about cultural participation for the survey. Through this experience, we learned a great deal about local beliefs and practices—and the NBGC staff learned about the challenge of developing simple questions to measure complex concepts. Third, we were able to build myriad opportunities for revision and input from partners. For example, we held monthly calls over the life of the project with NBGC staff to further the sense of community among our partners, as well as to provide a means to update, receive feedback, and talk through concerns. We also held several in-person workshops with NBGC directors and the CAB to generate ideas on recruitment, project implementation, and community engagement.

      Building research infrastructure and capacity

      We spent much of our planning period building local evaluation infrastructure through partnership in three key areas: (1) personnel capacity building; (2) designing and implementing technological systems that were secure, accessible across long distances, and user-friendly; and (3) developing administrative processes in line with University and federal fiscal and research overview regulations. We initially defined roles and responsibilities carefully. The NBGC staff were experts in program implementation with youth, but most had little or no experience with research. The UPT were experienced researchers but new to many of the communities represented in this project.
      In our early conversations with the NBGC directors, many saw this as an opportunity to learn more about evaluation and the research process. To build research capacity with NBGC staff, we developed a series of trainings. Initially, we planned to provide the trainings using Web conferencing, but our partners strongly advocated for in-person training. We consequently scheduled trainings for each club, dividing the trainings into three sessions: (1) project overview, research methods, and recruitment and consent; (2) database entry and survey administration; and (3) mCOL program implementation/comparison program orientation. Through these trainings and workshops, we planned for substantial involvement of NBGC staff across all phases of the project.
      The UPT in turn developed the research systems, including the design of the tracking database, online audio-assisted computer surveys that upheld youth confidentiality and data security requirements, obtaining informed parent/guardian consent, and fiscal payment and monitoring systems.
      The defined roles and responsibilities resulted in many successful joint efforts, for example, recruitment. Since local staff knew the members of the clubs, they could identify families and invite parents and guardians to meetings about the project. The UPT then described the project components, provided project materials for parent review, answered questions about the program or the research process, and explained the consent form. This separation of roles had the added benefit of buffering any perception that NBGC staff expected participation of member families. Additionally, the NBGC staff spent many hours orienting the UPT to NBGC unit daily routines, community issues, and expectations of member families for their children.

      Planning for distance

      The large distances involved in this project required a research process that could be accommodated remotely. That is, we planned ways to problem solve or trouble-shoot long distance. For example, we used Web-based conferencing to facilitate collaboration across distances. After we trained club staff on project materials, we used conferencing to identify and fix problems with computers at the clubs and for refresher training. Since the project was heavily dependent on computers, and the computers at the NBGCs tended to have recurring problems, a specialist in our office could remotely “take over” a computer to diagnose and fix problems, including the interface of diverse computer platforms, very new and very old versions of browsers, and limited bandwidth. To support staff, we created short YouTube training videos to review before teaching each class. Finally, throughout the project, we established regular communication with club staff to monitor activities and assist with any problems. We paired UPT personnel with specific NBGC staff to provide continuity and facilitate relationship building.

      Fiscal transparency

      Resources are always a concern of financially strapped NBGCs. Knowing this, we made clear in initial conversations before the application submission that all project expenses would be covered. However, early in the project, NBGC directors asked that we give an accounting of all financial resources allocated to the project and how their clubs and communities would benefit. Our budget included substantial resources targeted for NBGCs—including all costs of the project, training opportunities provided by the funder, computer equipment, and an overhead rate on direct costs. Nevertheless, we had difficult conversations about the much higher indirect rate received by the University and the large University personnel budget. As difficult as the conversations were, our willingness to be transparent about resources helped to build trust and openness.
      Although the project resources were welcomed by the NBGCs, many struggled with invoicing and documentation required by the University. The procedure for reimbursement presented challenges for some clubs because they had to pay project costs upfront and seek reimbursement later. To assist directors, we created worksheets they could use to determine staff time spent on the project and other expenses. The spreadsheets automatically calculated an indirect rate to cover overhead expenses. We also monitored all invoices as they moved through the University system to ensure that payments were made in a timely fashion.

      Beyond a Controlled Design

      Together with our partners, we planned for a project that was poised for success. We had an intervention developed specifically for young AIAN adolescents, engaged and committed community partners, an implementation plan designed specifically for remote rural areas, intensive in-person training with a variety of technology-based tools for refresher modules, computer trouble-shooting, and fiscal processing systems. Still, our ongoing efforts on this project are a reminder that in community-based work, conditions cannot always be fully controlled.

      Local impact on research of national economy

      The economic conditions surrounding the NBGCs were perhaps the single most influential factor affecting our research. The fiscal health of NBGCs was directly related to club stability and youth participation. For many clubs, the recession of 2008–2011 hit hard. Usual funding diminished substantially, and paying bills was challenging. Several NBGCs were forced to close for periods due to an inability to pay for utilities, to repair broken heaters or sewers, or to make payroll. When clubs were not consistently open, youth and their families found alternative after-school options and did not always return. When youth could not attend club regularly, enrollment into the project and subsequent programming was challenging. In contrast, one club experienced declining membership due to increased wealth and social changes associated with oil and mining, including increased crime and availability of illegal substances [

      Gibson J. Crime accompanies flow of oil workers into North Dakota. 2012; http://www.earthisland.org/journal/index.php/elist/eListRead/crime_accompanies_flow_of_oil_workers_into_north_dakota/. Accessed August 23, 2013.

      ,

      Kusnetz N. North Dakota's oil boom brings damage along with prosperity. 2012; http://www.propublica.org/article/the-other-fracking-north-dakotas-oil-boom-brings-damage-along-with-prosperity. Accessed August 23, 2013.

      ]. Many youth from homes with new-found wealth no longer came to the club, even though wealth did not necessarily bring improved well-being to children. With shrinking membership across participating NBGCs, by about two thirds in our age group, the potential number of study participants also declined. To attempt to counteract the lower numbers, we proactively assisted NBGCs with fundraisers and membership drives at community events. We also worked closely with club staff to design incentive programs to attract youth.
      The difficult fiscal environment also affected NBGC staff. Some clubs were not able to pay staff consistently. Turnover was a challenge. Moreover, although project resources were provided to cover staff time related to project activities, NBGCs were often understaffed to cover just club-specific tasks, let alone additional ones required for the project. NBGC hiring options varied, but all had difficulties filling positions. With staff stretched so thinly, research project activities were not always prioritized. To ameliorate the issue, we supported staff in many project activities and brainstormed with NBGC supervisors about facilitating time for staff to carry out project activities.

      Lived trauma

      The communities we worked with were traumatized communities. The history of oppression and deculturation has been borne out in the day-to-day experience of each generation [
      • Yellow Horse Brave
      • Heart M.
      • Chase J.
      • Elkins J.
      • Altschul D.B.
      Historical trauma among indigenous peoples of the Americas: Concepts, research, and clinical considerations.
      ]. For example, compared with other race and ethnic groups, AIANs experience the highest rates of violent crimes and youth suicides, while interfamily and intrafamily violences are pervasive [
      • US Department of Justice
      Criminal victimization, 2010.
      ,
      • Centers for Disease Control and Prevention
      CDC Health Disparities and Inequalities Report—United States, 2011.
      ,
      • Bachman R.
      • Zaykowski H.
      • Kallmyer R.
      • et al.
      Violence against American Indian and Alaska Native women and the criminal justice response: What is known. report funded by and submitted to the U.S.
      ]. The daunting statistics, however, cannot capture the sharp pain of traumatic events, which quickly place research in perspective. In these small, close-knit communities, trauma is a shared experience, and the NBGCs often closed unexpectedly in response to a death or community tragedy; several had to deal with violent deaths of club members. When the deceased was related to the club, the turmoil and emotional toll was acute. Often, we waited for cues from our partners about when and how to proceed when trauma struck.

      Tribal–University context

      In addition to the effects of economics and trauma on implementation capacity, we also confronted various institutional and administrative hurdles. First, we faced challenges obtaining Institutional Review Board (IRB) approval. The project, designed as a GRT, did not conform easily to the medical model of research most familiar to IRBs: ours was community based, not clinically based; it took place in other states and in sovereign nations, each with their own set of rules and regulations governing research; it dealt with sensitive subject matter; and participants were young. Our IRB was willing to work with us, but it took time. Second, we also had to obtain tribal approval from multiple tribes, each with its own approval entity (e.g., Tribal Council, Tribal Committee, Tribal Research Review Board, and Tribal College IRB) and unique process. Club directors were invaluable in helping to navigate this process, although few had previous experience obtaining tribal approvals. Likely as a result of their assistance and local advocacy on our behalf, we found approving entities supportive. However, this process also took time—up to 6 months—due to tribal agenda priorities and other delays. Third, administrative obstacles occurred because of University research policies—again, many of these assuming a local clinical research site. For example, rigid financial oversight policies resulted in a process whereby only UPT could distribute compensation to respondents. This significantly limited data collection opportunities, substantially increased staff travel, and was perceived by our tribal partners as a demonstration of a lack of trust. In sum, many of the challenges of implementing the project were located within the assumptions of our institution about where and how research projects should be conducted.

      Discussion and Conclusions

      The planning strategies we put in place early in the project provided a strong foundation for the challenges of implementation of a rigorous design. These strategies helped to bridge the often divergent worlds of University research and NBGC youth programming. Clearly, building partnership and infrastructure—and having adequate time to do so—was critical for implementing a multisite project in these settings, with a strong emphasis on transparency and flexibility. Even with this strong base, a series of financial, community-related, and administrative challenges still exerted considerable sway on the implementation parameters of our project. Although technology increasingly has assisted in facilitating research in remote areas, it cannot replace the value of meeting with partners in person. To date, for example, our UPT has logged over 52,000 miles to reach out to partners and communities. At the time of this writing, the project is still collecting data and implementing the interventions. The ultimate test of our preparations will materialize as the study unfolds.
      The many obstacles described here likely seem immense—and may give pause to those interested in rigorous evaluation projects with AIAN communities. However, dismissing NBGCs or the communities in which they operate as ill-suited for research partnerships precludes the opportunity to generate evidence in settings where interventions are most needed. It also precludes the opportunity for the research community to learn from AIAN communities about how rigorous evaluation might best be conducted. A debate about the feasibility of community-based GRTs continues, informed by an assessment of the value of evidence derived from alternative designs [
      • West S.G.
      • Duan N.
      • Pequegnat W.
      • et al.
      Alternatives to the randomized controlled trial.
      ,
      • Bonell C.P.
      • Hargreaves J.
      • Cousens S.
      • et al.
      Alternatives to randomisation in the evaluation of public health interventions: Design challenges and solutions.
      ,

      National Institute of Drug Abuse. Alternatives to randomized clinical trials for American Indian and Alaska Native community research (Presenters: C. Hendricks Brown, T Dishion). 2011; http://ctndisseminationlibrary.org/othertraining.htm, March 14, 2011 (Webinar).

      ]. The lessons we are learning from this experience contribute important details to that conversation.

      Acknowledgments

      The authors thank the staff of the Native Boys and Girls Clubs, the communities in which the clubs are located, and the many families and youth, all of whom gave so generously of their time and support to make this project possible.

      Funding Sources

      This publication was made possible by Grant Number TP2AH000003 (Kaufman, PI) from the Office of Adolescent Health.

      References

      1. National Center for Health Statistics. Births: Final data for 2010. Available at: http://www.cdc.gov/nchs/data/nvsr/nvsr61/nvsr61_01.pdf. Accessed June 12, 2013.

      2. Orbis Associates. Circle of life HIV/AIDS and STD prevention curriculum: Wellness education for American Indian and Alaska Native middle school students. Washington, DC, 2002.

        • Kaufman C.E.
        • Litchfield A.
        • Schupman E.
        • et al.
        Circle of life HIV/AIDS-prevention intervention for American Indian and Alaska Native youth.
        Am Indian Alsk Native Ment Health Res. 2012; 19: 140-153
      3. Kaufman CE, Whitesell NR, Keane EM, et al. Effectiveness of circle of life, an HIV-preventive intervention for American Indian middle-school youth: A group-randomized trial in a Northern Plains tribe. Am J Public Health. In press.

        • Wallerstein N.
        • Duran B.
        Community-based participatory research contributions to intervention research: The intersection of science and practice to improve health equity.
        Am J Public Health. 2010; 100: S40-S46
        • Goins R.T.
        • Garroutte E.M.
        • Fox S.L.
        • et al.
        Theory and practice in participatory research: Lessons from the Native Elder Care Study.
        The Gerontologist. Jun 2011; 51: 285-294
        • Walkup J.T.
        • Barlow A.
        • Mullany B.C.
        • et al.
        Randomized controlled trial of a paraprofessional-delivered in-home intervention for young reservation-based American Indian mothers.
        J Am Acad Child Adolesc Psychiatry. 2009; 48: 591-601
      4. U.S. Census Bureau. United States Census 2010. 2012; http://www.census.gov. Accessed October 15, 2012.

        • Thornton R.
        American Indian holocaust and survival: A population history since 1492.
        University of Oklahoma Press, Norman, OK1987
        • Deloria V.
        • Wildcat D.R.
        Power and place: Indian education in America.
        Fulcrum Publishing, Golden, CO2001
        • Harrod H.L.
        Becoming and remaining a people: Native American religions on the northern plains.
        University of Arizona Press, Tucson1995
      5. Nagel J. American Indian ethnic renewal: Red power and the resurgence of identity and culture. New York City, NY: Oxford University Press. USA; 1997.

        • Norton I.M.
        • Manson S.M.
        Research in American Indian and Alaska Native communities: Navigating the cultural universe of values and process.
        J Consult Clin Psychol. 1996; 64: 856-860
        • Kaufman C.E.
        • Ramarao S.
        Community confidentiality, consent, and the individual research process: Implications for demographic research.
        Popul Res Policy Rev. 2005; 24: 149-173
      6. Colon L, Stimmer M, Froschl M, Sprung B. After-school science PLUS activity guide. 2012; Education Equity Center at FHI360, http://www.edequity.org/programs/science-and-math-programs#23.

        • Lawrence J.
        The Indian health service and the sterilization of Native American women.
        Am Indian Q. 2000; 24: 400-419
        • Yellow Horse Brave
        • Heart M.
        • Chase J.
        • Elkins J.
        • Altschul D.B.
        Historical trauma among indigenous peoples of the Americas: Concepts, research, and clinical considerations.
        J Psychoactive Drugs. 2011; 43: 282-290
      7. Gibson J. Crime accompanies flow of oil workers into North Dakota. 2012; http://www.earthisland.org/journal/index.php/elist/eListRead/crime_accompanies_flow_of_oil_workers_into_north_dakota/. Accessed August 23, 2013.

      8. Kusnetz N. North Dakota's oil boom brings damage along with prosperity. 2012; http://www.propublica.org/article/the-other-fracking-north-dakotas-oil-boom-brings-damage-along-with-prosperity. Accessed August 23, 2013.

        • US Department of Justice
        Criminal victimization, 2010.
        National Crime Victimization Survey Bulletin, 2011; September
        • Centers for Disease Control and Prevention
        CDC Health Disparities and Inequalities Report—United States, 2011.
        MMWR - Morb Mortal Wkly Rep. 2011; 60
        • Bachman R.
        • Zaykowski H.
        • Kallmyer R.
        • et al.
        Violence against American Indian and Alaska Native women and the criminal justice response: What is known. report funded by and submitted to the U.S.
        Department of Justice, 2008
        • West S.G.
        • Duan N.
        • Pequegnat W.
        • et al.
        Alternatives to the randomized controlled trial.
        Am J Public Health. 2008; 98: 1359-1366
        • Bonell C.P.
        • Hargreaves J.
        • Cousens S.
        • et al.
        Alternatives to randomisation in the evaluation of public health interventions: Design challenges and solutions.
        J Epidemiol Community Health. 2011; 65: 582-587
      9. National Institute of Drug Abuse. Alternatives to randomized clinical trials for American Indian and Alaska Native community research (Presenters: C. Hendricks Brown, T Dishion). 2011; http://ctndisseminationlibrary.org/othertraining.htm, March 14, 2011 (Webinar).

        • Menacker F.
        • Martin J.A.
        • MacDorman M.F.
        • Ventura S.J.
        Births to 10–14 year-old mothers, 1990–2002: Trends and health outcomes. National Vital Statistics Reports; vol 53 no 7.
        National Center for Health Statistics, Hyattsville, MD2004