Home Engineering Behavioral Intervention Research: Designing, Evaluating, and Implementing
SYNTHESIS AND FUTURE DIRECTIONS
This book addresses the complex considerations in developing, evaluating, and implementing behavioral interventions. We highlight the major issues in the conduct of this form of inquiry, summarize what is known, and highlight areas where more research is needed to advance best practices. A broad range of topics are covered including strategies for developing an intervention, study design considerations, strategies for evaluating an intervention, lessons learned from the implementation of proven interventions, and professional issues associated with being behavioral intervention researchers. Our goal is to provide important insights into the critical issues for advancing interventions to enhance the likelihood that they will be implemented and that their impact will be optimized.
One of the overarching themes of this book that underscores its unique and important perspective is the need for new models and strategies to develop and evaluate behavioral interventions that foster successful implementation of proven interventions in real-world settings. In this regard, we propose that a systems and user-centered perspective can propel behavioral intervention research forward and lead to a better fit between our interventions and targeted contexts (clinical, social service, and community settings) and populations.
The evidence is overwhelming that behavioral interventions directed at changing behavioral, health, psychosocial, and environmental outcomes can make a significant difference (statistically, clinically, and from a cost perspective) in preventing, mitigating, or reducing pressing public health issues such as chronic disease management, health disparities, mental health, or cognitive and physical functioning. Yet novel strategies are needed for moving interventions more rapidly from inception to implementation. In this final chapter, we provide a summary of the major points discussed in this book, emergent themes, and key directions for future research.
In Part I, we focused on what we refer to as the “heart of the matter,” that is, designing behavioral interventions. Taken as a whole, all five chapters discuss different but interrelated considerations in the design of interventions. We began in Chapter 1 with a justification for behavioral interventions showing that social, emotional, cognitive, and behavioral health changes are more powerful than genetics in shaping key behavioral and health outcomes. That chapter also provides an important overarching social ecological framework that locates behavioral interventions within interactive layers of influence including the immediate environment in which an intervention is embedded as well as within communities and social policies. This social ecological framework is referred to throughout the book as it provides the broad context in which interventions are designed, evaluated, and implemented and identifies the many interrelated factors that need to be considered in behavioral intervention research. By focusing on the ecological context in which we develop our interventions, we may be able to make wiser decisions regarding an intervention’s delivery characteristics and the approaches used to evaluate an intervention to advance the evidence base more rapidly. This is our essential premise and a key theme throughout this book, although we recognize that this point needs empirical verification.
In Chapter 2, we examine in some specificity what has been referred to as the “pipeline” for developing the evidence for an intervention. Whereas drug discovery and biomedical research follow a prescribed set of research steps moving from bench to bed to public health impact, for behavioral intervention research there is no consensus, agreed-upon approach, or recipe for advancing interventions and then implementing and sustaining them in real-world settings. Here we suggest that the four-phase traditional pathway (Phase I—proof of concept; Phase II—feasibility, safety, pilot testing; Phase III—efficacy; and Phase IV—effectiveness) imposed on behavioral intervention research from the experience with drug discovery is inadequate. We postulate, as others, that additional phases must be considered that reflect the specific activities or work related to implementation. We thus suggest that the pipeline be elongated or extended to include a Phase VI (translation/implementation activities), a Phase VII (dissemination activities), and a Phase VIII (sustainability or maintenance activities). We also group these phases into four larger domains of interconnected activities: development, evaluation, implementation, and sustainability. Chunking phases in this manner may better promote and reflect the iterative process of intervention development and enable a more fluid approach within each domain such that implementation processes may shape developmental steps and so forth. Although we refer to the elongated seven-phase pipeline throughout this book, we also highlight a more iterative style for intervention development that integrates various strategies such as a more user-centered perspective (see Chapter 2, Figure 2.3). We also identify various innovative strategies to help move intervention work forward more rapidly. We recognize that evidence is needed that supports the use of any of these strategies or their combination for shortening the pipeline for advancing interventions.
In Chapter 3, we take a deep dive into intervention development and identify eight essential tasks for designing an intervention. Obviously, interventions are designed to address problem areas, so one of the first tasks in intervention development is to clearly articulate a problem area, the population most at risk or affected, the pathways contributing to the problem area, specification as to what can be modified, the outcomes of interest, and current practices for addressing the problem. While there is not an agreed-upon formulaic approach to intervention development, we provide guidance as to how to systematically move forward with a viable intervention. A critical theme is assuring that the context and user perspective is identified early on and integrated into the developmental process of evolving an intervention.
Theory remains a neglected stepchild of intervention development although ample evidence shows that, without a theory or conceptual framework, interventions are not as effective. In Chapter 4, we discuss the roles of theory in behavioral intervention work at each phase of the pipeline and also highlight some of the issues in using theories. For example, one issue is that there may not be an adequate theory that addresses the particular problem area and/or intervention idea. Another issue is that it is often necessary to use more than one theory or conceptual framework, particularly for multimodal (interventions directed at changing different pathways by which a problem area occurs) and multicomponent (interventions that involve different elements to address a complex problem) interventions. How theories are combined or how one theory relates to another can be challenging. Many theories do not adequately account for the role of cultural influences on behavior and also lack specificity as to how behavior change occurs. Thus, although theory is critical to behavioral intervention research, future research is needed to attend to developing new and refining existing theories.
In Chapter 5, we discuss delivery characteristics of behavioral interventions, which are the backbone of any intervention. Delivery characteristics include the content of the intervention, dose and intensity, the mode of delivery (e.g., face- to-face, group, through videoconferencing), and staffing requirements. Behavioral interventions address a wide range of issues and populations and can target individuals, communities, organizations, or the social, physical, or policy environments. Interventions can thus involve many different approaches such as counseling, training, psychotherapy, education, skill building, stress management techniques, environmental modifications, or some combination thereof. They may also target different aspects of behavior such as coping skills, knowledge, or affect, and/or involve modifications to the physical and social environment. That chapter offers an overview of the range of issues/factors that need to be considered in the design of behavioral interventions such as what should be delivered, how it should be delivered, and by whom, at what intensity, and for how long.
An important issue in behavioral intervention research is validity: both internal validity, which refers to the accuracy and reliability of the outcomes of a study, and external validity or the generalizability of the study findings. Ensuring internal and external validity requires a systematic approach to the design, evaluation, and implementation of behavioral interventions. Thus, in Chapter 6, we discuss the topic of standardization. We identify the aspects of an intervention that need to be standardized as well as strategies to help foster standardization such as developing manuals of operation and training team members in intervention protocols. We also touch on the topic of tailoring. Our intent is to highlight the importance of taking steps to ensure that research activities at all stages of the pipeline are of the highest scientific quality so that potential benefits of the intervention are maximized and potential threats to validity are minimized.
Given the ubiquitous and important role of technology in health care and other settings, technology is increasingly being used in behavioral intervention research as a vehicle for intervention delivery, as a data collection tool, and as an aid to data analyses. Clearly, the use of technology within behavioral intervention research holds promise in terms of enhancing the outreach, efficiency, and cost-effectiveness of behavioral interventions. In Chapter 7, we provide examples of technology applications in intervention delivery and data collection. We also discuss the advantages and challenges associated with technology-based approaches and highlight issues such as privacy, usability, and the informed consent process that warrant further investigation.
Part II examines strategies for evaluating interventions. Highlighted are essential considerations in evaluation and issues specific to behavioral intervention research. We begin the discussion in Chapter 8 (invited contributor, Dr. Rebok) by providing a thorough examination of the role of control groups in evaluating behavioral interventions. As interventions are often compared to control groups in terms of their impact, attention to what type of control group should be used is critical. Unfortunately, in many studies, little attention is given to the selection of a control group. This chapter carefully describes the range of options available to investigators. Clearly, there is not one type of control group that can be applied uniformly across intervention studies; selecting a control group should be based on the scientific question posed, the resources available, and ethical considerations.
In most cases, it is not feasible to evaluate a behavioral intervention with populations, as populations are typically large and geographically diverse. Thus, a critical issue in behavioral intervention research, at all phases of the pipeline, is the selection of the study sample. Biases in sample selection can lead to errors in the interpretation of results and limitations in the ability to generalize findings from the study to other groups of people. In Chapter 9, we discuss two important considerations in sampling: sample size and sample composition. We also discuss approaches to sampling and provide some guidelines to help optimize the selection of samples for behavioral intervention research studies. A critical point is that the composition of the sample must reflect the targeted group and problem area that an intervention is designed to address. Although this point may seem obvious, historically, behavioral intervention research has been plagued with studies in which there is a mismatch between the intent of the intervention and composition of the study sample such that it is not possible to demonstrate treatment effects. For example, clearly, an intervention designed to address chronic pain must include a study sample that possesses a certain level of pain.
In Chapter 10 (invited contributor, Dr. Jimenez), we move on to a discussion of participant recruitment and retention. A carefully developed, implemented, and evaluated recruitment plan ensures adequate representation in intervention studies from diverse groups of individuals (e.g., age, gender, ethnicity) and is essential to developing interventions that are generalizable and that will ultimately positively impact on the health and well-being of populations. Yet recruitment and retention of study participants remain a central challenge to many investigators. We hope to provide guidance on this issue and describe general strategies to enhance recruitment-specific recommendations for the recruitment process. We also discuss issues that impact on participant retention.
Next, Chapter 11 (invited contributors, Drs. Gallo and Lee) provides a solid introduction to the use of mixed methodologies in behavioral intervention research. Mixed methods afford important insights concerning how interventions work, why they have a desired benefit, what aspects of interventions are more acceptable than others, and what are some facilitators and barriers to implementation of an intervention. There are numerous mixed methodologies that can be employed at any phase of intervention development and can be considered by investigators. Mixed methods promise to propel intervention development forward by enabling the distillation of complex processes in one study design.
Chapter 12 (invited contributor, Dr. Parisi) tackles the issue of treatment fidelity. That chapter offers a comprehensive understanding of this evolving construct and considers three of its dimensions—delivery, receipt, and enactment—and how treatment fidelity can be enhanced, monitored, and measured. Although fidelity has been considered in various fields such as psychology and education, it is not always integrated into study designs for behavioral interventions. Nevertheless, this is a critical consideration—we do not really know if our treatment effects are real if we do not monitor fidelity.
Behavioral intervention research by its nature involves human participants; thus, ethical considerations with respect to the involvement and treatment of research participants is a critical issue at all stages of the pipeline and throughout the research process. In Chapter 13, we provide an overview of the topic of research ethics and discuss some of the guidelines/requirements surrounding the ethical conduct of research. We also highlight issues related to the informed consent process, institutional review boards, and the role of data- and safety-monitoring boards. The topic of research ethics is complex, evolving, and a subject of much discussion.
Part III focuses on considerations related to measurement, outcomes, and analytics. We begin this discussion in Chapter 14 (invited contributor, Dr. Loewenstein), which focuses on the selection of measures, one of the most important and challenging aspects of behavioral intervention research. A common source of misleading results from intervention trials often stems from inadequate attention to the choice of measures—a mismatch between the intent of the intervention and the measurement strategy. Different measures can relay different stories about the impact of an intervention, so measures need to be carefully aligned with the research questions of interest. To address this issue, we discuss the types of measures available, provide general criteria for measure selection, and discuss the emerging role of technology in measurement.
We continue with the discussion of measurement in Chapter 15 (invited contributor, Dr. Harvey) and focus on performance-based measures. The primary reason for using objective performance-based measures, rather than other types of measurement strategies such as self-report or informant-report, is that these types of measures are purported to provide less biased assessments of performance and can identify specific areas of needed intervention. In addition, objective measurement approaches often include a selection of tasks or behaviors that can be assessed, and thus, they can be adapted to the unique needs of a population and task/behavior of interest.
The design of a study and data analysis are fundamental aspects of behavioral intervention research as they provide investigators the means by which to determine whether the obtained results show reliable differences between one or more treatment or control groups, or are merely obtained as a matter of chance. Chapter 16 (invited contributors, Drs. Savia and Loewenstein) highlights some of the key issues to consider when designing a research study to evaluate a behavioral intervention. Also discussed are key analytic strategies including mediating and moderating effects that can provide understanding for whom an intervention is most effective and the mechanisms by which an intervention affects targeted outcomes.
In behavioral intervention research, the outcomes of a study or evidence for a treatment can be evaluated according to different criteria: effectiveness with respect to important outcomes, relevance, feasibility, cost versus benefit, and sustainability potential. Traditionally, the focus has been on identifying the efficacy of an intervention on the basis of a randomized trial with respect to specific outcomes.
However, with an increased emphasis on the importance of evidence-based treatments, a higher bar must also be met and that is identifying the clinical significance of an intervention. Thus, in Chapter 17, we discuss the topic of clinical significance. We define what is meant by “clinical significance” within the realm of behavioral intervention research and review methods for measuring clinical significance. We also provide guidance on how to maximize clinical significance within a behavioral intervention trial.
Yet another analytic consideration is an economic evaluation of our interventions. Chapter 18 (invited contributors, Dr. Pizzi, Mr. Jutkowitz, and Dr. Nyman) provides a comprehensive overview of economic evaluations and their application in behavioral intervention research. Until recently, understanding the costs of interventions has received limited attention. However, with the current emphasis on cost-effectiveness, there is a sea change and greater appreciation for the importance of systematically evaluating the economic value of behavioral interventions and comparing their costs and cost-effectiveness to traditional practices. While economic evaluation may be foreign to many behavioral interventionists, this chapter provides a clear and methodologically sound presentation of its essential aspects and a basic foray into key considerations of this approach.
In Part IV, we turn our attention to considerations related to implementation and dissemination. Chapter 19 (invited contributor, Dr. Hodgson) provides foundational knowledge concerning the emerging science of implementation and the relevant theories that have emerged to help understand the implementation of evidence-based programs. The premise, in keeping with the major theme of this book, is that theories can also inform intervention development by alerting investigators to considerations or factors early on in the pipeline that may impact implementation of an intervention.
Next, Chapter 20 (invited contributor, Dr. Leff) provides an interesting discussion on lessons learned from implementing evidence-based programs. Using three evidence-based programs as exemplars, the importance of context (settings, organizations, or agencies), interventionist characteristics, and value alignment (e.g., between intervention outcomes and stakeholder values and needs) are illustrated. The lessons derived from the implementation experience with these three cases can inform decision making when developing a behavioral intervention.
Chapter 21 (invited contributor, Mr. Beilenson) takes a focused look at the dissemination phase and asks the essential question: What does it take to get an evidence-based program widely distributed? Dissemination is typically viewed as a simple act of either publishing an outcomes paper or providing a presentation. However, highlighted by this chapter, this phase of the pipeline has its own unique processes that require considerable time, commitment, and resources on the part of an investigative team. An innovative concept is the notion of a dissemination infrastructure, particularly in the absence of an industry base for behavioral intervention research such as exists in pharmacological development.
Finally, in Part V, key professional issues are examined. As behavioral intervention research is unlike other forms of inquiry, Chapter 22 examines some of the most pressing and challenging aspects including staffing, hiring, management and support, and building and leading teams. We also emphasize the importance of collaboration and discuss career development paths. For example, it may be that an investigator does not have the desire, resources, or know-how to move his or her proven intervention forward for implementation and dissemination. Rather, an investigator may choose to work on enhancing the evidence base of the intervention or other related scientific questions or develop a new intervention. These become critical personal, professional, and ethical questions that behavioral intervention researchers must grapple with and that we raise in this chapter.
Chapter 23 focuses on grant writing as it is an essential responsibility of behavioral intervention researchers that must occur throughout the pipeline. We emphasize basic grant writing strategies as well as considerations specific to intervention proposals. The primary challenge for all behavioral intervention researchers is the lack of available funding to address the many scientific questions concerning an intervention. Despite the difficulties in obtaining money, proposing a novel intervention and strong science, as well as having the skill to package one’s ideas, are all critical for successful grant writing.
Finally, we look at the important role of publishing in Chapter 24. We debunk the myth that intervention work delays publication productivity and highlight the many opportunities to publish even when main outcomes from a trial are not available.
Overall, this book illustrates the many challenges in behavioral intervention research and the complex decisions that need to be made within any one particular intervention study while one advances the evidence base for an intervention across the pipeline. Each chapter has offered best practices, identified methodological gaps, and has sought to push the field forward to adopt new ways of advancing novel behavioral interventions.