Center for Methods in Implementation and Prevention Science (CMIPS) and Biostatistics Series: "New Opportunities for Designing Implementation and Hybrid Implementation/Effectiveness Trials and Other Evaluations"

Abstract: Despite the recent proliferation of implementation and hybrid implementation/effectiveness trials, there are many more opportunities for rigorous designs that serve the needs of service systems, policy makers, community, as well as researchers. Because these trials are most often based on an intervention that is already determined to be “evidence-based,” the belief in equipoise regarding this intervention is not tenable, so it is generally inappropriate to withhold such a beneficial intervention the way traditional RCTs with control conditions would. In their place, trials that examine alternative strategies to implement this intervention can be designed to meet diverse stakeholders’ needs without sacrificing scientific integrity.

To this end, we discuss a range of Rollout trial designs, where units are randomly assigned at different times to different implementation conditions. Rollout designs include the stepped wedge design, but there are also many variations that address questions that a stepped wedge design does not address. These include head-to-head implementation trials, multilevel implementations, and trials that compare full, partial, or sustained implementation; they also permit examination of learning collaboratives where units influence one another. For pure implementation and hybrid trials that examine both effectiveness and implementation, we discuss factors affecting statistical power. Finally, we discuss examples of simulation experiments using agent-based models that provide in-silico evaluations of alternative implementation strategies that provide timely decision support to organizations in real time.