Background: To inform scaling decisions, pilot/feasibility studies must be free of biases. The Risk of Generalizability Biases (RGBs), a set of biases for behavioral interventions, focus on factors introduced during early-stage studies that can lead to inflated early effects and large-scale disappointments. RGBs include researchers delivering an intervention (delivery agent bias), providing unscalable levels of support for implementation (implementation support bias), delivering the intervention to a non-representative audience (target audience bias) or testing an intervention for shorter durations than intended in the larger-scale trial (duration bias). The purpose of this study was to identify the presence and impact of RGBs in behavioral interventions that have a published pilot and larger-scale trial on a topic related to adult obesity.

Methods: First, searches were conducted across 5 databases to identify systematic reviews/meta-analyses of behavioral interventions on a topic related to adult (>18yrs) obesity (e.g., weight loss/management, improve activity/diet); Second, studies included within reviews were searched for reference to early-stage work; Third, published pilot/feasibility studies were confirmed as the early-stage study informing the published larger-scale trial. RGBs were coded in the pilot and larger-scale trial pairs, along with quantitative extraction of outcomes. Multi-level meta-regression models were used to examine the impact of the presence/absence of RGBs on the change in the standardized mean difference (SMD) from pilot to larger-scale trial.

Results: A total of 114 pairs, representing 230 studies, were identified. The four most prevalent RGBs were duration (33%), delivery agent (30%), implementation support (23%), and target audience (22%) bias. The presence of the biases in the pilot and absent from the larger-scale trial was associated with a reduction in the SMD from pilot to larger-scale trial ranging from -49% to -77%, compared to a reduction in the SMD ranging from -31% to -38% for pairs without these biases. 

Conclusions: The presence of RGBs in early-stage studies contributes to inflated effects, potentially leading to premature testing of behavioral interventions in larger-scale trials. Removal of RGBs in the design and execution of early-stage studies is critical to evaluate whether a behavioral intervention is ready for larger-scale testing.

Purpose:   Partnerships between charitable food systems and health care systems have been forming across the country to support individuals and families experiencing food insecurity, yet little research has been given to the unique implementation challenges or essential practices of these partnerships, particularly from a food bank perspective.  The purpose of this study was to describe existing implementation challenges and essential practices to facilitate equitable partnerships between food bank systems and health care systems.

Methods:  Semi-structured interviews were completed with food bank leaders and food insecurity experts (n=6).  Interviews focused on understanding existing partnerships, barriers to implementation and sustainability, and facilitators to sustainable program development.  Notes were taken during interviews and then discussed with the research team.  Themes were generated through iterative discussions of interview notes.

Results:  Results suggest unique implementation challenges exist at all levels of food bank/healthcare partnerships including the partnership, program, and system levels.  Partnership-level implementation challenges focused on issues of scale and data collection, sharing, and analysis.  Program-level implementation challenges focused on food and produce expectations, while structural-level implementation challenges included issues of food safety, subsidized food regulations, and patient privacy.  Participants also identified essential practices of partnerships between food banks and health care systems that will support sustainable and equitable partnerships such as the necessity of leadership support for social determinants of health, mission compatibility, food insecurity training, and partnership champions.

Conclusions:  Although significant implementation challenges exist for food bank/health care partnership, sustainable and equitable partnerships that support the physical and social health of families at risk of food insecurity are possible.  To support the development of this work, a systematic approach to understanding partnership models is needed.  Further, leaders of healthcare systems and charitable food systems should collaboratively discuss the outlined implementation challenges to facilitate the sustainable implementation of food bank/healthcare partnerships.

Purpose: Choose to Move (CTM) is an effective, choice-based health promoting intervention for older adults co-created with government and community stakeholders. Few such interventions were scaled up; none were comprehensively evaluated across stages of scale-up. Our objectives are to describe 1) our approach to scale-up and 2) summarize our findings to date.

Methods: We embedded CTM scale-up in Yamey’s (2011) conceptual framework, applying tenets of successful scale-up. For example, we used a phased and integrated approach to scale-up. Phased implementation and scale-up spanned 7 years from formative evaluation [2015] to CTM Phases 1 & 2 [2016-2017; small scale] to CTM Phase 3 (2018-2020; large scale) to CTM Phase 4 (2020-2021). We systematically adapted CTM between phases to respond to stakeholders (2017); to scale-out to rural and remote communities (2018); to reduce costs (2019) and for the COVID-19 context (2020).

Results/Findings: Our community partners implemented > 290 CTM programs that engaged > 2700 older adults across >70 sites in British Columbia. CTM Phases 1, 2 and 3 increased older adults’ physical activity, mobility and social connectedness--although we experienced ‘voltage drop’ as scale-up proceeded. Our delivery system deemed CTM implementation was of high quality, feasible and appropriate. Critical to scale-up success: 1) an evidence-based, flexible and adaptable program, 2) committed government and community partners, 3) alignment with organizational priorities, 4) active participation of stakeholders to design, adapt and implement CTM, and 5) strong leadership and governance by our support team.

Conclusions: CTM offers a rare example of implementation to scale-up of an effective health promoting intervention for older adults. CTM can be effectively adapted for different contexts and delivery systems.

Purpose: To examine the impact of the Childcare PhysicaL ActivitY (PLAY) policy on the objectively measured physical activity levels of young children (2-5 years) in childcare. With 8 recommendations, The Childcare PLAY policy was an evidence-informed, institutional-level document targeting children’s physical activity, outdoor play, and sedentary behaviours.

Methods: Nine childcare centres in London, Ontario participated in the cluster, randomized controlled trial. Centres in the control condition (n = 4) continued their typical daily routine, while centres in the intervention condition (n = 5) implemented the Childcare PLAY policy for 8 weeks. To assess activity levels, toddlers and preschoolers wore ActiGraph wGT3X-BT accelerometers for 5 consecutive days during childcare hours at baseline, mid- and post-intervention, and at 6-month follow-up. Raw accelerometry data were converted to 15s epochs and age- and device-specific cut-points were applied. Participants with 2+ days of > 5 hrs of wear-time at baseline and at one additional time point were included in the linear mixed effects modeling. An adjusted alpha (p < .017) was used to account for multiple comparison bias.

Results: A total of 128 young children (2.66 ± 0.62 years) had valid accelerometry data. The interaction between group and time was statistically significant for light physical activity, F(3,341) = 6.73, p < .017. No evidence of an association was ascertained between the PLAY policy and total physical activity, moderate-to-vigorous physical activity, or sedentary time.

Conclusion: The findings indicate the Childcare PLAY Policy was effective at increasing toddlers’ and preschoolers’ light physical activity. This pilot intervention appears promising for supporting some degree of physical activity among children in childcare settings; however, additional investigations are needed to explore the feasibility and effectiveness with larger and more diverse samples.

Purpose: With type two diabetes on the rise, there is a need for more prevention programs to reach the large number of at-risk individuals. The purpose of this research was to examine the implementation process, strategies, and multilevel contextual factors as an evidence-based diabetes prevention program was implemented into two local community organization sites. In-depth reporting of implementation strategies and context are needed to support future studies. 

Methods: Small Steps for Big Changes is a brief-counselling diet and exercise modification program for individuals at-risk of developing type 2 diabetes with demonstrated success.  A one-year partnered planning process with a local not-for-profit community organization co-developed an implementation plan for the translation of this project. This research was guided by a pragmatic epistemology. Semi-structured interviews were conducted with community organization staff who delivered the program (n = 8), and a focus group was completed with implementation support staff (n = 5) from both community sites. Interviews were transcribed verbatim and thematically analyzed using a template approach. The consolidated framework for implementation research (CFIR) was used to guide the analysis of this study. The CFIR is a multilevel implementation determinant framework with strong theoretical heritage. Within the template approach, themes were first inductively identified to ensure salient ideas were captured, then identified themes were deductively linked to CFIR constructs.

Results: Implementation strategies used were appropriate, well-received by staff and promoted successful implementation. Several CFIR constructs were identified from all five domains: (a) process, (b) intervention characteristics, (c) outer setting, (d) inner setting, and (e) individual characteristics. Specifically, results revealed the partnered one-year planning process, program components and structure, level of support, and synergy between program and context were important factors in the implementation.

Conclusion: This study describes the strategies and contextual factors used to implement a community-based diabetes prevention program into two community sites. Successful implementation was supported by a fully engaged, partnered approach to planning, and subsequently executing, an implementation effort. The CFIR provided a thorough framework to identify and evaluate multilevel contextual factors impacting implementation. Results demonstrate a successful approach to working with a community partner to support implementation.

Choose to Move (CTM) is an effective, choice-based health-promoting intervention for older adults. With our partners, we are scaling up CTM across British Columbia (BC), Canada using a phased approach [Phases 1 and 2 initial scale-up; 2016-17]. In Phase 3 [2018-20] we adapted CTM for broad scale-up. Adaptations enabling implementation at scale may lead to ‘voltage drop’. Therefore, we aimed to: 1) determine the impact of CTM Phase 3 on older adults’ PA, mobility and social connectedness, and 2) quantify the voltage drop. We conducted a pre-post study with 1013 older adults (72.9±6.3 yrs; 81% women) from 141 Phase 3 CTM programs delivered by two community partners in 38 BC communities. We assessed PA, social isolation, loneliness and mobility via survey at 0 (baseline), 3 (mid-intervention) and 6 (end-intervention) months. We fit mixed-effects models to describe change in each outcome. We quantified voltage drop as the percent of Phase 1/2 effect size (i.e., change from baseline to 3- and 6-months) retained in Phase 3. PA increased during the active intervention (0-3 months) in younger (60–74 yrs; +1 day/week; p<0.001) and older (≥75 yrs; +0.8 days/week; p<0.001) participants. PA gains were maintained in younger (+0.7 days/week) and older participants (+0.5 days/week) (p<0.001) at 6 months. In younger participants, social isolation and loneliness declined and mobility improved at 3 and 6 months (p<0.05). In older participants, social isolation and loneliness declined at 3 and 6 months, respectively (p<0.05), and mobility did not change significantly from baseline. In younger participants, a ‘voltage drop’ of 63% and 50% was evident for PA at 3 and 6 months, respectively, and, in both age groups, the voltage drop ranged from 20-90% for loneliness and mobility. Effect sizes for PA in older participants and social isolation in both age groups indicated greater benefit in Phase 3, compared with Phases 1/2. Although we adapted CTM for broad scale-up, its positive benefit on older adults’ health persisted. However, we noted a ‘voltage drop’ for the intervention effect, particularly in loneliness and mobility outcomes. In future, we will investigate the influence of implementation fidelity on the ‘voltage drop’.