BetterBirth

Partners: Beth Israel Deaconess Medical Center, Brigham and Women’s Hospital, Harvard Medical School

Study Results: NeoReviews, Lessons from the BetterBirth Trial: A Practical Roadmap for Complex Intervention Studies
, February 2019

The growing recognition that multi-level approaches are needed to address population health issues and improve clinical outcomes has spurred an increase in complex interventions – those that contain several interacting components. This, in turn, has underscored the need for methodologies to effectively study, evaluate, and extrapolate recommendations from such complex interventions. The implementation of the BetterBirth Trial, conducted November 2014 to December 2017 in Uttar Pradesh, India, yielded insights that could aid the design, implementation, and dissemination of future, large-scale studies of complex intervention.

The BetterBirth Trial was a large, cluster randomized controlled trial that targeted maternal and perinatal mortality as primary outcomes – a particularly difficult research target because of the large numbers needed for the relatively rare outcome of maternal mortality. The BetterBirth Trial addressed the effect of the World Health Organization’s Safe Childbirth Checklist, a bundle of 28 essential birth practices proven to save lives during the hours around childbirth when women and newborns are at greatest risk. The BetterBirth intervention also incorporates peer coaching and data feedback in addition to use of the checklist. The trial examined whether the BetterBirth intervention was effective in reducing early (seven-day) maternal mortality and morbidity and perinatal mortality in facilities in Uttar Pradesh. Implemented through a collaboration among private and public partners, the trial included 120 primary-level facilities and enrolled approximately 160,000 woman-newborn pairs. The study found that adherence to essential birth practices increased among birth attendants in intervention facilities, but there was no impact on maternal mortality and morbidity or perinatal mortality. The scale of the trial provides a roadmap of practical lessons – ranging from formative work and pilot testing to dissemination strategy – for planning other studies with large-scale, complex interventions.

What did we learn?

• Budget and plan sufficient time for early formative work in developing the implementation strategy for a complex intervention.
• Achieve a balance between the science and art of implementation within the confines of a trial design.
• Align local implementation teams and research teams through a standardized strategy with clear roles and scope of work.
• Consider including front-line users of the intervention in addition to local leadership on the implementation team to maximize buy-in and integrate the intervention into existing workflows.
• Embed implementation teams in the local context as much as possible to facilitate troubleshooting and ensure the intervention is implemented with fidelity.
• Measure the fidelity to the intervention implementation strategy during the trial or evaluation.
• Provide refresher training to the implementation team, depending on staff turnover, fidelity to implementation strategy, and duration of the study.
• Partner with a communications specialist and graphic designer to develop messaging from the trial results and roll out a dissemination strategy to reach different audiences, including the study participants.
• Consider whether some data analyses could be completed for earlier dissemination while waiting for the main trial results.

Conclusions
It is important for the research community to consolidate lessons learned from the BetterBirth Trial and similar trials and use (or create as needed) standard frameworks for these large-scale trials with complex interventions. This way, the best practices are identified and implemented for clinical effectiveness in diverse global populations.

Partners: Community Empowerment Lab, Jawaharlal Nehru Medical College, Harvard T.H. Chan School of Public Health, Population Services International, Indian Council of Medical Research, World Health Organization, Bill & Melinda Gates Foundation and Populations Services International.

Study results: International Journal of Gynecology and Obstetrics, Integration of the Opportunity‐Ability‐Motivation behavior change framework into a coaching‐based WHO Safe Childbirth Checklist program in India, June 2018.

Completion of essential birth practices (EBPs) during childbirth reduces deaths, but it can be challenging for care providers in areas that are resource-constrained to deliver them consistently. To improve the delivery of EBPs in these areas, the BetterBirth program implemented the WHO Safe Childbirth Checklist in India with a program of peer-to-peer coaching. BetterBirth coaches integrated a framework of Opportunity‐Ability‐Motivation plus Supplies (OAMS) into their strategy to maximize Checklist usage. The OAMS framework is designed to address these four major barriers to behavior change, applying the methodology to the challenges seen in resource-constrained settings and facilitating improvement.

As part of the BetterBirth randomized control trial, Ariadne researchers conducted a sub-analysis to examine how the OAMS framework impacted 8 intervention facilities in the BetterBirth trial between December 2014 and October 2015.

Over eight months of coaching, the coaches:

1. Encouraged behavior change.
2. Observed, documented and delivered feedback about EBP performance and Checklist use.
3. Helped with joint problem-solving to overcome obstacles that prevent change.

What did we learn?

• There were substantial barriers to care. Using the OAMS framework, coaches recorded 1,048 barriers to care throughout the 8 months of the study, with opportunity and motivation as the ones most frequently listed.
• The coaches successfully implemented OAMS. The coaches appropriately categorized 99.8% of all barriers and provided an appropriate strategy for 85.8 percent of them.
• EBP adherence increased. Throughout the study, birth attendants’ adherence increased from 84 to 95.5 percent while a coach was present. Coaches documented that the two most common barriers, motivation and opportunity, decreased over the course of the study.
• Though quality improved, there was no change in mortality.

Conclusions:

The OAMS framework to implement the Checklist offered a concrete and measurable way to teach coaches and team leaders how to recognize and combat barriers to care by developing quality improvement strategies. Successful implementation of the framework enabled coaches to diagnose barriers and respond quickly. This suggests that coaching that uses OAMS to implement the Checklist can inspire behavior change in providers and encourage them to use the skills they gained during training. OAMS is therefore an important framework to consider for programs that aim to improve quality of care and adherence to EBPs.

Partners: Jawaharlal Nehru Medical College, Population Services International, Community Empowerment Lab, World Health Organization, Governments of India and Uttar Pradesh

Study results: Trials, Effectiveness of the WHO Safe Childbirth Checklist program in reducing severe maternal, fetal, and newborn harm in Uttar Pradesh, India: study protocol for a matched-pair, cluster-randomized controlled trial, December 2016; New England Journal of Medicine, Outcomes of a coaching-based WHO Safe Childbirth Checklist program in India, December 2017.

Effective, scalable strategies to improve maternal, fetal, and newborn health and reduce preventable morbidity and mortality are urgently needed in low- and middle-income countries. Building on the successes of previous checklist-based programs, the World Health Organization (WHO) and partners led the development of the Safe Childbirth Checklist, a 28-item list of essential birth practices linked with improved maternal and newborn outcomes. Pilot-testing of the Checklist in southern India demonstrated dramatic improvements in adherence by health workers to essential birth practices. The BetterBirth study sought to measure the effectiveness of Checklist impact on essential birth practices, deaths and complications at a larger scale.

This matched-pair, cluster-randomized controlled study was conducted in 120 facilities across 24 districts in Uttar Pradesh, India. The intervention was a package consisting of the Checklist, peer-to-peer coaching and feedback using a cloud-based data collection and reporting system. The peer-to-peer coaching consisted of supportive supervision and real-time data feedback over an 8-month period with decreasing intensity. A facility-based childbirth quality coordinator was trained and supported to drive sustained behavior change after the departure of the BetterBirth team. This quality improvement program also included leadership engagement and a 2-day educational launch of the Checklist.

Birth attendants, women and their newborns who present to the study facilities for childbirth were enrolled in the study. The study’s primary outcome was a composite measure that included maternal death, maternal complications, stillbirth, and newborn death, occurring within 7 days after birth.

What did we learn?

• After two months of coaching, intervention sites performed significantly more essential birth practices than control sites. Intervention sites completed 73 percent of Checklist items, while the control sites completed just 42 percent.
• At the end of the study, intervention sites continued to complete more essential birth practices than control sites. Four months after coaching ended in the intervention sites, they performed 62 percent of Checklist items. Though this was lower than the two-month number, it was still significantly higher than control sites, which remained unchanged at 44 percent.
• The intervention had no impact on maternal and neonatal mortality rates or maternal complications. Despite the high rates of adherence to Checklist items, the intervention facilities did not see lower rates of deaths among mothers and newborns, stillbirths, or complications for mothers.

Conclusions: The coaching-based WHO Safe Childbirth Checklist program produced increased adherence to essential birth practices, but did not reduce maternal and perinatal mortality and maternal morbidity. One interpretation of the findings is that increasing adherence to these practices is not worthwhile. The study team strongly believes this to be false, as each of the individual items on the Checklist have been proven effective in previous studies. One possible explanation for the results is that levels of adherence to essential birth practices in the intervention sites may not have been sufficient to affect outcomes. High-quality research on large-scale childbirth improvement programs must continue to measure both processes and outcomes of care as the field currently lacks a complete understanding of the complex interaction between quality of care, context and outcomes.

Partners: Gobabis District Hospital, Ministry of Health and Social Services, Gobabis, Namibia

Study results: BMJ Open Quality, Implementing the WHO Safe Childbirth Checklist: lessons learnt on a quality improvement initiative to improve mother and newborn care at Gobabis District Hospital, Namibia, August 2017

There are several essential birth practices proven to reduce the risk of mother and newborn death, such as handwashing and and the appropriate provision of antibiotics. However, these practices are not always followed. Leadership at Gobabis District Hospital in Namibia identified that despite relevant trainings, including emergency obstetric care and life-saving skills done at least twice a year, poor adherence to essential birth practices persisted. The hospital initiated a quality improvement initiative to increase adherence to practices proven to reduce harm to mothers and babies around the time of delivery.

The study team implemented use of the WHO Safe Childbirth Checklist using an approach that combined leadership support, coaching for birth attendants and organizational redesign. Implementation was led by a facility champion supported by a quality improvement team, consisting of one doctor and six nurse-midwives. The Checklist was modified to fit the local context and introduced to the maternity ward staff during a training that demonstrated its use. The expectation was that the birth attendants would use the Checklist as a recall and documentation tool for the delivery of the essential birth practices. The quality improvement team made adaptations to the implementation plan through a series of three, eight-week Plan–Do–Study–Act (PDSA) cycles. In a PDSA cycle, the study team makes plans for the work to be done, then executes the plan and collects information on what works and what doesn’t. Based on this information, the team makes the necessary adjustments.

What did we learn?

• Birth attendants completed more essential birth practices. During the 6-month period, the number of Checklist items completed increased, from 68 percent to 95 percent.
• The number of perinatal deaths decreased. The hospital’s perinatal mortality rates dropped from 22 deaths per 1,000 deliveries to 13.8 over the six-month period. This reduction in deaths was largely due to a drop in stillbirths.

Conclusions: Implementing the WHO Safe Childbirth Checklist using PDSA cycles was an approach reflecting the local context, which resulted in improvement in adherence to essential birth practices and stillbirth rates. Important organizational factors likely contributed to the success, including an adequate nurse-to-patient ratio in the maternity unit and previous quality improvement experience and leadership at the facility. Hospital leadership recognized that initial skills building was needed and addressed this gap by providing necessary training at the beginning of the project. Success of further scale-up of this Checklist-based program to other facilities will depend on using principles of quality improvement and supporting the development of local champions at the new facilities.

Partners: Children’s Investment Fund Foundation and World Health Organization

Study results: PLoS One, Improving Quality of Care for Maternal and Newborn Health: Prospective Pilot Study of the WHO Safe Childbirth Checklist, May 16, 2012

This pilot study examined the use of the WHO Safe Childbirth Checklist in 795 consecutive births at a sub-district level birth center in Karnataka, India.  The 29 practices in the checklist target the major causes of childbirth-related mortality, ranging from hand washing to the availability of essential equipment and medications to the measuring of key medical indicators of maternal and fetal health such as the mother’s temperature and blood pressure. Facilitating behavior change to increase adherence to evidence-based healthcare practices is challenging. In this study, we aimed to determine if a checklist implementation strategy could be devised and produce measurable change.

What did we learn?

• Delivery of essential childbirth-related care practices increased from an average of 10 out of 29 practices during the 499 consecutive births prior to checklist introduction to an average of 25 out of 29 practices after its introduction.
• Overall, there was an average 150 percent increase in adherence to accepted clinical practices at any given birth event, and 28 of 29 individual practices were delivered with significantly greater frequency.

Conclusion: A simple paper checklist alone is unlikely to result in lasting behavior change. In this study, the WHO Safe Childbirth Checklist was the central component of an implementation program based on a well-described change model carried out by hospital administration and clinical leaders, and the intervention markedly improved delivery of essential birth practices by health workers.

Partners: World Health Organization

Study results: BMJ Global Health, Implementing the WHO Safe Childbirth Checklist: lessons from a global collaboration, August 2017.

The WHO Safe Childbirth Checklist was developed to ensure the delivery of essential birth practices. Previous experience implementing such novel tools revealed the importance of testing them in a diverse range of settings and providing appropriate implementation support. A research collaboration was subsequently established to explore factors that influence use of the Checklist in a range of settings around the world.

Between November 2012 and March 2015, 34 institutions working in over 200 sites in 29 countries across all WHO regions registered projects with the collaboration. The sites agreed to conduct implementation research and share their experiences. Groups explored a range of questions that addressed compliance, barriers to and success factors of effective and sustained Checklist use.

Collaboration members were involved in frequent webinars and provided regular progress reports. A formal evaluation looked at facilitating factors and barriers to using the Checklist across different settings. Project leads from each collaboration site were asked to distribute surveys to their Checklist end users and implementation teams. A total of 134 end users and 38 implementation teams from 19 countries across all levels of income responded to the surveys.

What did we learn?

• Checklist modification is an important implementation step. Over half of the participants modified the Checklist before introduction (58 percent) to align it with their local context. For example, some teams added a hepatitis B check because they felt to be more of an issue for their respective populations. Adaptation allowed for a sense of local ownership among facility staff.
• Perceptions of training and coaching differed between end users and implementation teams. Nearly all teams reported that they had trained end users to use the Checklist (95 percent). Most end users reported receiving education about the purpose of the Checklist prior to its introduction (83 percent), but fewer than two-thirds stated they were trained to use it. Only a third reported receiving coaching or supervision while using the Checklist.
• Opinions on the usefulness of the Checklist varied by provider type. Midwives were less likely to believe that the Checklist improved practice, awareness or communication and teamwork than nurses and doctors. Training and ongoing supervision were frequently noted as effective solutions to address provider resistance.
• Though leadership support was present, implementation teams faced staffing, financial and supply chain challenges. Most implementation teams found adequate support from senior leadership (84 percent) throughout the introduction phase of the Checklist. However, teams reported common challenges: having fewer financial resources than needed, inadequate human resources to support implementation of the Checklist and supply chain issues. Training, ongoing supervision and ensuring availability of key supplies were cited as effective solutions.

Conclusions: The experiences of end users and implementation teams using the pilot version WHO Safe Childbirth Checklist highlight the need to engage local leadership, enable local ownership and ensure local relevance and acceptability in facilitating successful implementation. Adaptation of the Checklist is essential to ensure consistency with local guidelines and willingness to adopt the Checklist among end users. Early engagement, education and supervision are important Implementation steps that help overcome reluctance from different professional groups and establish comfort in using the tool among birth attendants. Implementation teams used an array of solutions to address human resource, financial, and supply availability challenges, many of which focused on multidisciplinary education and regular training sessions. These findings were invaluable in developing the final version of the WHO Safe Childbirth Checklist and its associated implementation guide. This experience provides useful insights for any institution wishing to implement the Checklist.

Partners: John D. and Catherine T. MacArthur Foundation

Study Results: BMC Pregnancy and Childbirth, Demand-side interventions for maternal care: evidence of more use, not better outcomes, November 2015

Limited access to quality routine and emergency care during pregnancy and delivery leaves a large number of women at risk of preventable death, particularly in low- and middle-income countries. Improving the quality of facility-based care is critical to reducing deaths of mothers and newborns in the first week after delivery.

While supply-side interventions, such as provider skills training, focus on improving the quality and reach of services, demand-side interventions are designed to address barriers that hinder a woman’s choice to seek or ability to reach accessible, high quality care. These interventions aim to increase the use of critical maternal health services and can be categorized into two approaches. Financial incentives are meant to reduce the cost of accessing services. Community mobilization efforts aim to improve knowledge and address cultural attitudes that may prevent uptake of life-saving services.

The study team sought to better understand the connection between demand-side interventions, utilization of services and health outcomes. The team conducted a review of literature in PubMed, using selected search terms to identify articles meeting the inclusion criteria. The bibliographies from these articles were reviewed to identify additional relevant papers. Over 580 articles were screened, 50 selected for full review and 16 met inclusion criteria. Of these, eight were about community mobilization interventions, seven were about financial incentive interventions and one included both approaches. The study team looked for increased utilization of antenatal visits, facility-based delivery and delivery with a skilled birth attendant.

What did we learn?

• Interventions were implemented across a range of low- and middle-income countries and settings. The identified interventions were implemented in Bangladesh, Burkina Faso, Malawi, Cambodia, Kenya, Tanzania, Nepal, India, Vietnam and Mexico. Nine of the interventions were implemented in rural settings, two in urban settings and five spanned both rural and urban settings.
• Demand-side interventions were effective in increasing uptake of key services. Associations between demand-side interventions and utilization measures were found in a number of studies. There was an increased use of maternal health services in five of the community mobilization and all seven of the financial incentive interventions.
• However, association with health outcome measures was varied across studies. Only 2 of the 16 studies reported reductions in maternal mortality, and only 4 reported reductions in neonatal mortality. No studies found a reduction in stillbirth rate. Ten studies reported on both utilization and outcomes, and only four of them reported improvement in both measures.

Conclusions: Demand-side interventions are associated with increased utilization of services, but the evidence of their impact on reducing early neonatal and maternal mortality varies greatly. More research is needed to understand how to maximize the potential of demand-side interventions to improve maternal and neonatal health outcomes including the role of quality improvement and coordination with supply-side interventions.

Partners: World Health Organization

Study results: International Journal of Gynaecology and Obstetrics, Designing the WHO Safe Childbirth Checklist program to improve quality of care at childbirth, June  2013

Poor quality care during births in hospitals and clinics in low- and middle-income settings is a contributing factor to preventable maternal and newborn harm. This paper describes the development of the Safe Childbirth Checklist by a group of experts seeking to improve outcomes for both mothers and newborns. The checklist they developed was tested for usability in nine countries, primarily in Africa and Asia, and refined.  A pilot testing in South India showed major improvements in health workers’ delivery of essential safety practices when the checklist was in use.

Conclusions: A novel checklist program has been developed to support health workers in low-resource settings. The WHO launched a global effort to support further evaluation of the program in a range of contexts, and a randomized trial is underway in North India to measure the effectiveness of the program in reducing severe maternal, fetal, and newborn harm.

Partners: World Health Organization and the Children’s Investment Fund Foundation

Study results: Global Health: Science and Practice, Bedside Availability of Prepared Oxytocin and Rapid Administration After Delivery to Prevent Postpartum Hemorrhage: An Observational Study in Karnataka India, June 2015

Postpartum hemorrhage is a leading cause of maternal death worldwide, but timely administration of oxytocin within one minute after delivery can reduce the incidence and severity of hemorrhage. Researchers set out to determine how prevalent this practice is in a birth facility in Karnataka, India, and whether advance planning can improve the likelihood that oxytocin is administered in a timely manner.  The researchers examined data collected through direct observation of childbirth practices for 330 vaginal deliveries.

What did we learn?

• In 330 deliveries, oxytocin was prepared and available for only 39 percent of those deliveries.
• However, in those cases where it was prepared, the likelihood that it was administered within 1 minute after delivery was greatly increased (15.6 percent of observed deliveries versus 3.0 percent)
• The overall time to oxytocin administration after delivery was 2.9 minutes sooner than in those cases where oxytocin was not prepared and available at the bedside in advance.

Conclusions: Efforts to reduce postpartum hemorrhage should include ensuring that oxytocin is prepared and available at the bedside in advance of all births.

Partners: Bill & Melinda Gates Foundation; MacArthur Foundation

Study results: Implementation Science, Learning Before Leaping:  Integration of an Adaptive Study Design Process Prior to Initiation of BetterBirth, a Large-scale Randomized Controlled Trial in Uttar Pradesh, India, April 2015

When researchers trained a local team of physicians and nurses to coach birth attendants to use the Safe Childbirth Checklist at two public facilities in Uttar Pradesh, India for 4-6 weeks, they saw little improvement in the performance of essential birth practices by the birth attendants. This paper describes the process and outcomes of adapting the BetterBirth Program prior to larger-scale implementation in a randomized controlled trial in Uttar Pradesh (UP), India.

What did we learn?

• Based on their observations during this initial phase, the researchers hypothesized that peer-to-peer coaching (nurse-to-nurse, physician-to-physician) and better training on topics such as behavior and systems change and leadership skills might improve success.
• After implementing these changes, birth attendants’ practices improved in several key areas included on the checklist.
• Taking of maternal blood pressure went from 0 percent at baseline to 16 percent; postpartum oxytocin use went from 36 to 97 percent; early breastfeeding initiation went from 3 to 64 percent); and checklist use itself increased from 32 to 88 percent.

Conclusions: An adaptive study design approach that includes implementation of a protocol, evaluation, and feedback can lead to the design of better interventions – in this case, a coaching protocol that will improve care for mothers and newborns in a measureable way.

Partners: Bill & Melinda Gates Foundation, World Health Organization, Population Services International, the Governments of India and Uttar Pradesh, The Community Empowerment Lab and The Jawaharlal Nehru Medical College

Study results: Global Health: Science and Practice, The BetterBirth Program in Uttar Pradesh, India:  Pursuing Effective Adoption and Sustained Use of the WHO Safe Childbirth Checklist Through Coaching-Based Implementation, June 2017

Global Health: Science and Practice, Improving Adherence to Essential Birth Practices Using the WHO Safe Childbirth Checklist with Peer Coaching:  Experience from 60 Public Health Facilities in Uttar Pradesh, India, June  2017

In these two studies, the BetterBirth team describes its methods of coaching birth attendants and leaders to adopt the WHO Safe Childbirth Checklist in the BetterBirth randomized control trial in Uttar Pradesh, India. BetterBirth implemented a program of carefully structured coaching that was multilevel, collaborative, and provider-centered, and scaled the checklist coaching to 60 sites. Coaches observed birth attendants’ behavior during 5,971 deliveries. The team found that a structured coaching implementation strategy improves uptake of safe childbirth practices in India—but adherence is not fully sustained once coaching is completed.

What did we learn?

• The results showed that, by the final month of coaching, teams achieved >90% adherence to 35 of 39 essential birth practices in a coach’s presence, versus 7 of 39 practices during the first month.
• Once coaching was withdrawn, however, the average adherence to practices and checklist use dropped 24 percentage points.
• Difficult-to-perform behaviors and those with delayed or theoretical benefits were less likely to be sustained without a coach present.
• Coordination and communication among facility staff, as well as behaviors with an immediate, tangible benefit, showed the greatest improvement.

Conclusions: Coaching may be an important component in implementing the Safe Childbirth Checklist at scale.

Partners: Bill & Melinda Gates Foundation, World Health Organization, Population Services International, the Governments of India and Uttar Pradesh, The Community Empowerment Lab and The Jawaharlal Nehru Medical College

Study results: Trials, Implementation and results of an integrated data quality assurance protocol in a randomized controlled trial in Uttar Pradesh, India, September 2017

The importance of high quality data cannot be understated, given it ultimately becomes the basis for scientific recommendations that inform policy and practice. Yet there exist few published methods or standards for integrating Data Quality Assurance (DQA) into large-scale health systems research trials. This paper describes one highly successful model —  the Data Quality Monitoring and Improvement System in the BetterBirth Trial, a large-scale randomized controlled trial conducted in Uttar Pradesh, India.

What did we learn?

• The model successfully reinforced six dimensions of data quality: data reliability, timeliness, completeness, precision, and integrity.
• The model used resulted in 98.33 percent accuracy across all data-collection activities in the trial.
• All data collection activities demonstrated improvement in accuracy throughout implementation.
• Data collectors demonstrated a statistically significant increase in accuracy throughout consecutive audits.
• The model was successful, despite an increase from 20 to 130 data collectors.

Conclusions:  Given the model’s success,  future efforts should focus on standardizing DQA process in health systems research.