Evaluation of Dakshata, a scale-up WHO SCC and mentoring-based program, for improving quality of intrapartum care in public sector in Rajasthan, India: repeated mixed-methods surveys

Abstract

Background

The Dakshata program in India aims to improve resources, providers’ competence, and accountability in labour wards of public sector secondary care hospitals. Dakshata is based on the WHO Safe Childbirth Checklist coupled with continuous mentoring. In Rajasthan state, an external technical partner trained, mentored and periodically assessed performance; identified local problems, supported solutions and assisted the state in monitoring implementation. We evaluated effectiveness and factors contributing to success and sustainability.

Methods

Using three repeated mixed-methods surveys over an 18-month period, we assessed 24 hospitals that were at different stages of program implementation at evaluation initiation: Group 1, training had started and Group 2, one round of mentoring was complete. Data on recommended evidence-based practices in labour and postnatal wards and in-facility outcomes were collected by directly observing obstetric assessments and childbirth, extracting information from case sheets and registers, and interviewing postnatal women. A theory-driven qualitative assessment covered key domains of efficiency, effectiveness, institutionalization, accountability, sustainability, and scalability. It included in-depth interviews with administrators, mentors, obstetric staff, and officers/mentors from the external partner.

Results

Overall, average adherence to evidence-based practices improved: Group 1, 55 to 72%; and Group 2, 69 to 79%, (for both p < 0.001) from baseline to endline. Significant improvement was noted in several practices in the two groups during admission, childbirth, and within 1 hour of birth but less in postpartum pre-discharge care. We noted a dip in several evidence-based practices in 2nd assessment, but they improved later. The stillbirth rate was reduced: Group 1: 1.5/1000 to 0.2; and Group 2: 2.5 to 1.1 (p < 0.001). In-depth interviews revealed that mentoring with periodic assessments was highly acceptable, efficient means of capacity building, and ensured continuity in skills upgradation. Nurses felt empowered, however, the involvement of doctors was low. The state health administration was highly committed and involved in program management; hospital administration supported the program. The competence, consistency, and support from the technical partner were highly appreciated by the service providers.

Conclusion

The Dakshata program was successful in improving resources and competencies around childbirth. The states with low capacities will require intensive external support for a head start.

Background

High quality care during childbirth is essential for better maternal and newborn outcomes [1, 2]. Although there is substantial evidence of clinical practices positively affecting healthoutcomes, practices are often missed due to a lack of knowledge, skills, resources, comprehensive protocols, monitoring or accountability [2,3,4]. The WHO Safe Childbirth Checklist (SCC) [5] identified crucial moments where a set of evidence-based practices should be performed to ensure high quality services. The Checklist can be used as a reminder tool or job aide, as well as a monitoring tool. The Checklist has been pilot tested and implemented in several countries using a variety of strategies, including training about the use of checklists, additional knowledge and skills training on the clinical practices, coaching and quality improvement approaches [6,7,8,9,10,11,12,13].

In India, the Checklist was pilot tested to assess the acceptability and evaluate change in provider behaviour in the states of Rajasthan, Karnataka, and Uttar Pradesh [7, 9, 10, 14]. The BetterBirth project in Uttar Pradesh built in coaching along with the use of SCC and showed improvement in adherence practices but did not show changes in maternal or newborn mortality [7, 8, 13]. The SCC pilot in 7 districts of Rajasthan (2012–15) demonstrated a decrease in early neonatal mortality [13], and was thus scaled-up into a national program—the Dakshata program [15] where the WHO SCC is adopted into a 29-item facility-based training tool coupled with mentoring. Additionally, the Dakshata program incorporated improving resources and program monitoring led by the state [15].

India has more than 80% hospital deliveries, and a slow declining maternal mortality ratio (MMR) currently at 103 per 100,000 [16] and a stillbirth rate of 4 per 1000 live births [17]. There was a noted improvement in the proportion of institutional deliveries over pasta decade [18, 19], but the quality of childbirth services and referrals did not improve correspondingly [20,21,22]. In recent years, the Government of India has emphasized strengthening infrastructure and quality of obstetric care through several programs [15, 23,24,25]. Amidst all these efforts, there is a need to assess the effect of the Dakshata program, which is one of the first comprehensive quality improvement programs for maternal/newborn care in the country.

Intervention- Dakshata program in Rajasthan

The Dakshata quality improvement program was planned to be implemented in government-run secondary and primary care hospitals. The JHPIEGO-India office supported the Rajasthan state in scaling-up the program to 20 districts (excluding the 7 pilot districts) in a phased manner, covering over 200 hospitals between 2015 and 2019. The program districts and hospitals were jointly selected by the respective state health departments and the JHPIEGO state consultants; the evaluation team did not have any role in this selection or implementation.

The Dakshata program aimed to improve providers’ competence and accountability, while ensuring appropriate resources in labour rooms and wards. The intervention package included: “a) bulk training of 3 days for use of SCC for essential practices in the labour room, b) mentoring and support visit (MSV) package of 3-4 months of on-site pulse training, c) technical support required to ensure availability of resources, and d) technical support to state and country for strategic planning, and monitoring” [15]. Dakshata’s skill building focused on practices from four crucial pause points as in WHO-SCC [5]: Pause point 1 at the time of admission; Pause point 2 just before and during childbirth; Pause point 3 immediately after childbirth (within 1 hour); and Pause point 4 at the time of discharge [5, 15]. Rajasthan government incorporated the SCC in standardised case sheets (medical records) for the labour rooms in the public sector.

JHPIEGO provided intensive support to state through a team of 3–4 state and 10 district program officers. The JHPIEGO state program officers provided 5 days of training to a pool of handpicked government trainers from each program district, regarding clinical as well as pedagogical skills. These government trainers, under the supervision of JHPIEGO program officers, trained the staff (Medical Officers, Staff Nurses, and Auxiliary Nursing Midwives (ANMs)) from the target hospitals in their districts, for 3 days as per the established labour room protocols and standards. The training was provided at the District hospitals in small batches to facilitate training of at least 80% of the eligible staff without disturbing the hospital-level functioning. Post-training, the JHPIEGO program officers (designated Dakshata mentors) provided mentoring through a structured 8-visit package (consolidated to 5 visits in late 2018). Mock-drills and feedback in real settings and repeat training were the standard mentoring approaches. The mentors also got involved in problem-solving and team building at the hospitals. The mentors conducted quarterly periodic assessments using a structured format to monitor the progress in the availability of supplies, adherence to essential practices, and record keeping. They provided feedback to hospitals, districts, and states. Table 1 describes the implementation components in further detail. Typically, in a district, it took 4–6 months to complete 3-day training for the eligible staff; 6–8 months to complete all 8 mentoring visits (4–5 months for 5 mentoring visits) per facility, and more than 3 months of need-based mentoring for lagging practices till planned handover to state.

Table 1 Summary description of intervention in Rajasthan as of January 2019

The 20 districts were covered in 2 phases: phase 1 initiated in 2015–16 in 13 districts and phase 2 initiated in 2017–18 for 7 districts, i.e. 20 in total. At the time evaluation was planned, all the program districts had received/just initiated the intervention, and were at different stages of implementation. For this evaluation, we defined four stages of implementation; 1) no intervention started, 2) 3 days of training completed, 3) all 8 or 5 mentoring package visits completed, and 4) at least three need-based mentoring visits completed, and the handover was planned. The progress of the program implementation between January 2016 and January 2020 is presented in Table 2. The stages were used for stratified sampling, planning evaluation follow-up intervals, and appropriate data interpretations. This Dakshata program evaluation was conducted to 1) evaluate the effect on adherence to maternal and newborn health evidence-based provider practices, and stillbirths; 2) assess the efficiency of the program components, monitoring, and accountability to improvement in the quality of services; and 3) assess the adequacy of external support, institutionalisation in government systems, sustainability and scalability of the program.

Table 2 Status of the Dakshata program in Rajasthan between 2015 and 2020

Methods

Study design

Using a quasi-experimental study design, we conducted 3 repeated mixed-methods cross-sectional surveys without controls. The first assessment was conducted between October 2017 and February 2018, the second assessment between May and September 2018, and the third assessment between May and September 2019 (Fig. 1). Feedback and discussions with stakeholders through March 2020 were also incorporated into the qualitative findings and discussion.

Fig. 1

Timelines for stages of intervention and repeated assessments in the study hospitals in Rajasthan. Legend: Quantitative data from Dholpur is not included in the analysis but qualitative analysed and is presented in the results

Study population

It considered of pregnant women who delivered and their newborns and health staff (doctors and nurses) in the labour rooms of secondary public health hospitals assigned for the Dakshata program. We interviewed JHPIEGO program managers and mentors, and hospital, district and state government administrators responsible for maternal health programs.

Outcome indicators

The evaluation team identified 5 measurable indicators per pause point, i.e. 20 in total, to assess high quality care. The indicators pertaining to the actual management of the complications, although suggested, could not be used as the number of women with complications was small and the case sheets did not provide sufficient detail due to incomplete or improper documentation during the pilot. The in-facility outcomes included the caesarean section rate, stillbirth rate, and referral of mother and newborn.

Sample size and sampling

We estimated the sample size for repeated cross-sectional assessments for cluster (hospital) sampling [26]. (Fig. 2) We used the percentage adherence to identified evidence-based practices from the baseline of program data [10] and computed the sample size for each to detect the desirable change (increase to 80% adherence from baseline) with 80% power and 95% confidence, intra-cluster correlation of 0.1 and cluster size 18 observations. The largest sample size was 30 facilities × 18 observations for a total of 540 observations. For less prevalent practices regarding complications, post-natal care, and in-facility outcomes, we planned for a higher number (based on feasibility) i.e. 50 case sheets, 30 post-natal mother interviews, and 400 labour room register entries per hospital. We randomly sampled 5 districts, stratified for the stage of the program implementation, region, and neonatal mortality rate. At the time of the first assessment, two sampled districts from Rajasthan (Barmer and Jaipur-1) were conducting three days of bulk training (Group 1); two districts (Udaipur and Nagaur) had just completed 8 visits mentoring package (Group 2), and the fifth district (Dholpur) was near phase out. Within each district, we selected six hospitals from within the intervention hospitals in consent with District health officers; these included District hospitals (2), Sub-district hospitals (5), Community health centres (15) and Primary health centres (2). At each repeated assessment, we consecutively included the required number of observations of admissions and childbirths, case sheets, register entries, and interview of post-natal women. For qualitative assessments, we purposively selected two districts based on the stage of program implementation: Dholpur had completed a few need-based visits; and Barmer was completing bulk training. We conducted extensive stakeholder analysis to identify the key personnel, and their roles, that could influence the program. We included all the eligible leaders, administrators, managers, mentors, and hospital in-charges in the sample frame. We interviewed a purposive sample of the available service providers and 5–6 beneficiaries per facility.

Fig. 2

Sample included from 30 facilities during the three assessments in Rajasthan. Legend: We observed 2–3 additional admissions or childbirths per facility to adjust for overall sample size, in case we had fewer numbers from any facility or if we had to discard any case during data cleaning. One facility did not have/ had very less vaginal deliveries within 10 days of our observation in the first assessment. Among the facilities; 13 in time 1, 11 in time 2 and 3 did not have NBSU/SNCU. Quantitative data from Dholpur is not included in the analysis but qualitative analysis is presented in the results

Data collection

We directly observed (non-participatory) service provided to pregnant women at the time of admission, during, and immediately following childbirth in the labour room 24 hours a day consecutively till the sample size was met, and recorded information using observation checklists. We interviewed pregnant women in the post-natal and post-op wards using a structured interview guide. We extracted information from case-sheets post-discharge onto a data extraction checklist. If the number of discharges was short of 50 during our visit, we included remaining from prior days going backward. We conducted a facility survey to assess general infrastructure and resources and extracted information on childbirth outcomes from birth and referral registers in labour rooms or obstetric wards. Data were collected by qualified nurses who were trained intensively for 5 days, under the supervision of team leads. It took about 5 to 15 days to complete data collection per hospital. We pilot-tested all the quantitative data collection tools before use and developed them into an Android-based application for recording data; and built-in logic checks and restrictions to minimize wrong entries. We used Lenovo tablets for collecting data in the field and uploaded them after finishing data collection from each facility. Data were directly saved on the server from where the central research team extracted data on Stata 14.0.

We designed In-depth Interview (IDI) guides, covering key domains of efficiency, effectiveness, institutionalization, accountability, sustainability, and scalability. We developed guides for different cadres in English. We translated them into the local language (Hindi and Telugu) and back-translated to English, corrected errors and pilot tested them. We conducted interviews in the providers’ workplaces. The interviews with district and facility level stakeholders were conducted by two trained researchers under the guidance of the co-investigator, all of whom were social anthropologists in the health sector. Another two trained research staff conducted in-depth interviews of post-partum women. We audio recorded the interviews, if consent was provided, and took handwritten notes. The team transcribed and translated the recordings. Some of the respondents refused a formal interview so we only had informal discussions and noted important aspects of the program. We also took notes in the meetings with JHPIEGO or State government officials, that could be of significance for understanding the implementation process and strategies.

Ethics

We obtained ethics approval from the Indian Institute of Public Health-Hyderabad. We obtained written informed consent before the interviews. As the observations did not require us to interact with the women giving birth, and we did not want to interfere with the routine process of care, thus we did not obtain beneficiary consent, but we did obtain permission from hospital superintendents and health staff.

Analysis

We used STATA 14.0 for data management and analysis. For each clinical practice, we computed the proportion of adherence for a hospital (cluster) and then computed the average of these proportions weighted for monthly delivery load in the hospitals. We also computed average adherence to the five practices for each pause point, and overall average adherence for the 20 practices. We conducted linear regression for a testing average of proportions over time, and Poisons regressions for stillbirths and referrals. We scored the performance of hospitals and gave a score of one if the adherence to practice was 80% or more. A hospital could get a maximum score of 5 for a pause point, and 20 overall. A score of 4 or more in pause points and 14 or more overall was considered satisfactory. We conducted a qualitative analysis based on pre-defined themes and sub-themes, and also identified new emerging themes. We used Atlas-Ti to code the qualitative interviews and notes. We present integrated qualitative results from all three timepoints accounting for implementation progression and effects on the course.

Results

We present results from only Group 1 and 2 sample hospitals, See Additional Table 1 for details. The program was not appropriately implemented in District Dholpur due to several administrative and local facility level determinants as mentioned by the district and state officers, thus we dropped it from the final quantitative analysis in consent with the state, and technical partner. However, there were health system learnings, so we retained it in the qualitative analysis.

The program in Group 1 progressed from bulk training during the first assessment to complete three need-based mentoring, while Group 2 completed one and half years of need-based mentoring and phased out by handover to Government mentors. Amongst the analysed 24 hospitals, the district hospitals noticed an increase in monthly deliveries, while the delivery load reduced in lower facilities, likely due to increased referrals. The availability of staff and caesarean services were consistent. The availability of 5 out of 6 assessed protocols improved, but the availability of protocols for antenatal corticosteroids and preterm labour were still limited-- found in only 12 and 9 of 24 hospitals, respectively. All amenities for maintaining hygiene improved over time in all hospitals. The availability of essential trays improved in Group 1 facilities to 10 or more, while it reduced in Group 2 to 8 or less, due to ongoing renovations in a few facilities. (Additional Table 1 ).

Adherence to evidence-based practices in obstetric care (Table 3).

Table 3 Adherence to evidence-based practices, per pause point and overall, in facilities under study, over time, % (95% C.I)

Pause point 1 (during admission)

Conducting abdominal and vaginal examinations were high and improved (> 95%) further. Maternal blood pressure and foetal heart sound assessment improved from 52 to 83% in Group 1 and 67 to 97% in Group 2. The change in hand hygiene before the vaginal examination was slow initially. After introducing the alcohol-based hand rubs, it improved tremendously (16 to 73% in Group 1 and 49 to 90% in Group 2). Overall, the average performance during admission consistently improved over time, from 63 to 87% (p = < 0.001) in Group 1, and from 77 to 96% (p = 0.001) in Group 2. Among other indicators, monitoring the progress of labour (i.e. assessing the uterine contractions, foetal descent and foetal heart sounds at least twice) improved to above 50% in both groups.

Pause point 2 (during childbirth by vaginal route)

The use of clean cord cut was almost universal throughout the study. Pre-filling the injection oxytocin significantly improved from 67 to 97% and 78 to 94% in Groups 1 and 2. But administering injection oxytocin within 5 minutes remained less than 80% in Group 1 while it improved from 83 to 94% in Group 2 over time due to sustained long-term mentoring. The availability of clean and warm towels improved from 47 to 76% in Group 1. Keeping the bag and mask ready dropped from 85 to 70% in Group 1 while it stayed above 90% in Group 2. The average performance during childbirth improved from 74 to 84% (p = 0.493) in Group 1, and from 86 to 92% (p = 0.017) in Group 2. Among other indicators, induction of labour reduced from 30 to 22% in Group 1 and significantly from 39 to 9% in Group 2. We also noted improvement in pre-heating the warmer, hand washing before delivery, delayed cord cut, mothers’ privacy and counselling for progression of labour (results not shown).

Pause point 3 (within 1 hour of childbirth)

Drying the baby immediately was 85% or above through all three assessments. Within one hour breastfeeding improved significantly from 42 to 79% and 45 to 78%, and the assessment of the mother’s vitals significantly improved from 13 to 67% and 45 to 82% in Groups 1 and 2. The changes in measuring of baby’s weight and assessing the uterine tone were inconsistent; they ended at about 70% or above by the third assessment in both groups. The average performance of evidence-based practices immediately after childbirth improved from 57 to 76% (p = 0.001) in Group 1, and from 69 to 76% (p = 0.002) in Group 2.

Pause point 4 (before discharge)

Measuring mothers’ temperature and counselling for family planning consistently improved in both groups, while counselling for newborn danger signs reduced significantly. The average performance at discharge improved from 24 to 40% (p = 0.070) in Group 1, and from 44 to 51% (p = 0.008) in Group 2.

Overall performance

The overall average of the 20 scored practices, improved significantly in Group 1 to 72% and Group 2 to 79% (p < 0.001 for both). We additionally charted the average performance as per the program implementation status at any assessment, Fig. 3. There was a positive trend association towards improvement in quality of services across all pause-points and overall averages with continuous structured mentoring followed by need-based mentoring.

Fig. 3

Average adherence to practices in pause points and overall concerning the stage of program implementation, % *average of Group 1 time3 and Group 2 time2

Performance scores of facilities

We scored facility performance and found that out of 24 hospitals, the performance improved to satisfactory levels in 22, 19, 6 and 0 hospitals for pause points 1, 2, 3 and 4 respectively, in only 11 hospitals overall (Fig. 4).

Fig. 4

Number of study facilities with score of 4 or more in pause points, and overall score of 14 or more over time, N = 24

Outcomes

Only 3% of case sheets classified any high-risk or complication in the diagnosis; we were not confident of the quality of reporting thus we could not make any quantitative assessments. However, we summarise the noted management of common complications in Additional Table 2. During direct observations of few complications, we noticed improved confidence in staff in stabilising the cases of PIH, managing PPH and resuscitating the child in asphyxia.

Among studied outcomes, the stillbirth rates reduced significantly (p < 0.001) across the groups over one and a half years: in Group 1 from 15 to 2 per 1000 live births and in Group 2 from 25 to 11 (Table 3). During this period the caesarean rates increased in Group 1 facilities but reduced in Group 2 while the mother referral rates remained steady. The newborn referrals to newborn care units significantly decreased in Group 2; the proportion of neonatal jaundice and birth asphyxia reduced, while hypothermia increased, and pre-term was similar over time. As we extracted data from registers, we noted fewer missing data on key indicators by the third assessment.

Qualitative results

We present the qualitative results in three broad sections: 1) Effectiveness of intervention; 2) State’s ownership and program implementation, and; 3) Sustainability. Lastly, we summarise the facilitators and challenges and suggestions for the sustainability and scalability of the program. Table 4 lists stakeholder statements under common themes and sub-themes.

Table 4 Select quotes form the stakeholders’ interviews conducted from Barmer and Dholpur districts of Rajasthan

Table 4 Select quotes from the stakeholders’ interviews conducted in the Barmer and Dholpur districts of Rajasthan.

Effectiveness of intervention

Knowledge and competencies

Staff gained new knowledge about standards and protocols for delivery care and management of complications. The nurses reported that they felt more competent and confident in conducting low-risk vaginal deliveries by following the checklist and protocols. They were able to manage complications such as PPH and newborn resuscitation and provide stabilising care for complications with existing resources, before referral. They learned how to talk to the client and counsel them. During the second assessment, we observed a deeper understanding of the program and clinical management. Nurses were more elaborate about the comprehensive and structured way of providing care to the mother and the newborn, from admission through discharge and the necessity to perform all the required steps for prevention of complications to get desired outcomes. They reported that earlier it took them much longer and a lot of effort to practice all the recommended tasks. But with time, the tasks seemed easier, and the learnt skills were getting embedded in their routine, as per protocols, followed in a systematic manner.

Resource modification

Staff noted considerable improvement in infrastructure, and availability of resources, including protocols, that motivated them and facilitated appropriate service delivery. The periodic assessments by the mentors helped them to identify the gaps and rectify them. Mentors helped in overcoming administrative procedures to avail the resources. They also helped in identifying and instituting local solutions for operational problems.

Perception of quality and service delivery

Earlier the staff perceived quality in terms of infrastructure improvement, availability of relevant trays, sterilisation of instruments, use of gloves and hygiene practices. But after the program, they distinguished quality as improvement in the actual provision of appropriate clinical care and management of complications rather than infrastructure and supplies alone. As told by staff, the emphasis increased on measuring vitals of women, high-risk screening, measuring foetal heart sounds, effectively performing steps in childbirth, administering injection of oxytocin after childbirth, management of post-partum haemorrhage, and use of Magnisium sulphate MgSO4 for prevention and management of eclampsia. By the second program assessment, the staff also emphasised infection control. The nurses mentioned that sterilisation of equipment and use of hypochlorite solution was not done before the Dakshata program. Hand hygiene was mentioned as one of the practices difficult to follow, especially when the load was high. With the introduction of hand rubs, there was some improvement in hand hygiene. The participants accepted that they had stopped unnecessary induction of labour, multiple PV examinations and inappropriate management such as fundal pressure. They also added counselling to the mothers and attendants before discharge as components of quality service delivery.

Effectiveness

Effectiveness of the program was told to be in being able to provide care as per the need of the pregnant women and reducing adverse outcomes. Over time, the staff put more emphasis on physical assessment at the time of admission, filling of partograph, counseling the mother and birth attendant for early identification and tracking of the high-risk cases. The staff talked about the concept of golden minute, newborn care, and resuscitation. A nurse mentioned using balloon tamponade before referring a PPH case which she never did in the past which was also part of training for post-partum hemorrhage management. They also learned and practiced good behaviour with the client.

Outcome of Dakshata program

The stakeholders believed that improved care as per Dakshata program protocols had led to improvement in quality of care and service, reformation in infrastructure and supply, efficient management of complications, reduced referrals, reduced stillbirth, maternal and neonatal deaths. There was an improvement in nurses’ role in decision-making for client care and labour room maintenance.

Accountability

We noted a positive change in the service providers’ attitudes toward providing quality services with a clearer understanding of the benefits associated. The staff believed that the mentoring and periodic assessments contributed to the service providers being aware and proactive towards their duties, with an increased sense of responsibility.

State’s ownership and program implementation

We reviewed the facilitators, challenges; sustainability, and scalability of program components.

Ownership and engagement by state health department

The state health department completely owned the implementation of the program and led it with support from the external agency. The national-level stature of the program drove motivated implementation. Rajasthan was among the first few states to successfully implement the program and much appreciated too. State regularly monitored the program activities and performance, undertook timely decisions to fill in the gaps, provided administrative support and resources, and conducted repeated sensitisation and training for lagging practices. We also noted that after our periodic feedback from assessments, the state took appropriate measures to improve services. The state realized that the success of the program needed to be sustained and improvement should continue. Thus, the state recruited 17 district mentors, and deputed in-service block mentors (nurses/doctors), to continue the program after external support was phased out.

Leadership

The district mentors appreciated the state leadership and attributed the success of the program implementation to the state government health officers. The commitment and accountability at the state-level trickled to the lower-level administrators, hospitals, and service providers too. Some hospital administrators were very proactively involved in bringing changes through the Dakshata program. Some other fulfilled the essential resource requirements through public-private partnerships or donations. On the other hand, a few interviewees mentioned poor leadership at the district and hospital, and the staff struggled to perform well due to inadequate support.

Monitoring

The state implemented a structured monitoring mechanism, later also strengthened by a newly designed android-based software application. The application helped mentors in conducting on-spot periodic assessments, scoring the performance, and giving feedback to the staff or in-charges. It also helped the district and state administrators to track real-time the hospitals’ performance and mentors’ visits. The district administration in Barmer also periodically reviewed the program in district maternal and child health review meetings.

Efficiency of training, mentoring and periodic assessments by partner agency

The hospital administrators and service providers appreciated the short duration and effective mode of training. The Dakshata training was perceived to be different from previous training, including that for skilled birth attendance. Mentoring and supervision visits and periodic assessments were perceived to be the most prominent feature and strong pillars of the Dakshata program. The staff mentioned that mentoring visits helped them internalize the standard practices without having to put a lot of effort into the learning process. The mock drills and the briefing-debriefing exercises helped in improving clinical skills. All the stakeholders admitted that mentoring in the staff’s workplace was more conducive and regular follow-up improved the sense of responsibility and accountability at all levels. Mentoring visits were of particular importance in setting all the infrastructure and resources right.

Motivation, encouragement, awards

The state provided recognition and award (mementos, certificates) to the best performers. A few providers and administrators felt that these measures encouraged and motivated staff and improved accountability. The review of the program in monthly maternal health meetings at the district and state levels also provided opportunities for cross-learning and finding solutions for common challenges.

Sustainability

The stakeholders shared their understanding of factors for the successful sustainability of the program, as well as the threats.

Improved infrastructure and demand supply chain

The labour room infrastructure and demand supply chain significantly improved and were among the key factors for improvement in staff motivation and service delivery. The mentors played the central role in facilitating these changes, and post phase-out the labour room in charge would need to pro-actively take over this role to maintain resources.

Improved competence and quality of service delivery

The evidence-based practices improved, and staff noted the difference in their practices pre- and post-Dakshata. The program encouraged and enabled them to improve the quality of services, and these improvements gave them job satisfaction. They aimed to continue the same practices and improve further with guidance.

Continuous skill upgradation

The stakeholders mentioned that regular skill upgradation training or mentoring were required to sustain quality services and incorporate newer evidence-based improvements to clinical care.

Quality of mentors

All the stakeholders stated dynamic relationship with JHPIEGO mentors; mentoring and periodic assessments made the program much desired. These JHPIEGO mentors were mostly MBBS doctors, while the new government mentors were a mix of trained para-medical staff and nurses and very few doctors. Administrators told that post-phase out, the government mentors have to do a lot to fill in the shoes of earlier mentors. The service providers were skeptical about the performance after shifting from JHPIEGO mentors to government mentors.

Motivation and accountability

The recognition and monetary awards were motivation to continue good work.

Linkage and support from other programs

The providers told that the incentives provided under Janani Suraksha Yojana ensured that the mother stayed at the hospital for at least 48 hours post-delivery, which provided an opportunity to deliver post-natal care. Programs such as Kayakalp helped in instituting sanitation and hygiene; Laqshya had standards for resources and services and provided certification of quality of services in labour rooms and pediatric units.

Challenges and suggestions for effectiveness, accountability and sustainability of the Dakshata program

Community level care and practices

Inadequate antenatal care, high prevalence of severe anaemia, late arrival at the health facility, poor awareness about delivery processes, unhygienic practices, and requests for early intervention or early discharge negatively affected the quality of services. Staff reported referring most of the late arriving complication cases.

Shortage of staff and inadequate resources

amidst high patient load worsened the ability to provide quality services and documentation; and postnatal care was negligible. The state recruitment for 800 nurses was on hold for 3 years due to a court litigation. Despite huge improvements, the gaps in infrastructure and supply chain were still pertinent.

Local political influence and interference

District-level officials were displeased with political interference that resulted in the irrational positioning of humans and other resources. Some service providers used political influence as a means to escape from their daily duties and responsibilities; they were difficult to supervise or discipline.

Unmotivated staff and reluctant doctors

Not all staff were similarly motivated. Most obstetricians and a few medical doctors did not want to participate in the Dakshata program. In their understanding, the program was only for nurses and support staff.

Malpractices

There were certain reports of malpractices, such as doctors referring the patient to his/her private clinic, and staff asking for monetary remuneration for otherwise free services.

Inconsistent support from the administration

District and hospital participants reported inconsistent support from the government for adequate human resources, reinforcement of training, and monitoring. Staff perceived that the government had a poor understanding of the practical problems at the bottom level. Staff felt that the Government would not be able to provide proper follow-up mentoring after the phase-out of JHPIEGO mentors.

Suggestions

We also asked the stakeholders about the suggestions on how to improve the quality of obstetric care and for Dakshata implementation program. The suggestions are compiled below in brief.

• 1) Provide pre-service training and regular refresher training in a systematic manner close to the workplace and update the latest protocols.

• 2) Establish stringent monitoring and feedback system; strengthen periodic assessments through mentors and reviews by administrators in routine MCH meetings.

• 3) Empower and support the new government mentors for efficient mentoring and periodic assessments. Monitor mentoring.

• 4) Ensure minimum human resources as per the load, particularly nurses. Do not rotate staff; commit trained obstetric staff to only obstetric wards or labour rooms.

• 5) Ensure availability of essential equipment and supplies; maintain consistency.

• 6) Adopt a quality improvement approach in line with programs such as Laqshya and Kayakalp; make use of these for the identification of local problems and solutions. Formation of quality circles/ teams supported by the administration may help.

• 7) Establish advisory support and redressal mechanism for technical as well as managerial concerns.

• 8) Strengthen and improve antenatal care and empower the community regarding obstetric care. Target specifically poor performing and deprived communities.

Discussion

In this evaluation of a complex intervention instituted in a public health system of a state with comparatively lower resources and poorer maternal and newborn indicators among states in India [16, 17], we noted significant improvements in the quality of care provided and acceptability of the Dakshata program by staff and implementers. We adopted the most suitable evaluation design in the provided time frame, ensuring that we cover all the stages of the intervention to understand short-term as well as mid-term changes.

Overall, of the studied 20 key practices, four were at > 90%, six at 80–90%, seven at 50–80%, and three less than 50%. The evidence from other states in India also suggests that not all the practices were adopted with the same rigour (pause points 2 and 3 were adapted better), and sustainability of the improved practices was challenging [8, 10, 11, 15, 27]. In our evaluation, a few practices from pause points 1, 2 and 3 were adopted quickly after the program. There seemed to be a felt need for knowledge and skills and a systematic approach to identifying and managing complications [4]. The program also empowered nurses and facilitated better teamwork and support from the doctors, which contributed to decision-making and better service delivery by nurses [4, 9, 28]. Many other practices improved slowly with continuous reinforcement and by ensuring adequate resources [2, 4, 28]. On the other hand, a few practices were resistant to change due to personal or health system issues [29]. These were mostly pertaining to post-partum and postnatal care [8, 10] where the staff did not directly feel responsible for post-hospital outcomes. Hand hygiene and postnatal counselling were amongst the difficult practices to improve; similar to other evaluations of quality improvement for intrapartum care [9, 10, 27]. As the adoption of evidence-based practices happens at its own pace, for an effective mentoring program, it is important to understand the staff’s priorities, assigned importance to certain practices, the resistance, and existing national standards and protocols; and plan for a long term adaptable mentoring curriculum [6].

Among the two large trials in India, in the BetterBirth trial of Uttar Pradesh, there was a sustained improvement during the 8 months of coaching but noted a drop after 4 months since the coaching ceased; there was no change in maternal or neonatal mortality [8]. We also noted in Dakshata the tendency for adherence to regress over time. In our assessment in Rajasthan, mentoring continued for over one-and-half to 2 years which improved or sustained certain practices led to higher caesarean section rates and a higher ratio of referrals in Group 1 representing the initial program stages. With increased resources and confidence gained after the sustained program, the referrals could decrease and stabilise as in Group 2 representing advanced program stages. The stillbirth rates decreased in both the groups; and were close to the state average of 6/1000 live births [17]. Global evidence on the use of WHO SCC shows a mixed impact on maternal or perinatal mortality [7, 8, 12, 13, 29].

The technical partner adopted a scientific approach to pilot testing and scaling up, using an adaptive and iterative prototyping approach and sharing information and feedback with key stakeholders [30]. The key to Dakshata’s success was client-oriented and service provider-centric mentoring complemented by periodic assessments by the technical partner which is reproducible by government mentors. The qualification and capabilities of the mentors, and the support and resources provided to them, especially in a hierarchy-driven environment such as in India, can be the determinants of the success or failure of the program [6]. The support of hospital leadership [6, 31, 32], adequate resources, and availability of staff at the time of mentoring are essential for successful mentoring. Dakshata program, unlike other SCC-based interventions, ensured better preparedness and resource availability before initiating capacity building. Frequent transfers and rotation of staff posed a necessity for repeat training and a continuous mentoring program [6, 33]. The program requires greater emphasis on the effective use of data from periodic assessments. Any mentoring or coaching program would require strong commitment; pro-active monitoring of the mentors; feedback and supportive management from the district and state administration [6, 9, 18, 20]. The state of Rajasthan was highly committed and involved in the planning, implementation, and monitoring of the program, and was the mainstay of its success. A similar commitment and effort shall have to be continued to further improve the practices [6]. For sustaining the current mentoring model, the state requires innovative solutions to embed mentoring culture within the health system; and devise local, programmatic and policy solutions for integrated quality improvement initiatives and programs in the country [1, 3].

The methods considerations and limitations are discussed in detail elsewhere (submitted); however, a few key limitations are worth noting. Some important indicators could not be included as we were not confident of their documentation. There is the possibility of inter-observer bias although we provided repeated training and supervision. There could be the Hawthorne effect [34] but non-participatory observations in substantial numbers were the best methodology for assessing practices. We interviewed postnatal women after 24 hours of birth to be able to capture most post-natal women who are otherwise likely to leave early against medical advice. By doing so, we were likely to under-report pause 4 practices on counselling that were to be provided at the time of discharge under the program. Evaluation with controls would have been a better study design; however, all the districts had some Dakshata-related intervention underway when the evaluation commenced. Another observation after phase-out would have provided better evidence on sustainability.

Conclusion

The adherence to practices recommended under the Dakshata program improved to high levels in admission and just before birth (pause point 1 and 2), improved to moderate levels in the first hour postpartum (pause point 3) but stayed poor at faciality discharge (pause point 4) even after one to two years of mentoring. Importantly, the stillbirth rates reduced significantly in both groups over 18 months, significantly better in Group 1 than Group 2. The structured mentoring followed by need-based mentoring with monitoring and administrative support from the state was a successful implementation model. The intensive support from a technical partner (Jhpiego) was essential for resource-constrained states to give them a strong head start. For long-term sustainability, policies are required for building mentoring culture and integrating quality improvement programs.