Fidelity and acceptability of implementation strategies developed for adherence to a clinical pathway for screening, assessment and management of anxiety and depression in adults with cancer

Abstract

Background

Implementation strategies are crucial to facilitate implementation success. To prepare and support implementation of a clinical pathway for screening, assessment and management of anxiety and depression in cancer patients (the ADAPT CP), six broad categories of implementation strategies; (1) Awareness campaigns, (2) Champions, (3) Education, (4) Academic Detailing and Support, (5) Reporting, (6) Technological Support, were developed. The aim of this paper is to describe the fidelity and acceptability of six categories of implementation strategies and any subsequent changes/adaptations made to those strategies.

Methods

The ADAPT CP was implemented in twelve cancer services in NSW, Australia, as part of a cluster randomised controlled trial of core versus enhanced implementation strategies. Fidelity to and any subsequent changes to the delivery of the planned six categories of implementation strategies were captured using the ADAPT contact log, which recorded the contacts made between the ADAPT research team and services, engagement meetings and monthly meetings. To explore acceptability and awareness/engagement with the implementation strategies, interviews with a purposively selected staff sample across both study arms were held prior to implementation (T0), six months into implementation (T1) and at the end of the 12-month implementation period (T2). Interviews were thematically analysed across the six categories of strategies.

Results

Delivery of all six categories of implementation strategies as planned was moderated by service context and resources and staff engagement. As such, for some implementation strategies, subsequent changes or adaptations to the content, mode of delivery, frequency and duration such as abbreviated training sessions, were made to optimise fidelity to and engagement with the strategies. Most strategies were perceived to be acceptable by service staff. Use of strategies prior to implementation of the ADAPT CP such as the engagement meetings and training sessions, positively impacted on ownership and preparedness to implement the ADAPT CP. Furthermore, ongoing support such as provision of additional training or monthly meetings facilitated increased awareness and engagement with the ADAPT program.

Conclusion

Flexibility in delivering implementation strategies, and ensuring staff engagement with, and acceptability of those strategies, can support implementation of interventions within healthcare settings.

Trial registration

The ADAPT CRCT was registered prospectively with the ANZCTR on 22/3/2017. Trial ID ACTRN12617000411347. https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=372486&isReview=true

Textbox 1. Contributions to the literature
• Deliberate selection and delivery of relevant implementation strategies is important for implementation success. Clear documentation of the delivery of strategies as planned, and understanding of acceptability of strategies is crucial to informing appropriate selection and design of implementation strategies.
• Whilst we found that implementation strategies were delivered and acceptable, delivery of implementation strategies as planned varied, and subsequent adaptations or changes were made to ensure fit to local context and to optimise delivery and engagement with strategies.
• These findings contribute to the limited literature, including the acceptability and how they are adapted in real-world trials.

Introduction

Successful implementation of health interventions requires the delivery of relevant implementation strategies, defined as the methods or techniques developed to facilitate adherence to, and adoption and sustainability of an intervention [1, 2]. Many strategies have been developed to address barriers to implementation of health interventions and increase engagement of staff with these interventions [3, 4]. Deliberate selection and delivery of relevant implementation strategies is important to ensure implementation success [5, 6]. However, fidelity and acceptability of implementation strategies are under-reported [7]. Lack of thorough documentation about the delivery of strategies may limit our understanding of implementation success and hinder refinement of future efforts [2, 8, 7].

We selected a range of strategies to support implementation of a clinical pathway for identification and management of anxiety and depression in adult cancer patients (the ADAPT CP) [9]. The ADAPT CP, described in detail elsewhere [9, 10], involves iterative screening, triage to one of five steps (based on severity of anxiety and/or depression), and referral to interventions appropriate to each step [11]. The ADAPT CP also provides recommendations on staff responsibilities, timing and type of intervention to address each severity step [11]. An online Portal was developed to operationalise the ADAPT CP [12].

The ADAPT CP was implemented across twelve oncology services in New South Wales (NSW), Australia as part of a cluster randomised controlled trial (CRCT) [9]. The ADAPT CRCT aimed to evaluate two “doses” of implementation support (core versus enhanced) to facilitate adherence to the ADAPT CP [9]. The ADAPT CRCT Working Group, comprised of experts in the fields of psycho-oncology, oncology, implementation science and consumers, developed the suite of implementation strategies, informed by a systematic review of barriers to implementation [13], a qualitative analysis of barriers to implementation of the ADAPT CP [14] and local implementation practices.

Prior to implementation of the ADAPT CRCT, our team defined measures of implementation success according to Proctor et al’s [15] Implementation Outcomes framework. Fidelity was defined as “the degree to which each service received the implementation strategies as planned within their randomization” [16]. Acceptability of the implementation strategies was defined as “cancer staff perceptions of the implementation strategies as agreeable, palatable or satisfactory” [16].

This paper describes fidelity and acceptability of the six categories of implementation strategies. Where applicable, we also identified reasons for differences in the extent to which services engaged with the strategies as planned and any subsequent changes/adaptation made to the strategies.

Method

Study context

The ADAPT CRCT study procedure is reported in full elsewhere [9]. Briefly, services were randomised to two levels of implementation support (core versus enhanced). Prior to implementation, each service participated in 6-8 Engagement Meetings at which one or more local champions were identified and a multidisciplinary lead team was formed to tailor the ADAPT CP to local service resources and requirements [17]. Services then progressed to the “Go-Live” stage to implement the ADAPT CP for 12-months. See Additional file 1 for Consort Flow Diagram.

The study was approved by the Sydney local Health District Human Research Ethics Committee, Protocol X16-0378 HREC/16/RPAH/522.

Implementation strategies

Six categories of implementation strategies were designed that aimed to prepare services for implementation and provide support throughout the 12-month implementation period: (1) Awareness campaigns, (2) Champions, (3) Education, (4) Academic Detailing and Support, (5) Reporting, and (6) Technological Support (Table 1).

Table 1 Description of core and enhanced implementation strategies^a

Core strategies were consistent with usual practice for implementation of new guidelines or clinical pathways in the Australian context [18, 19] and delivered to both study arms. Strategies recommended depend on the perceived barriers and typically involve education materials and training to increase knowledge or understanding of the intervention being implemented, endorsing clinical champions to encourage other clinicians to use guidelines and promotion or communication campaigns such as posters to increase awareness [19]. Services randomised to the enhanced strategy arm received additional strategies and more active and sustained engagement with the ADAPT team throughout the 12-month implementation period: specifically, additional awareness campaigns and monthly face-to-face Lead Team meetings with the ADAPT Research team to discuss monthly reports and progress and identify and resolve any emerging issues. These ‘enhanced’ strategies were underpinned by Weiner’s (2009) [20] theory of organizational readiness for change, where we provided opportunities for the ADAPT team to facilitate services’ focus on what was working well and what wasn’t and to make adaptations and offer assistance where needed. A degree of flexibility and tailoring of implementation strategy delivery was permitted, to fit service scheduling requirements and enable content of awareness campaigns, meetings and training to be tailored to service or staff needs [16].

Data collection: implementation strategy fidelity and engagement/adaptations

Engagement Meetings were recorded to capture discussion, decisions around tailoring of the ADAPT CP, and planned delivery of the implementation strategies.

An ADAPT Contact Log captured contacts made between services and the ADAPT team, including those related to scheduling and delivery of implementation strategies. Staff attendance at meetings where implementation strategies were delivered was noted, as were discussions about scheduling and delivery of the strategies. Ad hoc contacts related to the implementation strategies (e.g., modification of posters, requests for additional training) were also recorded by the research team.

Monthly Meetings held through the 12-month implementation period for sites randomised to the enhanced strategy arm were recorded, capturing discussions, decisions and resolution of any emerging issues with delivery of implementation strategies.

The number of newsletters sent to staff, opened and number of clicks on the newsletter links in the enhanced strategy arm services were recorded in the Mailchimp email platform.

Data collection: implementation strategy acceptability and engagement

Staff Interviews: A subset of staff from each service, purposively selected to include members of the lead team, non-lead team and multi-disciplinary representation, to ensure representation of views of staff with varying degrees of interaction with ADAPT and from both study arms, were invited to participate in a semi-structured telephone interview at three time points: prior to ADAPT CP implementation (T0), 6-months into implementation (T1) and at the end of the 12-month implementation period (T2). An email invitation was sent to the identified service staff. The same recruitment method was used across all services, regardless of trial arm. The interview guide was developed to explore, amongst other issues, acceptability and awareness/engagement with the implementation strategies that supported ADAPT CP implementation. Interviews were conducted by three female trained qualitative researchers who had no direct involvement with ADAPT CP implementation. Interviews were audio-recorded and transcribed verbatim.

Data analysis

Fidelity and adaptation/engagement data: Dates (e.g., date of delivering or displaying the awareness campaigns), timing (e.g., length of training sessions) and attendances (e.g., who attended) which were extracted from the ADAPT contact log and descriptively summarised in Microsoft Excel. Data related to implementation strategies from the engagement meetings, ADAPT contact log and monthly meetings were extracted and content analysed to identify subsequent changes/adaptations made to the strategies and reasons for differences between implementation strategies planned and delivered.

Acceptability and awareness/engagement data: Staff interviews across the three timepoints were thematically analysed [21] using NVivo12 software [22]. Two researchers (MH, SH) initially coded six transcripts from each timepoint to develop a draft coding framework, which was discussed with a third researcher (PB) and refined. All interviews were then independently coded line-by-line, with differences resolved through consensus. Similarities and differences in coding were examined to develop initial themes and reviewed to identify higher order themes. Themes and quotes were mapped to randomisation arm and timepoint, to enable any differences according to randomisation and implementation stage to be identified, as per the Framework Analysis method [23].

To explore the concepts of fidelity further, we identified the conceptual framework for implementation fidelity by Carroll et al., (2007) [24]. Carroll et al., (2007)’s conceptual framework of implementation fidelity, depicts four main components of fidelity (details of the content, coverage, frequency and duration) and four domains that act as moderators to fidelity (complexity of intervention, implementation strategies, quality of delivery and participant responsiveness) [24]. This framework was used in our discussion to illustrate the complexity of health services implementation.

Results

All six implementation strategies were delivered to the twelve services. However, the extent to which services received the relevant strategies as planned differed within core and enhanced services. Adaptations to strategies were made to facilitate delivery of implementation strategies. Two hundred and fifty-three interviews were conducted with 122 different staff members over three timepoints at T0 (n=88), T1, (n= 89) and T2 (n=76). See Table 2 for interview participant characteristics. There were 167 staff interviews analysed in the enhanced arm and 86 staff interviews in the core arm. The acceptance and attrition rates for staff who participated in interviews have been described elsewhere [25]. In brief, the response rates across all three time points were 64% (70% at T0, 66% at T1, and 57% at T2). There were 87 staff (71%) who participated in multiple interviews. Staff perceptions of the strategies generally remained stable over time and between randomisation arm, however, any changes or differences were noted. Quotes are identified by study arm: enhanced (E) versus core (C); profession, oncology service (S1-12), personal ID and time of assessment (T0-2). Additional quotes are provided in Additional file 2.

Table 2 Staff interview participants: demographic and professional characteristics^a

Fidelity and adaptations to awareness campaigns

Roadshows

At least one in-person Roadshow was delivered to each service to inform staff about the ADAPT CP (Mean=3 per service, range 1-5) to raise awareness and increase engagement with the ADAPT CRCT. Adaptations to the number, duration and timing of Roadshows were made based on availability of staff and space within cancer services. Roadshows were held later than the intended eight weeks prior to implementation, due to: non-availability of forums/meetings till later (n=2), intended meeting falling on a holiday (n=1); or compressing the whole engagement process to enable Go-Live before year-end (n=1). In total, 271 staff across the 12 services attended a Roadshow (Mean=23 per service) (Table 3).

Table 3 Roadshow delivery and attendance

For many participants, the Roadshow was their first exposure to the ADAPT CP; those who attended had positive perceptions of this strategy, feeling it gave staff a “better understanding of what [the ADAPT CP]’s actually all about” (E_NURS_S12P02T0).

Some participants suggested holding the Roadshow or a refresher session closer to Go-Live, or holding more Roadshows to maximise staff attendance and ensure information was retained.

Reminder strategies:

Posters

All services received two sets of posters, the first (ADAPT Is Coming) displayed prior to go-live, the second (ADAPT Support information) displayed at Go-live (on the first day of implementation). Enhanced strategy arm services received an additional ADAPT is LIVE poster at Go-live for one month, and two refresh/replacement of the ADAPT Support poster at 4 and 8 months into implementation. Of the 48 posters displayed across all services, the majority (n=34) were displayed as planned; some were displayed earlier or delayed. Reasons for delay included governance or staff approval, requests for additional information on the poster, the Champion forgetting, and a decision to delay display until the ADAPT Team site visit.

Further adaptations to the poster strategy were noted in some services. Six services from the enhanced study arm requested additional patient-facing posters, or changes to the posters to fit local context (i.e., e-posters for display on electronic poster boards). Two services requested further communication emails to staff or patients to increase awareness.

Newsletters

Five e-newsletters, tailored to each of the eight enhanced strategy arm services, were sent to service staff across the implementation period, containing information on service progress, and reminders about the ADAPT CP and Portal. Most newsletters (n=37/40) were emailed to service staff within one week of their intended delivery date. Service staff (typically the Champion) were asked to review and approve newsletters prior to dissemination; delays or failure to respond meant some newsletters were delayed.

Across the 1036 newsletters sent, only 239 (24%) of newsletters received were opened. Across all services, 48% of staff opened at least one newsletter. In addition to written content, newsletters contained on average one link (range 0-2) to additional information, including online health professional training. However, clicks on newsletter links were very low: 7 clicks across all newsletters received (0.6% of maximum possible clicks). Services were encouraged to tailor newsletter content to their service; there was generally low engagement with this tailoring. See Fig. 1.

Fig. 1

Percentage of newsletters opened over time across all enhanced strategy arm services

Go-live email correspondence (See Table 1, 1.3)

The ADAPT team sent a planned email, the “Go-Live Email Correspondence” which contained essential information about the ADAPT CP and Portal, the date of CRCT implementation and what staff needed to do in preparation for this, for site Champions to disseminate to staff, as planned in the eight core services.

Some participants reported having seen ADAPT posters and email correspondence, whilst others had not. Whilst some commented that these strategies acted as “a constant reminder that… the program is there” (NURS_S03P03T0) and helped to increase awareness particularly amongst non-lead team staff, others felt these were easily overlooked and thus not effective.

Participants commented that briefly mentioning ADAPT during existing meetings may be more effective to raise awareness amongst wider service staff. Overall, participants commented that the combined awareness strategies (Roadshows, posters, emails, newsletters) helped to increase wider service staff awareness prior to Go-Live.

“with the posters and the emails, and the face-to-face sessions, it was really… capturing everybody’s attention… so when it went live… everybody was on board with it… we…registered lots of patients in the first few weeks which was really good.” (E_AH_S06P03T2)

Fidelity to champion correspondence

Champions at each service were sent six Portal “Tips and Tricks” emails, each containing a “Portal Tip” to be circulated by Champions to registered ADAPT Portal Users at 4, 8, 12, 16, 28 and 40 weeks into implementation. Most were disseminated as planned (62/72), the remaining were sent earlier (3/72) or later (7/72). We were unable to capture how many Portal Users received the “Portal Tip” email.

Most participants were aware who was nominated as the ADAPT champion at their service, believing that Champions played a critical role in driving service change, and in relation to ADAPT. More generally, these champions increased awareness and engagement of service staff including senior management and ensured responsibility for implementation was not diffused.

“having key players… who check in and… rally a team… can be dependent if any issues come up… and just having someone who has relationships with the key players who can move things forward, I think that’s probably pretty critical.” (E_PSYCH_S09P03T0)

Participants saw the ADAPT champion as a ‘go-to’ person who could train and upskill staff, communicate necessary information about the ADAPT CP and Portal, answer questions and troubleshoot issues as they arose. This eased the perceived burden of implementation, as one participant said:

“I think yeah, the champions were key in that finding solutions to the implementation and being committed to doing it. And making sure everybody else who needed to do something, did their thing.” (C_MED_S01P09T2)

Attributes of a successful champion

Participants commented that the champion’s role, authority and workload capacity could impact the success of this strategy.

Role

Clinical staff with regular patient contact were seen as well-placed to champion, tailor and enact the CP. They had well-established collegial relationships with staff implementing the CP, making them approachable if staff required assistance.

“[champion]’s got a lot of clinical experience and especially in psycho-oncology…she’s very good at seeing how systems work and what the patients need clinically and trying to modify or trying to make ADAPT fit with the care, with the needs of the service so… I think she’d be good.” (C_NURS_S11P04T0)

Having a champion who held a clinical role also meant this role could continue beyond the 12-month implementation period. Where the champion role was taken on by, for example, a clinical trials staff member there were concerns this role would not continue beyond the implementation period.

Authority

Participants commented on the importance of service change being championed or supported by senior management staff. At one service participants commented that champions “lacked the power” to secure staff engagement, which made this strategy less effective.

Workload Capacity

Participants acknowledged that championing ADAPT would require additional time and energy on top of the Champion’s existing clinical load. Hence the success of this strategy depended upon the champion’s ability to dedicate time to this role. For example, where funding was secured for a dedicated part-time role, the ADAPT CP implementation could be successfully prioritised.

“I think that was the key because I think having someone dedicated to that role, means actually having it done. Whereas I think… if they get that added to their role as part of their current role… it can be quite difficult to add that to their workload.” (C_NURS_S12P01T2)

On the flip-side, however, staff in the same service commented that having a designated champion limited wider service staff engagement.

“having the champion also, to have it as her focus role is a positive thing, because I guess in that regard a number of us have stepped back thinking, well, that’s her role now.” (C_AH_S12P06T0)

Fidelity and changes to education

Clinical Pathway Overview Training

All services received CP Overview Training. Ideally, cancer service staff co-led this training with the ADAPT Program manager to increase ownership and staff engagement. Psychosocial staff co-led training at 9 services (Table 4). Delivery of training was delayed at some services. Changes to planned delivery of training was due to: service awaiting governance approval, delayed nursing staff approval of screening and scheduling issues.

Table 4 Attendance and duration of the clinical pathway, portal overview, portal user and super user training¹

Portal overview training

Twenty Portal Overview Training sessions of on average 24 minutes (range 5-135 minutes) were held. Adaptations to this strategy included, providing abbreviated 5-minute, 2-3 slide Portal Overview presentation during their CP Overview Training upon site request instead of the training session (n=6 services) (Table 4). Reasons for this were due to difficulty finding sufficient time for staff to attend.

Portal user training

A total of 79 Portal User training sessions (average 43 minutes) were held (average 7 trainings per service, range 0-14 trainings). Most (n=59) were one-on-one, the rest (n=20) were in small groups (average group-size, n=3, range 2-9). A total of 121 Portal User Training attendances were recorded (Mean=10 attendances per service). All portal site administrators (n=12) at participating cancer services received the super user training as planned.

Training sessions participants reported these as useful, succinct, and professionally delivered. Participants particularly appreciated the one-on-one nature of the Portal User Training, which was interactive (a “learn by doing” approach; E_PSYCH_S08P04T1), tailored to their role and allowed them to ask questions.

Fidelity in delivery of education relies on capacity of services to facilitate staff attendance, which is dependent on the needs of the clinical area on a given day and time. Flexibility and adaptability in meeting needs of part-time and shift-based workforce, and staff turnover is also required. Participants commented that education needed to accommodate this to optimise effectiveness. Advance scheduling, flexible and adaptable delivery and keeping sessions short helped to maximise attendance as it enabled staff to fit this in alongside their clinical loads.

“we’ve got a very small permanent staff base at the moment so [training] will need to be rolled out again as we recruit… repeating them keeping in mind that the, kind of, issues associated with a rotating roster and actually capturing people.” (E_NURS_S09P02T0).

At T1 and T2, some participants commented that they received additional training from the ADAPT Team during the implementation period (e.g., staff new to the service, returned from secondment/ leave, or missed initial training), which was viewed positively.

“nothing was too much trouble. Every person was trained…any new staff that we had new training needs… they did that, and were really very flexible, very accommodating… I guess without that…everyone would have gone, no, this is too hard.” (E_NURS_S02P08T2)

Impact of education on preparedness

Most staff felt they had been provided with adequate information about the CP and Portal to start implementing the ADAPT CP at their service.

At T0, participants felt they needed to use the ADAPT CP and Portal ‘in practice’ to solidify the knowledge and skills learnt during training. Whilst they thought implementation would initially be challenging, they expressed faith that it would get easier over time and with practice, and that problems which arose would be overcome.

“there’s probably going to be things that we need to iron out and problem solve as they crop up… we haven’t got it perfect… but I think we’re prepared enough to start actioning it.” (E_PSYCH_S07P02T0)

“once I start playing around with the portal…registering patients… really getting stuck into it, it’s going to be a lot easier” (C_ADMIN_S01P05T0)

Training was perceived to be most effective when delivered close to staff enactment of the CP and Portal. Time-lags between training and first use, or intermittent use of the CP and Portal, were barriers to preparedness as it meant information was less likely to be retained and staff had to “re-learn things again” (C_NURS_S01P01T2). Refresher training just prior to Go-Live, or throughout the 12-month implementation period when staff were enacting the CP, were suggested.

Using the ADAPT CP and Portal ‘in practice’, contacting the ADAPT Support Service, and referring to the User Guides and Quick Guides helped to overcome issues and increased preparedness.

Fidelity to academic detailing and support

Champions were provided with a written (de-identified) report summarising amongst other issues that were explored during the staff interviews, staff acceptability of the ADAPT CP implementation (T0, T1, T2). Reports were sent to the ADAPT Champion at all services, as planned, after completion of staff interviews at each timepoint.

Engagement meetings

All services participated in Engagement Meetings to prepare for the implementation of the ADAPT CP, and challenges and benefits of convening a local Lead team regularly with the ADAPT team were evident. Across the results, we noted a lack of consistency in using the Engagement Meetings strategy (6-8 planned meetings spaced about 2 weeks apart). Reasons for this included non-availability of Lead Team staff, and service desire to move efficiently through decision-making. For example, at one service, delays in forming a Lead Team and securing multidisciplinary attendance stalled decision-making and led to a protracted Engagement Phase (n=12 meetings across 41 weeks) (Table 5). Meetings were in-person (n=55, 69%) or via teleconference (n=25, 31%) and involved 402 attendees (Mean=34 per service), the majority (n=370, 92%) lead team members. The time from first Engagement Meeting to CRCT implementation/Go-Live was, on average, 25 weeks (range 10-41 weeks).

Table 5 Number, duration and attendances to the engagement meetings

On a more positive note, those who attended appreciated their structured nature of having clear agenda facilitated focused decision-making, were succinct, and had a clear purpose.

“they were really quick and they ran on-time… straight to the point… there was a purpose behind them and they answered the questions, and yeah, the agenda was set up really well.” (C_PSYCH_S03P03T0)

Having time between meetings enabled tailoring decisions made during meetings to be discussed outside meetings and later finalised or revised. However, some participants noted too many meetings, or too much time spent during meetings revising tailoring decisions made previously, which ensured preparedness but was perceived as “overkill”. These participants suggested that meetings be condensed, for example by minimising the time spent reviewing tailoring decisions or having fewer meetings that lasted longer.

At one service, adaptations to meeting mode (i.e., held via email) at the service’s request, which reduced burden and enabled them to fit within staff workload/flow.

“[ADAPT Program Manager] tailored it to suit us a little better by disbanding the actual telehealth meetings and actually doing a lot of it via email, which was really good… [engagement meetings] were only an hour but that was a big, big drain on trying to do your work and getting the meeting and dedicating that time. A lot of information very quickly, but at least it was backed up with emails and sort of discussion that way.” (E_NURS_S02P05T0)

Impact of engagement meetings on preparedness and ownership

Having a series of meetings prior to Go-live increased staff awareness of ADAPT and ensured staff from different disciplines were “on the same page” (E_PSYCH_S07P02T0) about how the CP would be enacted at their service and what was required of them.

“we just managed to build that momentum through those meetings so that the key people, the psychologists, the clinical trial staff, senior nursing staff in the clinic and doctors all had enough awareness, that meant when it went live, it was fresh, and they were really able to capture people.” (E_MED_S06P02T0)

However, one participant commented that staff roles/responsibilities needed to be more clearly specified from the earliest engagement meetings to maximise engagement.

“if you were slightly ignorant of what’s going on and not quite clear about what you are being asked to do, there’s sort of a sense of look, I’m pretty busy anyway, and if this means extra work for me, and, you know, my team, well, then maybe I won’t do it.” (E_MED_S06P02T0)

Participants described working collaboratively with the ADAPT team during the engagement meetings to tailor the CP to their site. The ability for staff to have input into their ADAPT roles also promoted a sense of ownership and ensured roles fit their skills, experience and capacity.

In contrast, nursing staff at one site felt they had limited input during the meetings and were concerned that their ADAPT roles had the potential to exceed their scope of clinical practice in delivering psychosocial care.

“I know in nursing we do deal with… mental health and that, but…in terms of triaging it feels like it is, you know, we’re not trained psychologists and even though a patient may be at a level two in their survey it feels like it could lead to a lot more.” (E_NURS_S04P03T0)

Monthly meetings (enhanced strategy arm services)

Monthly meetings with the eight enhanced strategy arm services were scheduled in advance (n=96); timing and mode of meetings was adapted to maximise lead team attendance. Eighty-four monthly meetings of on average 34 minutes, were held (on average 10 meetings per service) (Table 6). Meetings were held in-person (n=41, 49%), online (n=35, 42%) or via teleconference (n=8, 10%). Reasons for discrepancies in the number of meetings delivered (n=84 compared to 96 planned) included cancellation upon service request and lead team unavailability (e.g., leave, holiday period).

Table 6 Number, duration and attendances to the monthly meetings

Participants from Enhanced services found the monthly meetings valuable; they gave staff a chance to review ADAPT progress at their site, identify emergent implementation barriers and brainstorm possible solutions. Discussing site progress throughout the implementation period also helped to maintain staff engagement and facilitated team cohesion.

“keeping people updated it creates a bit of buy in as well so people remain interested in, enthusiastic and involved” (E_AH_S06P03T2)

“we always use that meeting as a forum for bring up problems and getting solutions of those problems” (E_NURS_S02P08T1)

Staff were positive about the ADAPT Team facilitators, who drew on previous implementation experience to help services come up with solutions to identified barriers.

“[the ADAPT Team] tried to really brainstorm, specifically what was going on at our local site. They didn’t pass judgement. They weren’t saying you’ve got to do better or anything like that… trying to facilitate us coming up with the answers which I thought was excellent.” (E_AH_S10P07T2)

However, some staff commented that meetings were difficult to attend for example due to competing clinical demands, shift work, or that there was inadequate representation (particularly of senior management staff) to optimise effectiveness.

“it’s all very well that we’d come up with any ideas of what can happen, what could happen… but unless there’s higher management there to action it … it makes it really difficult” (E_AH_S06P03T2)

Participants from Core services reported having informal conversations about ADAPT with other staff, adding short discussions about ADAPT to existing meetings (e.g. team or steering committee meetings) or trouble-shooting issues individually as they arose.

A few Core participants commented that regular meetings throughout implementation would have helped to identify and address emergent implementation barriers and maintain staff engagement – which were otherwise difficult to raise.

“[meetings would] open up a dialogue about what’s working, what’s not working, what we can do differently, what we can do better” (C_PSYCH_S11P01T1)

Fidelity to reports

Twenty-two staff across all services generated at least one report in the ADAPT Portal (Mean=2 staff per service, range = 0-6 staff); most were local Champions or Portal Site Administrators (n=13, 59%). Across all services 525 reports were generated, most frequently the Screening Report (n=190, 37%), which provided a summary of patient screening events in the reporting period and the Planned Notifications report (n=108, 21%), which showed upcoming notifications to be sent to cancer service staff.

Content of monthly reports were adapted upon service requests. Few (n=3) enhanced strategy arm sites requested additional information for inclusion in these reports, to enable them to focus on specific data of interest, such as patient registrations by tumour streams or clinical departments. The monthly reports enabled staff to reflect on their service’s progress and identify and resolve emergent issues. They also helped to maintain staff engagement by “keeping the program on people’s radar” (C_MED_S11P05T1). At one site, reports showed that no referrals to psychosocial support had been made despite a number of patients screening high prompting staff about the lack of psychosocial supports available, and subsequently new referral pathways were identified. Several staff commented that the Portal-generated patient reports helped them to identify and understand patients’ issues, which facilitated triage conversations and treatment.

“because some of the people that were screening, I was already seeing or…were getting referrals for from other sources. So, the information on their screening…was useful for me when I was seeing them.” (E_PSYCH_S02P01T1)

Fidelity of technical support

There were approximately 485 contacts between service staff, patients and the ADAPT Team during the engagement and implementation periods. Approximately 37% of contacts (~ 179 contacts) related to registering patient or staff on the ADAPT Portal and log-in issues followed by technical issues related to online screening (approximately 99 contacts).

Most participants reported being aware of the ADAPT Support Service and an ability to access this at any time. Some participants reported having used the service, whilst for others there had been no need. Participants who accessed the service were positive about this strategy and reported it was easy to access and that the ADAPT Team were approachable, “readily available” (E_PSYCH_S10P04T0) and responded quickly. The ADAPT Support Service helped participants to implement the CP and troubleshoot issues as they arose.

“If I had to email about something or I’d queried something, it’s been immediate, they’ve come back… they’ve been supportive and they’re wanting this to work and wanting to work with us. So, I’ve felt that has really set the standard for me that, you know, this is a relationship and I’m sure that we can make this work.” (C_PSYCH_S01P03T0)

Discussion

In this paper, we have reported fidelity to and the acceptability of six categories of implementation strategies, as illustrated by staff perceptions during a 12-month supported implementation of the ADAPT Clinical Pathway. We have also reported any subsequent changes/adaptions made to the implementation strategies and reasons for these differences. All six categories of implementation strategies were delivered and were acceptable to staff. Although most strategies were delivered as planned, there were variations or modifications in terms of the content, frequency and duration of some of the strategies delivered to services, meaning that adaptation was a necessary and potentially, even desirable, feature of implementation strategies.

As noted in Carroll et al., (2007)’s framework, quality of delivery (i.e., delivery of intervention in an appropriate way to achieve what was intended) and participant responsiveness may impact on the coverage, frequency or duration of implementation fidelity [24]. In our study, securing staff engagement at some services was easy, and time was made for delivery of ADAPT implementation strategies; at other services, this was difficult, and scheduling of Roadshow, training sessions and meetings had to “fit-in” within existing meetings, be condensed or delivered via email, potentially affecting fidelity to the strategy and hence implementation success. Furthermore, training gave staff an insight into the ADAPT CP and Portal and how they would work in practice, which catalysed a shift in staff engagement at some services.

Adaptation of implementation strategies and flexibility to accommodate service needs appeared to have a positive impact on implementation fidelity. Most staff felt the Roadshows, training sessions and meetings were important in facilitating staff preparedness, ownership, engagement and to trouble-shoot emerging implementation barriers. However, busyness of service staff, staff turnover and nature of staff employment (such as part-time or shift work patterns) affected engagement. Awareness of implementation context, modification and flexibility (content, duration, frequency and mode of delivery) could act as facilitators to engagement and implementation fidelity.

Our results also highlighted the importance of senior management to endorse implementation, promote staff engagement and optimise the effectiveness of the local Champion strategy. Similar to existing research [26, 27], our results demonstrate the need to consider, not only the role of the Champion, but also attributes that may influence a Champion’s ability to drive service change. Authority or support from upper management and designated time to dedicate to the role may maximise the effectiveness of Champions as an implementation strategy.

Ongoing implementation support is crucial, especially in initial phases of service change or adoption where logistics of implementation are fine-tuned to optimise success. Our research team provided additional training to staff throughout implementation upon site request. This adaptation was acceptable to staff and facilitated awareness and engagement with implementation. Furthermore, most staff at services randomised to the enhanced strategy arm, believed the ADAPT-facilitated monthly meetings contributed to implementation success.

There are limitations to this paper. Although we were able to collect quantitative data on staff engagement with the newsletters, this was not possible for most of the awareness campaign strategies (e.g., number of staff or patients that viewed posters, or number of staff that opened the emails). This limited our understanding of the extent to which staff engaged with these strategies and thus the extent to which these strategies may have been adapted or modified and any impact that this had on adherence to the ADAPT CP. We also had large variations in the duration of the Portal Overview training as some services only received an abbreviated five-minute presentation at the end of the Clinical Pathway training due to lack of additional time. Whilst there was low fidelity to the Portal overview training which provided an overview of the ADAPT Portal and its functionalities, the tailored Portal User training and/or Super User training were provided to all staff who implemented or used the ADAPT Portal. Future research should also include quantitative measures of fidelity to determine impact of implementation fidelity on intervention success.

Conclusion

This current study describes and documents the fidelity, engagement with and acceptability of six implementation strategies developed to assist with implementation of the ADAPT CP into routine cancer care. The findings add to the scarce literature detailing implementation strategies and how they are used and adapted in real-world trials. Clear documentation of fidelity to and understanding of the acceptability of strategies will inform the appropriate selection and design of implementation strategies in future studies and helps to support implementation of interventions in healthcare settings.