Skip to main content
Download PDF

The effects of a prolonged exposure workshop with and without consultation on provider and patient outcomes: a randomized implementation trial

Implementation Science volume 15, Article number: 59 (2020) Cite this article

Abstract

Background

Prolonged exposure therapy (PE) is an evidence-based treatment for posttraumatic stress disorder (PTSD) that is underutilized in the military health system. Standard workshop training in PE may not be sufficient to alter provider behavior, but post-workshop consultation requires significant resources. Therefore, it is important to determine the incremental utility of post-workshop consultation.

Methods

This study used a hybrid type III randomized implementation trial at 3 US Army installations. Providers were randomized to receive a 4-day prolonged exposure workshop (Standard training condition, n = 60), or the prolonged exposure workshop followed by 6–8 months of post-workshop expert case consultation (Extended training condition, n = 43). The effects training condition were examined on provider attitudes (self-efficacy in delivering PE, expectations for patient improvement, and beliefs about PE), use of PE and PE components, and clinical outcomes of patients with PTSD (using the Clinician-Administered PTSD Scale (CAPS-5)).

Results

Extended condition providers reported greater improvements in self-efficacy, b = .83, 95% CI [.38, 1.27], t(79) = 3.71, p = .001, and d = .63. A greater proportion of patients in the Extended condition (44%) than in the Standard condition (27%) received at least 1 PE session, b = .76, t(233) = 2.53, p = .012, and OR = 2.13. Extended condition providers used more PE components (M = .9/session) than did Standard condition providers (M = .5/session), b = .54, 95% CI [.15, .93], t(68) = 2.70, p = .007, and d = .68. Finally, decrease in patients’ PTSD symptoms was faster for patients of Extended condition providers than for patients of Standard condition providers, b = − 1.81, 95% CI [− 3.57, − .04], t(263) = − 2.02, p = .045, and d = .66, and their symptoms were lower at the second assessment, b = − 5.47, 95% CI [− 9.30, − 1.63], t(210) = − 2.81, p = .005, and d = .66.

Conclusions

Post-workshop consultation improved self-efficacy for delivering PE, greater use of PE, faster PTSD reduction, and lower PTSD severity at the second assessment. To our knowledge, this is the first demonstration that post-workshop case consultation for PE improves patient outcomes.

Trial registration

Clinicaltrials.gov, NCT02982538. Registered December 5, 2016; retrospectively registered

Peer Review reports

Background

Despite the robust evidence supporting the efficacy of prolonged exposure (PE) for posttraumatic stress disorder (PTSD) [1], PE is underutilized [2,3,4]. This underutilization is consistent with the low use of exposure therapy [5] and evidence-based psychotherapies (EBPs) in general [6, 7], highlighting the need for research on strategies to improve EBP implementation. One barrier that impedes EBP use is lack of therapist skill in delivering the treatment [8]. Indeed, Becker et al. [9] found that few providers who treat PTSD were trained in PE, and inadequate training was the most common reason for not using PE.

The US Departments of Defense (DoD) and Veterans Affairs (VA) have each pursued large-scale dissemination-implementation initiatives to increase the use of PE for PTSD, with differential success. In the VA PE roll-out, training included 4-day workshops followed by individual weekly consultation with 2 patients. An uncontrolled evaluation of this model found that consultation was associated with improved provider attitudes toward PE [10], which is independently associated with subsequent therapy adoption [11]. Eighteen months after PE consultation, 71% of VA providers reported using PE [12]. In contrast, the DoD adopted a workshop-only PE training model. Only 3% of DoD behavioral health providers listed PE as one of their primary treatments for patients with PTSD [13]. Consistent with broader implementation research findings, this pattern suggests that therapists require support after initial training in order to adopt the treatment (see [14] for a review). Consultation may also impact patient outcomes. For example, Monson et al. [15] found that consultation in cognitive processing therapy (CPT), including discussion of cases and conceptualization without audio review, resulted in significantly greater reductions in patients’ PTSD outcomes relative to no consultation in a VA setting. This study also found that audio review was not a necessary component for consultation to improve patient outcomes. Several other uncontrolled studies have found similar effects [16,17,18]. However, to our knowledge, prior PE implementation studies evaluating post-workshop consultation have not examined provider behavior and patient outcomes. This is an important gap in the literature, as therapists sometimes express reluctance to utilize exposure therapy, particularly in the presence of comorbid conditions [19]. Furthermore, whereas Monson et al.’s [15] study was conducted in the VA, there may be important organizational factors within the military that alter the impact of consultation on provider and patient outcomes in this setting. We are not aware of any controlled trials of consultation as an implementation strategy for improving provider and patient outcomes within military clinics.

Relative to workshops alone, case consultation requires significant time and financial resources for both the providers and consultants. Therefore, it is important to determine the incremental utility of this implementation strategy, both on PE use and on patient outcomes in large systems such as the military. This study used a hybrid type III effectiveness-implementation design, in which the primary goal was to determine the utility of an implementation strategy and the secondary goal was to assess clinical outcomes associated with the strategy [20]. The current study evaluated the impact of post-workshop consultation over 6–8 months on provider self-efficacy, expectations for patient improvement, beliefs related to PE, use of PE, and patients’ outcomes. We hypothesized that, compared to workshop alone (“Standard” training), the addition of post-workshop consultation (“Extended” training) would (1) increase provider self-efficacy for delivering PE, expectations for patient improvement following PE, and positive beliefs about PE; (2) increase the use of PE to treat patients with PTSD; and (3) significantly reduce PTSD symptoms.

Methods

The Madigan Army Medical Center Institutional Review Board (IRB) served as the IRB for the three military sites. The IRBs at the University of Pennsylvania and the University of Texas Health Science Center at San Antonio as well as the US Army Human Research Protections Office also approved the protocol. Providers and patients who participated provided written informed consent.

Providers

Providers were active duty or civilian behavioral health providers (N = 103) working at behavioral health clinics at three US Army medical treatment facilities: Evans Army Community Hospital at Fort Carson, Colorado; William Beaumont Army Medical Center at Fort Bliss, Texas; and Blanchfield Army Community Hospital at Fort Campbell, Kentucky. Providers met inclusion criteria if their responsibilities included delivering individual psychotherapy to adult patients and more than 20% of patients in their caseload had trauma-related difficulties. Providers were excluded if they planned to terminate their position within the next year or if they had extensive training in PE, defined as previous participation in a 4-day PE training workshop and use of PE with > 3 PTSD patients in the past year.

Patients

Patients were active military personnel with PTSD symptoms (N = 242) who were receiving or seeking individual psychotherapy from a participating provider. The inclusion criterion was that they have clinically significant PTSD symptoms (score of ≥ 25 [21] on the Clinician-Administered PTSD Scale for DSM-5 [CAPS-5] [22]). Exclusion criteria were current bipolar disorder I or psychotic disorder, evidence of a moderate or severe traumatic brain injury, or current suicidal ideation severe enough to warrant immediate attention.

Procedures

The study design was a 2-armed randomized implementation trial comparing two PE training models: Standard training (workshop only) and Extended training (workshop plus consultation). Behavioral health providers were invited to participate in a 4-day PE workshop conducted on post. Initial PE workshops at each site were followed by additional on-site workshops as new providers enrolled in the study. Interested providers then completed informed consent with research staff. Local leadership agreed to support changes to participating providers’ schedule templates to allow providers to have 2 timeslots of 90 min each week during the study. Providers at each site were randomized 1:1 by the study coordinators using an online random-number generator. Unfortunately, the random-number method for treatment assignment resulted in more providers being assigned to the Standard condition (n = 60) than to the Extended condition (n = 43). Unbalanced randomization can occur naturally (though infrequently) when using pure randomization methods. Fortunately, as reported later in the “Results” section, this uneven assignment did not result in treatment condition differences between providers on any of the baseline demographic or study variables. Data were collected from January 2014 to January 2018, when target enrollment was met, and the final data point was collected. After the workshop, those randomized to the Extended condition received weekly individual telephone consultation from a PE expert, including the review of session videotapes, with a goal of 2 PE training cases over 6–8 months. Providers obtained consent from patients who were PE training cases to video-record the sessions for the purpose of consultation. PE training cases were not invited to participate in the study. Sessions of patients who were study participants were not video-recorded.

Providers were asked to complete online surveys prior to the workshop, immediately after the workshop, and at 3-month intervals for up to 18 months after finishing training. However, since one purpose of this study was to train as many providers as possible, providers were trained up until about 9 months before completion of the study. So, some providers who enrolled in the study only had time to provide a few (as little as 2) assessments of attitudes before the study ended. During the training phase, providers in the Extended condition were encouraged to implement PE with at least two training cases. During the data collection phase, providers were encouraged to use their clinical judgment when deciding which clinical issue to focus on and which treatment approach to use, allowing for an evaluation of the impact of the implementation strategy (Standard versus Extended) on providers’ selection of treatment approaches. These instructions aimed at eliminating the bias of demand characteristics that might influence greater use of PE while providers were in the study. Providers completed a checklist of the treatment procedures used with patients after each psychotherapy session.

Patients with significant PTSD symptoms (PTSD Checklist for DSM-5 [PCL-5] ≥ 25 [23] [completed as part of routine care]) receiving treatment from participant-providers were invited by their provider to provide data for the study. Interested patients met with research staff to provide informed consent. They then met with an independent evaluator (IE), blind to the type of therapy the patient received and to the type of training received by the provider, who assessed their PTSD symptoms on two occasions: at study enrollment and after 8–15 sessions or 5 months, whichever came first. The second assessment occurred from sessions 3–15 (M = 9.4 for Standard patients, M = 9.1 for Extended), with variability due to factors such as patient transfer to another military installation, deployment, and inability to continue treatment. IEs provided a summary of the assessment findings to the patient’s provider. The study protocol and the statistical analysis plan are included in the online supplement.

Provider measures

Provider demographics

This form assessed demographics and professional information (see Table 1).

Table 1 Provider demographic characteristics and training background
Full size table

Provider measure of attitude

This 52-item measure inquired about treatment practices, attitudes, expectancies, and beliefs. It included 8 items assessing self-efficacy in delivering PE (e.g., I can effectively guide patients through imaginal exposure), 1 item assessing expectations for patient improvement following PE (How effective do you think PE, if delivered competently, would be at improving your patients’ PTSD symptoms and functioning?), 21 items assessing “beliefs about PE” (e.g., Asking patients to discuss traumatic memories in PE may retraumatize them) using the Therapist Beliefs about Exposure Scale [24] modified to refer specifically to PE, 7 items assessing the proportion of PTSD patients with whom various psychotherapy practices are used (e.g., Provide a detailed rationale of how a treatment works), and 15 items assessing the degree to which various patient factors (e.g., Stabilized comorbid bipolar disorder; Anger; Dissociation) would deter providers from using PE. Self-efficacy items were scored 1 = not at all confident to 7 = completely confident. Expectations for patient improvement following PE were scored 1 = not at all effective to 7 = very effective. Beliefs about PE were scored 1 = disagree strongly to 5 = agree strongly. The internal consistencies of the self-efficacy measure and the beliefs measures in the present sample were α = .918 and α = .924, respectively.

Procedures used in treatment checklist

This form assessed the use of 14 different procedures in the previous treatment session, including an “other” option. Providers could check multiple procedures, including PE-related procedures (imaginal exposure, in vivo exposure, processing of traumatic memory, and providing rationale for PE) and non-PE procedures (e.g., supportive therapy, eye-movement desensitization and reprocessing, psychodynamic therapy).

Patient measures

Patient demographics

This form assessed standard demographics (see Table 2).

Table 2 Patient demographic characteristics
Full size table

Clinician-administered PTSD scale for DSM-5 (CAPS-5)

The CAPS-5 [22] is a structured interview that was used to assess PTSD symptom severity based on the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). The CAPS-5 was administered by an IE blind to treatment condition at both patient assessment points (Cronbach’s α = .915).

Interventions

Four-day PE workshop

Four-day PE workshops were conducted at each military installation throughout the duration of the study in order to continue to recruit providers throughout the course of the study. Workshops were conducted by PE experts from the University of Pennsylvania’s Center for the Treatment and Study of Anxiety (CTSA). The PE experts conducting the workshop were trained by the first author (E.B.F), the developer of PE. Each workshop was delivered over 4 consecutive days and included the following: (1) didactics on theoretical and empirical foundations of PE, (2) overview of the PE protocol, (3) instruction in individual therapy procedures with video-recorded examples, (4) monitored role-play practice with feedback, and (5) questions/discussion. Workshop participants received training materials and the PE treatment manual.

Post-workshop consultation

Consultation (for Extended condition providers) involved weekly 1-h telephone consultation from a PE expert with the goal of following 2 PE training cases. PE experts were certified PE providers and PE supervisors trained by E.B.F. Videos of the training cases were mailed to the CTSA and reviewed by the PE expert prior to the consultation call. Consultation calls included feedback on PE session recordings and/or discussion of PE training videos when providers did not have a training case to discuss. Twelve consultation phone calls were completed before the provider began inviting study patients.

Data analytic plan

First, treatment condition differences on both provider and patient baseline characteristics were examined. Since there were 26 different characteristics compared, inflation of type I error was controlled using the Benjamini-Hochberg test. Multilevel models (MLMs) and generalized linear mixed models (GLMMs) were used to analyze the data. These intent-to-treat models included all providers and participants who provided at least 1 assessment. Per protocol, baseline level of outcome was used as a covariate when available. P was set at .05, two-sided.

For hypothesis 1, we expected greater improvement in PE attitudes (self-efficacy, expectancy, and beliefs about PE) over time in Extended vs. Standard providers. Since these variables improved initially and then leveled out, we used a quadratic growth curve to model change over time. The quadratic model fit the data better than other curvilinear models (e.g., log, hyperbolic, etc.) according to Akaike Information Criterion (AIC) and the Bayes Information Criterion (BIC).

For hypothesis 2 (provider use of PE components), we first examined the total number of PE components used by each provider across all their patients and sessions using a Poisson regression. Since some providers had more patients enrolled in the study than others, the total number of treatment sessions differed between providers, so number of treatment sessions was included as an “offset” in the analysis, converting the result to number of PE components used per session. In addition, we also compared the proportion of patients in each condition who were treated with any PE (defined conservatively as at least one session that included both imaginal and in vivo exposure). Patients were coded 1 (treated with PE) or 0 (not treated with PE) in a GLMM analysis (patients nested within providers). Since there were two outcomes for this hypothesis, P values were corrected for inflation of Type 1 error using the Benjamini-Hochberg correction.

The analysis for hypothesis 3 had three levels (the two repeated measurements of PTSD severity [baseline and final assessment] were nested within patients, who were nested within providers). Per our protocol, the primary outcome was PTSD symptoms at the second assessment. Since symptoms were expected to be lower for patients whose second assessment was after more sessions, session number of the final assessment was included as a moderator of treatment condition differences. And because the effect of sessions leveled out over time (i.e., the decrease in PTSD severity leveled out over sessions), “sessions” was coded as the square root of session number (this coding provided the best fit [lowest AIC and BIC] for the data). Follow-up GLMM analyses examined the secondary outcomes (change in PTSD diagnosis and clinically significant change, both dichotomous outcomes) using similar models. P values for the secondary outcomes were corrected for inflation of type 1 error using the Benjamini-Hochberg correction.

Initial analyses controlled for site as a covariate and moderator. However, site was not a significant factor in any of the analyses. Therefore, site was dropped and the analyses recomputed.

A priori power analyses, using RMASS2, indicated > .85 power to detect a medium effect size (d = .5) for hypothesis 1 (treatment condition differences in self-efficacy, expectancy, and beliefs toward PE) assuming 100 providers with at least 3 assessments each, thus setting sample size at N = 100. Post hoc power analyses, using actual N in each analysis, indicated that for hypothesis 1, power was > .90 power to detect a medium effect size. For hypotheses 2 and 3, the power to detect medium effects was > .80.

Results

Figure 1 displays participant flow through the study. Provider and patient demographics and assessment information are reported in Tables 1 and 2, respectively. Out of the 26 variables reported in these tables, no treatment condition differences emerged after Benjamini-Hochberg correction. Substantive (but not significant) differences between conditions did exist on patient gender and “any comorbid anxiety disorder” (Extended condition females = 19.2%, comorbid anxiety disorders = 56.4%, Standard condition females = 7.1%, anxiety disorders = 28.6%). However, neither patient gender nor patient comorbid anxiety disorders was a significant moderator or covariate in the analyses of the patients, nor did including them in these analyses change any of the reported results.

Fig. 1
figure 1

Diagram of provider participant flow

Full size image

Differences in provider attitudes toward PE

For hypothesis 1, the number of assessments of PE attitudes per provider ranged from 1 to 8 in both treatment groups (M = 5.49, SD = 2.14). Consistent with hypothesis 1, Extended condition providers (N = 43) reported greater improvements in self-efficacy for delivering PE over time than did Standard condition providers (n = 60), training × time interaction: b = .83, 95% CI [.38, 1.27], t(79) = 3.71, p = .001, and d = .63 (Fig. 2). Self-efficacy for delivering PE improved significantly over time for Extended condition providers, b = .94, 95% CI [.58, 1.30], t(81) = 5.22, p < .001, and d = 1.16, but not for Standard condition providers. By the fourth assessment of attitudes, corresponding to 6–8 months after training, Extended condition providers reported significantly greater self-efficacy than Standard condition providers, b = 2.40, 95% CI [.49, 4.22], t(94) = 2.64, p = .01, and d = .54. There were no significant differences between training conditions in expectations for patient improvement following PE nor for beliefs about PE.

Fig. 2
figure 2

Months since end of training

Full size image

Use of PE therapy components

Due to the fact that new therapists entered the study throughout the study, not all therapists had patients with PTSD enter the study; many had no qualifying patients before the conclusion of the study. There were 49 Standard condition providers (81.7%) and 23 Extended condition providers (53.5%) who had patients in the study and were included in the analyses for hypotheses 2 and 3. The proportion of Extended condition providers without patient data was larger because they could not enroll patients until after case consultation was completed, which took ≥ 6 months. There were no significant differences on demographics or on beliefs about PE (self-efficacy, effectiveness, positive attitudes) between providers who enrolled patients in the study and those that did not.

Poisson regression analyses showed that Extended condition providers used more PE components per treatment session (M = .9/session) than Standard condition providers (M = .5/session), b = .54, 95% CI [.15, .93], t(68) = 2.70, p = .012 (corrected), and d = .68. Importantly, this effect was limited to PE components; no significant difference between conditions emerged in non-PE components used during treatment sessions (MExtended = 1.25 vs. MStandard = 1.19). Further, GLMM analysis showed that the proportion of patients of Extended providers who actually received PE (defined as at least 1 session that included both in vivo and imaginal exposure) was greater than the proportion of patients of Standard providers who received PE (44% vs. 27%), b = .76, t(233) = 2.53, p = .012 (corrected), OR = 2.13.

Patient outcomes

For hypothesis 3, Standard condition providers had 172 patients (3.5 patients per provider) in the study. Extended condition providers completed consultation in time to have 70 patients (3.0 patients per provider). The small difference in patients per provider was primarily due to the consultation delay before Extended condition providers could take patients into the study. Thirty-six percent of patients did not complete the second PTSD assessment, in part due to transfers/deployment. These patients did not differ from those who completed both assessments on demographics or baseline PTSD severity.

Three-level MLM analyses included random effects for patients and for providers (which were non-significant) and baseline severity coded above/below median (this allowed inclusion of baseline CAPS in the MLM, providing intent-to-treat analyses). Improvement in PTSD symptoms per session was greater for patients of Extended condition providers than for patients of Standard providers, training × sessions interaction: b = − 1.81, 95% CI [− 3.57, − .04], t(263) = − 2.02, p = .045, and d = .62 (Fig. 3). For patients who completed 12 sessions, estimated PTSD symptoms were 28.0 for Standard condition patients vs. 22.5 for Extended condition patients, b = − 5.47, 95% CI [− 9.30, − 1.63], t(210) = − 2.81, p = .005, and d = .66.

Fig. 3
figure 3

CAPS-5 for Patients at Their Second (Final) Assessment. Note: CAPS-5, Clinician Administered PTSD Scale for DSM-5. The session at which the second assessment occurred varied from session 3-15. This graph illustrates the treatment condition differences at 4 of those assessment points

Full size image

Further analyses examined whether treatment condition differences were also apparent for the secondary outcomes: the loss of PTSD diagnosis and for clinically significant change (CSC) [25, 26]. These dichotomous outcomes were analyzed using GLMM with a logistic linking function. The independent variables in these models included training, sessions, and the training × sessions interaction as above. Using data from a large, multisite PTSD study [15] and using the Jacobson and Truax definition of CSC [25], CSC was defined as reliable change (1.96 × SD = 13) [26] plus having the outcome within the normal range (CAPS-5 < 20.89) [15]. Analyses indicated that the proportion of patients with loss of PTSD diagnosis was higher in patients of Extended condition providers compared to patients of Standard condition providers, Mextended = 76.9%, Mstandard = 44.3%, bdifference = 1.20, 95% CI [.20, 2.20], t(391) = 2.37, and p = .019 (corrected). Similarly, the rate of CSC was significantly higher in patients of Extended providers compared to patients of Standard providers, Mextended = 50.2%, Mstandard = 23.9%, bdifference = 1.16, 95% CI [.24, 2.08], t(21) = 2.63, and p = .019 (corrected).

Post hoc sensitivity analyses were conducted to control for the fact that Extended providers saw two more PTSD patients than Standard providers before treating PTSD patients for the study. To test whether this accounted for treatment condition differences, two sets of analyses controlled for number of PTSD patients treated by each provider during the study, one including, and one not including, the two consultation patients in the count of patients. Both sets of analyses showed that (1) number of PTSD patients treated was not related to patient outcomes either as a moderator or as a covariate and (2) patients of Extended providers improved more than patients of Standard providers even when analyses controlled for number of PTSD patients treated (either as a covariate or as a moderator). These results suggest that simply treating more PTSD patients does not seem to account for the superior outcomes for patients of Extended providers.

Discussion

This hybrid type III effectiveness-implementation trial is the first study to demonstrate that consultation as an implementation strategy improved military providers’ self-efficacy for using EBP, use of EBP, and patients’ outcomes. These findings suggest that consultation is an effective implementation strategy for EBPs in the military health system and is also likely to be effective in other health systems.

Consistent with prior studies of post-workshop training [10], consultation increased provider self-efficacy in delivering PE, indicating that expert consultation improves confidence in providers’ ability to use PE effectively. Inconsistent with hypotheses, consultation did not increase expectations for improvement in PTSD following PE, or positive beliefs about PE, more than workshop training alone. A lack of effect of consultation on expectations for improvement is particularly interesting since it suggests that our results were not merely a result of expectations/self-fulfilling prophecies [27].

Consistent with hypothesis, consultation was associated with greater use of PE. The rate of use of PE components by Extended providers was almost 1 component (exactly .9 components) per treatment session, which was 80% greater than the use of PE components by Standard providers (.5 per treatment session). Further, Extended condition providers implemented PE (defined as at least 1 session with both in vivo and imaginal exposure) with a greater proportion of their PTSD patients. The rate of PE use was 50% higher among providers who received consultation compared to those who did not. This occurred despite the fact that providers were not directly incentivized to use PE in this study.

Also consistent with our hypothesis, consultation resulted in faster improvement and lower PTSD symptoms among patients. This study also found that patients of Extended providers were much more likely to lose their PTSD diagnosis than patients of Standard providers (77% compared to 44%). In addition, over twice as many patients of Extended providers demonstrated clinically significant change than did patients of Standard providers (50% vs. 24%). To our knowledge, this is the first study to demonstrate that PE consultation impacts not only providers but also the clinical outcomes of their patients. This finding is consistent with prior research in the context of veterans receiving CPT for PTSD [15] and with several uncontrolled studies [16,17,18]. This finding is important because improving patient outcomes is the ultimate purpose of provider EBP training, and hastening recovery is critical to minimizing patient suffering, cost of treatment, and achievement of military readiness efficiently.

These findings are consistent with a large body of evidence showing that workshop attendance alone is not sufficient to alter provider behavior, whereas follow-up consultation after the workshop can improve implementation [14]. This is the first study to our knowledge to demonstrate the benefit of consultation for improving implementation and patient outcomes.

Several limitations should be noted. First, to preserve external validity, we asked providers to self-report on the treatment procedures used in session, rather than using more intrusive data collection methods such as recording of sessions. It is possible that demand characteristics biased provider reporting, although providers were encouraged to use any treatment approach they felt was indicated for their patients. Second, many providers did not enroll any PTSD patients. Although providers reported having a caseload with least 20% having trauma-related difficulties, those who did not enroll patients to the study indicated that they either did not have patients with PTSD who were seeking psychotherapy (e.g., they may have presented for an evaluation or for crisis management) or did not have PTSD patients who met study criteria and agreed to participate. Importantly, there were no significant differences between providers who had study patients and those who did not on any of the assessed measures (on attitudes or on the 16 demographics). Third, relying on providers to invite patients may have biased the patient-participant pool. Study staff informed the participating providers frequently about the patients on their case load who met the screening criteria and encouraged providers to recruit all their eligible patients. However, ultimately, the provider made the decision of who to recruit. This is a limitation inherent to a type III hybrid implementation design, which required minimal influence on providers’ selection of patients in order to obtain more generalizable observations about the effect of the implementation strategy on providers’ behavior. Finally, our initial random number method for randomizing providers to treatment condition resulted in more providers being assigned to the Standard condition than to the Extended condition. Although this was unfortunate, unbalanced randomization can occur naturally (though infrequently) when using pure randomization methods rather than block randomization. Since there were no significant differences between providers assigned to the two treatment conditions, there is no indication that this uneven treatment assignment reflected a bias in treatment assignment. Thus, it is unlikely to be responsible for our obtained pattern of results. Finally, the findings may not generalize outside the military health system, as systems vary with regard to their organization, training and supervision practices, quality assurance practices, and the patient groups they serve.

Conclusions

The results of this study demonstrate the importance of post-workshop consultation. Compared to Standard training, Extended training was associated with: (1) greater self-efficacy delivering PE, (2) greater use of PE and PE components, (3) faster improvement and lower PTSD symptoms among patients, (4) greater probability of loss of PTSD diagnosis among patients, and (5) more than twice the probability of experiencing clinically significant gains among patients. To our knowledge, this is the first study to demonstrate that PE consultation impacts not only providers but also the clinical outcomes of their patients. However, consultation remains costly. For example, in this study, sessions were video-recorded and mailed by providers to expert consultants who reviewed sessions and met with providers individually for 1 hour by phone weekly over several months. Recent evidence suggests that review of session recordings may not be necessary in effective consultation [15]. Future research could focus on critical components of consultation in order to provide benefits with lower cost.

Abbreviations

AIC:

Akaike Information Criterion

BIC:

Bayes Information Criterion

CAPS-5:

Clinician-Administered PTSD Scale for DSM-5

CSC:

Clinically significant change

CTSA:

University of Pennsylvania’s Center for the Treatment and Study of Anxiety

DoD:

Department of Defense

DSM-5:

Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition

EBP:

Evidence-based psychotherapy

GLMMs:

Generalized linear mixed models

IRB:

Institutional review board

MLMs:

Multilevel models

PCL-5:

PTSD Checklist for DSM-5

PE:

Prolonged exposure therapy

PTSD:

Posttraumatic stress disorder

VA:

Department of Veterans Affairs

References

  1. Powers MB, Halpern JM, Ferenschak MP, Gillihan SJ, Foa EB. A meta-analytic review of prolonged exposure for posttraumatic stress disorder. Clin Psychol Rev 2010;30(6):635-641. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.cpr.2010.04.007.

  2. Borah EV, Wright EC, Donahue DA, Cedillos EM, Riggs DS, Isler WC, Peterson AL. Implementation outcomes of military provider training in cognitive processing therapy and prolonged exposure therapy for post-traumatic stress disorder. Mil Med 2013;178(9):939-944. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.7205/MILMED-D-13-00072.

  3. Rosen CS, Chow HC, Finney JF, Greenbaum MA, Moos RH, Sheikh JI, Yesavage JA. VA practice patterns and practice guidelines for treating posttraumatic stress disorder. J Trauma Stress 2004;17(3):213-222. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1023/B:JOTS.0000029264.23878.53.

  4. van Minnen A, Hendriks L, Olff M. When do trauma experts choose exposure therapy for PTSD patients? A controlled study of therapist and patient factors. Behav Res Ther 2010;48(4):312-320. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.brat.2009.12.003.

  5. Freiheit S, Vye C, Swan R, Cady M. Cognitive-behavioral therapy for anxiety: is dissemination working? Behav Ther. 2004;27(2):25–32.

    Google Scholar 

  6. Goisman RM, Warshaw MG, Keller MB. Psychosocial treatment prescriptions for generalized anxiety disorder, panic disorder, and social phobia, 1991-1996. Am J Psychiatry. 1999;156(11):1819–21.

    CAS  PubMed  Google Scholar 

  7. Jameson JP, Chambless DL, Blank MB. Empirically supported treatments in rural community mental health centers: a preliminary report on current utilization and attitudes toward adoption. Community Ment Health J 2009;45(6):463-467. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1007/s10597-009-9230-7.

  8. Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol 2015;3(1):32. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1186/s40359-015-0089-9.

  9. Becker CB, Zayfert C, Anderson E. A survey of psychologists’ attitudes towards and utilization of exposure therapy for PTSD. Behav Res Ther. 2004;42(3):277–92 https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/pubmed/14975770.

    Article  Google Scholar 

  10. Ruzek JI, Eftekhari A, Rosen CS, Crowley JJ, Kuhn E, Foa EB, Hembree EA, Karlin BE. Effects of a comprehensive training program on clinician beliefs about and intention to use prolonged exposure therapy for PTSD. Psychol Trauma. 2016;8(3):348 https://0-www-ncbi-nlm-nih-gov.brum.beds.ac.uk/pubmed/26524541.

    Article  Google Scholar 

  11. Harned MS, Dimeff LA, Woodcock EA, Contreras I. Predicting adoption of exposure therapy in a randomized controlled dissemination trial. J Anxiety Disord 2013;27(8):754-762. https://0-dx-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.janxdis.2013.02.006.

  12. Rosen CS, Eftekhari A, Crowley JJ, et al. Maintenance and reach of exposure psychotherapy for posttraumatic stress disorder 18 months after training. J Trauma Stress 2017;30(1):63-70. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1002/jts.22153.

  13. Borah EV, Holder N, Chen K. Providers’ use of evidence-based treatments for posttraumatic stress disorder: the influence of training, attitudes, and barriers in military and private treatment settings. Best Pract Ment Health. 2017;13(1):34–46.

    Google Scholar 

  14. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev 2010;30(4):448-466. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.cpr.2010.02.005.

  15. Monson CM, Shields N, Suvak MK, Lane JE, Shnaider P, Landy MS, Wagner AC, Sijercic I, Masina T, Wanklyn SG, Stirman SW. A randomized controlled effectiveness trial of training strategies in cognitive processing therapy for posttraumatic stress disorder: impact on patient outcomes. Behav Res Ther 2018;110:31-40. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.brat.2018.08.007.

  16. Eftekhari A, Ruzek JI, Crowley JJ, Rosen CS, Greenbaum MA, Karlin BE. Effectiveness of national implementation of prolonged exposure therapy in veterans affairs care. JAMA Psychiatry 2013;70:949–955. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1001/jamapsychiatry.2013.36.

  17. Ehlers A, Grey N, Wild J, Stott R, Liness S, Deale A, Handley R, Albert I, Cullen D, Hackmann A, Manley J, McManus F, Brady F, Salkovskis P, Clark D. Implementation of cognitive therapy for PTSD in routine clinical care: effectiveness and moderators of outcome in a consecutive sample. Behav Res Ther 2013;51:742–752. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.brat.2013.08.006.

  18. Karlin BE, Brown GK, Trockel M, Cunning D, Zeiss AM, Taylor CB. National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs Health Care System: therapist and patient level outcomes. J Consult Clin Psychol 2012;80:707–718. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1037/a0029328 PMC 22823859.

  19. Cook JM, Dinnen S, Simiola V, Thompson R, Schnurr PP. VA residential provider perceptions of dissuading factors to the use of two evidence-based PTSD treatments. Prof Psychol Res Pract. 2014;45:136–42.

    Article  Google Scholar 

  20. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs. Med Care 2012;50(3):217-226. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1097/MLR.0b013e3182408812.

  21. Schnurr PP, Chard KM, Ruzek JI, Chow BK, Shih MC, Resick PA, Foa EB, Marx BP, Huang GD, Lu Y. Design of VA cooperative study #591: CERV-PTSD, comparative effectiveness research in veterans with PTSD. Contemp Clin Trials 2015;41:75-84. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.cct.2014.11.017.

  22. Weathers FW, Blake DD, Schnurr PP, Kaloupek DG, Marx BP, Keane TM. The Clinician-Administered PTSD Scale for DSM-5 (CAPS-5). U.S. Department of Veterans Affairs, National Center for PTSD. 2013. https://www.ptsd.va.gov/professional/assessment/adult-int/caps.asp.Accessed 13 September 2019.

  23. Wortmann JH, Jordan AH, Weathers FW, Resick PA, Dondanville KA, Hall-Clark B, Foa EB, Young-McCaughan S, Yarvis JS, Hembree EA, Mintz J. Psychometric analysis of the PTSD Checklist-5 (PCL-5) among treatment-seeking military service members. Psychol Assess 2016;28(11):1392-1403. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1037/pas0000260.

  24. Deacon BJ, Farrell NR, Kemp JJ, Dixon LJ, Sy JT, Zhang AR, McGrath PB. Assessing therapist reservations about exposure therapy for anxiety disorders: the therapist beliefs about exposure scale. J Anxiety Disord. 2013;27(8):772-780.https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1016/j.janxdis.2013.04.006.

  25. Jacobson NS, Roberts LJ, Berns SB, McGlinchey JB. Methods for defining and determining the clinical significance of treatment effects: description, application, and alternatives. J Consult Clin Psychol 1999;67(3):300-307. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1037//0022-006x.67.3.300.

  26. Jacobson NS, Truax P. Clinical significance: a statistical approach to defining meaningful change in psychotherapy research. J Consult Clin Psychol 1991;59(1):12-19. https://0-doi-org.brum.beds.ac.uk/https://0-doi-org.brum.beds.ac.uk/10.1037//0022-006x.59.1.12.

  27. Rosenthal R, Jacobson L. Pygmalion in the classroom: teacher expectation and pupils' intellectual development. Crown House Pub.: Carmarthen, Wales; 1992.

    Google Scholar 

Download references

Acknowledgements

We thank the following people who carried out study procedures: Mrudula Raparla, MS, Julie Blow, PhD and Loreli Leos, LPC (William Beaumont Army Medical Center); Jennifer Deluzio, MS, Elizabeth Eversole, BA, Kelly Nypaver, LMHC (Blanchfield Army Community Hospital); Heather Campbell, PhD, Lori-Ann Landry, LCSW, Nicole Peak, PhD, and Adam Ward, BA (Evans Army Community Hospital); Jody Zhong, BA (University of Louisville); Kathy Benhamou, BA and Jesse McCann BS (University of Pennsylvania); Deanne Hargita, MPA, Ray Aguilar, BS and Antoinette Brundige, MA (University of Texas Health Science Center at San Antonio).

Availability of supporting data

The data that support the findings of this research are available upon reasonable request from the repository@strongstar.org.

About the STRONG STAR consortium

The South Texas Research Organizational Network Guiding Studies on Trauma and Resilience, known as the STRONG STAR Consortium, is a multidisciplinary and multi-institutional research network that brings together the expertise of over 150 collaborating investigators at more than 40 civilian, military, and VA institutions in an effort to find the best ways to prevent, diagnose, and treat combat-related PTSD and co-occurring conditions in active duty and recently discharged veterans. The consortium’s portfolio of more than 65 affiliated projects has different foci, principal investigators, and research teams and are funded by a variety of federal and private sources. However, they benefit from the consortium’s established infrastructure, specialized research cores, and synergistic collaboration. The STRONG STAR Consortium is directed by Alan Peterson, PhD, professor of psychiatry and behavioral sciences in the Long School of Medicine at The University of Texas Health Science Center at San Antonio, petersona3@uthscsa.edu. For more information, visit www.strongstar.org.

Disclaimer

The views expressed in this article are solely those of the authors and do not reflect an endorsement by or the official policy of the US Army, the Department of Defense, the Department of Veterans Affairs, or the US Government. The investigators have adhered to the policies for protection of human subjects as prescribed in 45 CFR 46.

Funding

This work was supported by the US Department of Defense through the US Army Medical Research and Materiel Command, Congressionally Directed Medical Research Programs [grant number W81XWH-12-2-0116 (Edna B. Foa)]. The funding source had no involvement in the study design, the collection, analysis and interpretation of data, the writing of this report, or the decision to submit this article for publication.

Author information

Authors and Affiliations

  1. Department of Psychiatry, School of Medicine, University of Pennsylvania, Philadelphia, PA, 19104, USA

    Edna B. Foa, Lily A. Brown, Yinyin Zang & Laurie J. Zandberg

  2. National Center for PTSD, Dissemination and Training Division, VA Palo Alto Health Care System, Menlo Park, CA, USA

    Carmen P. McLean

  3. Department of Psychiatry and Behavioral Sciences, School of Medicine, Stanford University, Palo Alto, CA, USA

    Carmen P. McLean

  4. School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China

    Yinyin Zang

  5. Department of Psychology, Southern Methodist University, Dallas, TX, USA

    David Rosenfield

  6. Blanchfield Army Community Hospital, Fort Campbell, KY, USA

    Wayne Ealey, Lora Rose Hunter, Ivett J. Lillard, Thomas J. Patterson, Valerie Scott & Joseph E. Wise

  7. William Beaumont Army Medical Center, Fort Bliss, TX, USA

    Brenda S. Hanson, Julio Rosado & Charles D. Zamora

  8. Landstuhl Regional Medical Center, Landstuhl, Germany

    Brenda S. Hanson

  9. Fielding Graduate University, California, Santa Barbara, USA

    Ivett J. Lillard

  10. Department of Behavioral Health, Madigan Army Medical Center, Washington DC, USA

    Thomas J. Patterson

  11. South Texas Veterans Health Care System, San Antonio, Texas, USA

    Julio Rosado

  12. Washington Army National Guard, Washington, DC, USA

    Valerie Scott

  13. Evans Army Community Hospital, Fort Carson, CO, USA

    Charles Weber

  14. Department of Psychiatry and Behavioral Sciences, School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA

    Jim Mintz, Stacey Young-McCaughan & Alan L. Peterson

  15. South Texas Veterans Health Care System, San Antonio, TX, USA

    Alan L. Peterson

  16. Department of Psychology, University of Texas at San Antonio, San Antonio, TX, USA

    Alan L. Peterson

Authors
  1. Edna B. Foa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carmen P. McLean
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lily A. Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yinyin Zang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David Rosenfield
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Laurie J. Zandberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wayne Ealey
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Brenda S. Hanson
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Lora Rose Hunter
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ivett J. Lillard
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Thomas J. Patterson
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Julio Rosado
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Valerie Scott
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Charles Weber
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Joseph E. Wise
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Charles D. Zamora
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Jim Mintz
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Stacey Young-McCaughan
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Alan L. Peterson
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

for the STRONG STAR Consortium

Contributions

Dr. Foa had full access to the study data and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Foa, McLean, Mintz, Peterson, and Young-McCaughan. Acquisition of data: Foa, McLean, Zandberg, Ealey, Hanson, Hunter, Lillard, Patterson, Rosado, Scott, Weber, Wise, and Zamora. Statistical analysis: Zang and Rosenfield. Interpretation of data: Foa, McLean, Brown, Zang, and Rosenfield. Drafting of the manuscript: Foa, McLean, Brown, Zang, and Rosenfield. Critical revision of the manuscript for important intellectual content: Foa, McLean, Brown, Zang, Rosenfield, Zandberg, Ealey, Hanson, Hunter, Lillard, Patterson, Rosado, Scott, Weber, Wise, Zamora, Mintz, Young-McCaughan, and Peterson. Obtained funding: Foa, McLean, Peterson, and Young-McCaughan. Administrative, technical, or material support: McLean, Zandberg, Campbell, Ealey, Hanson, Hunter, Lillard, Patterson, Rosado, Scott, Weber, Wise, Zamora, Young-McCaughan, and Peterson. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Edna B. Foa.

Ethics declarations

Ethics approval and consent to participate

The University of Pennsylvania Institutional Review Board (IRB) served as the coordinating center for this study and the Madigan Army Medical Center (MAMC) IRB provided review and oversight for the coordinating center and each of the three study performance sites, in compliance with all applicable federal regulations governing the protection of human subjects. The MAMC IRB reviewed and approved the University of Pennsylvania protocol (MAMC protocol # 214055) on April 9, 2014; the Evans Army Community Hospital protocol (MAMC protocol # 214056) on April 9, 2014; the William Beaumont Medical Center protocol (MAMC protocol # 215025) on January 27, 2015; and the Blanchfield Army Community Hospital protocol (MAMC protocol # 215045) on April 16, 2015. All study staff, including investigators at military installations, have completed the requisite human subjects training preparatory to research. All participants provided informed consent.

Consent for publication

Not applicable.

Competing interests

Dr. Foa reports having received research funding from the Department of Defense, Department of Veterans Affairs, and National Institutes of Health. In addition, Dr. Foa published books on PTSD treatment for which she receives royalties and also conducts PTSD trainings for which she receives an honorarium. Dr. McLean reports having received research funding from the Department of Defense. Dr. Brown reports having received research funding from the Department of Defense and National Institutes of Health. Dr. Rosenfield reports having received research funding from the National Institutes of Health, National Institute on Drug Abuse, and Department of Defense. Dr. Mintz reports having received research funding from the Department of Defense, Department of Veterans Affairs, and National Institutes of Health. Dr. Young-McCaughan reports having received research funding from the Department of Defense, Department of Veterans Affairs, and National Institutes of Health. Dr. Peterson reports having received research funding from the Department of Defense, Department of Veterans Affairs, National Institutes of Health, and Robert Wood Johnson Foundation. No other disclosures were reported.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1:.

Study protocol and the statistical analysis plan

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Foa, E.B., McLean, C.P., Brown, L.A. et al. The effects of a prolonged exposure workshop with and without consultation on provider and patient outcomes: a randomized implementation trial. Implementation Sci 15, 59 (2020). https://0-doi-org.brum.beds.ac.uk/10.1186/s13012-020-01014-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13012-020-01014-x

Keywords

Implementation Science

ISSN: 1748-5908