Effectiveness of Early Psychosis Intervention: Comparison of Service Users and Nonusers in Population-Based Health Administrative Data

Rapid growth in specialized early psychosis intervention (EPI) services has occurred over the past two decades (1). These programs focus on early symptom detection and comprehensive, phase-specific treatment during the initial stages of psychosis (2). EPI programs were established on the basis of robust evidence that an extended duration of untreated psychosis is associated with poor clinical and functional outcomes (3, 4). Large randomized trials suggest that EPI services reduce psychopathology and improve patients’ quality of life, vocational outcomes, treatment adherence, and satisfaction with care (5–7). There is also increased evidence that EPI services are cost-effective when compared with standard care (8–11). However, these benefits may not be sustained once patients are discharged to standard psychiatric care (12, 13). Therefore, randomized trials designed to establish the optimal duration of EPI services are currently under way (14–16).

Less is known about whether findings from randomized trials generalize to populations served by EPI programs when these programs are implemented within the broader health care system. Differences may arise due to factors such as stringency of inclusion criteria or fidelity to the EPI treatment model. Furthermore, randomized trials typically have relatively small sample sizes and short follow-up periods, which limit the ability to assess rare but important outcomes (e.g., self-harm or suicide) (17, 18). Large health administrative databases can provide a population-based perspective on the impact of EPI services in the broader health care system, as well as sufficiently large sample sizes to estimate rare events.

In this study, we examined the real-world effectiveness of a well-established EPI program over a 17-year period using linked health administrative data. The objective of this study was to compare indicators of health service use, self-harm, suicide, and mortality between people with first-episode psychosis who were using EPI services and a propensity-matched group of concurrent control subjects who were not accessing EPI services. We assessed use of primary care services, use of outpatient psychiatry, use of the emergency department, and inpatient hospitalizations, as well as rates of self-harm, suicide, and all-cause mortality. We hypothesized that persons who used EPI services would have better outcomes across these indicators compared with those who did not use EPI services.

Method

Study Setting

This study benefited from the fact that the Prevention and Early Intervention Program for Psychoses in London, Ontario, is the only EPI service in a region with a population of approximately 425,000 people. This enabled us to ascertain cases of incident nonaffective psychosis in the program catchment area and identify persons who received EPI care. The Prevention and Early Intervention Program for Psychoses provides team-based, coordinated specialty care for people with early-stage psychosis, as described in detail elsewhere (19). Core features of the program include early case detection, an open-referral policy, rapid assessment of suspected cases, development of a flexible treatment plan, psychoeducation, and provision of comprehensive pharmacological and psychosocial interventions with an assertive case-management approach (19). The typical trajectory for a program client is intensive case management during the 2 years after admission, with care stepped down to medical management by a program psychiatrist between years 2 and 5. However, patients with greater needs or ongoing symptoms may continue to receive case management beyond 2 years (19). The program has evolved over the past two decades in accordance with emerging evidence pertaining to best practices and implementation of provincial standards for EPI services (20). It shares many core features with the National Institute of Mental Health NAVIGATE-RAISE (Recovery After an Initial Schizophrenia Episode [21]) program and the Early Psychosis Intervention Center program (22), with the main differences being the minimum age requirement for inclusion (16 years old) and an inpatient component of the Prevention and Early Intervention Program for Psychoses to provide continuity of care across settings.

Source of Data

We used linked health administrative data from the Institute for Clinical Evaluative Sciences, which captures all hospitalizations, emergency department visits, and outpatient physician visits covered under the universal Ontario Health Insurance Program. All medically necessary health services for nearly the entire population of Ontario are covered by the Ontario Health Insurance Program. Persons not covered are students from out-of-province jurisdictions and recent migrants (≤3 months of residency). The linked databases used and the services covered are listed below:

• Registered Persons Database (1990–2014): demographic and mortality data

• Immigration, Refugees, and Citizenship Canada (1985–2014): migrant status

• Ontario Health Insurance Plan (1991–2014): physician billings

• Discharge Abstract Database (1988–2014): inpatient hospitalizations

• Ontario Mental Health Reporting System (2005–2014): psychiatric hospitalizations

• National Ambulatory Care Reporting System (2000–2014): emergency department visits

• Ontario Registrar General–Death (1990–2013): vital statistics

Cohort Creation

We constructed a retrospective cohort comprising incident case subjects with nonaffective psychotic disorder who resided in the EPI catchment area between 1997 and 2013. We mirrored the inclusion criteria for the Prevention and Early Intervention Program for Psychoses as closely as possible (19) and included case subjects who had an incident diagnosis of nonaffective psychosis as determined using a previously validated algorithm (23) and defined by the presence of either an inpatient admission with a primary discharge diagnosis of nonaffective psychotic disorder or two physician or emergency department visits with a diagnostic code for nonaffective psychotic disorder occurring within a 12-month period. The diagnostic codes varied by database and included ICD-9, ICD-10, and DSM-IV codes (for further details, see Table S1 in the data supplement accompanying the online version of this article). Additionally, we included case subjects who had no previous contacts for nonaffective psychotic disorder before 1997 (i.e., prevalent cases), with a look-back window of up to 20 years; who presented to services within the catchment area of the EPI program; and who were 16–50 years of age.

Primary data on EPI clients were linked deterministically to data from the Institute for Clinical Evaluative Sciences through Ontario Health Insurance Plan numbers (96% linkage rate), which enabled us to identify persons in our cohort who used EPI services. We linked data on EPI program psychiatrists using registration numbers from the College of Physicians and Surgeons of Ontario. All persons in the cohort who had a visit with a program psychiatrist but were not subsequently admitted were considered to have been screened for the program but found to be ineligible and were excluded from further analyses. The remaining members of the cohort (i.e., those who were not identified as EPI service users but were not excluded from further analyses) were classified as “nonusers.”

As a post hoc exclusion, we removed from further analysis people who had not had their diagnosis confirmed by a psychiatrist at any point in their medical record and had no subsequent contacts with services for psychosis, since these cases were likely “false positives.” We also excluded people with fewer than two contacts with any service in the catchment area in the 6 months after cohort entry, since they were likely to have moved from the area.

Data linkage procedures were approved by the Health Sciences Research Ethics Board at the University of Western Ontario, London. Patient-level data were linked using encoded identifiers and analyzed on-site at the Institute for Clinical Evaluative Sciences. We followed the RECORD ([REporting of studies Conducted using Observational Routinely-collected health Data] [24]) guidelines for observational studies. A description of the codes and algorithms used to create study variables is presented in Table S1 in the online data supplement, and the RECORD checklist is presented in Table S2.

Baseline Covariates

The date of the first diagnosis of nonaffective psychosis was used as the index date. We extracted information on sociodemographic characteristics, including age, gender, neighborhood-level income quintile, migrant status, and rurality of residence.

We constructed several clinical indicators, including type of diagnosis (schizophrenia, delusional disorder, or other), source of index diagnosis (inpatient admission and outpatient visit), and history of contact with services for alcohol-related or substance-related disorders.

Finally, we constructed indicators of previous service use for mental disorders, including the number of prior mental health contacts with a family physician, a psychiatrist, and the emergency department, as well as previous psychiatric hospitalizations.

Outcome Measures

The primary outcome variables were obtained for two time periods: the period from EPI admission to 2 years postadmission, when clients typically receive intensive case management, and the period from 2 to 5 years postadmission, when care typically transitions from intensive case management to medical management by a psychiatrist. However, we do not have the exact dates for when this transition occurred.

We constructed outcome indicators for each observation period, including contact with a primary care physician, contact with a psychiatrist, emergency department visits, psychiatric hospitalizations, and involuntary admissions. We obtained data on any use of these services as an indicator of access (binary variable), intensity of service use (count variable), and time between EPI admission and the first encounter with each service.

We also obtained information on presentations to the emergency department for self-harm, and vital statistics and hospitalizations were used to identify deaths by suicide. All cases of unnatural death in Ontario are investigated by a coroner, and cases with clear evidence of suicidal intent are recorded as suicide. Given the rarity of suicide and the high likelihood of misclassification in coroner records (25), we made a post hoc decision to compare all-cause mortality between the groups.

Propensity Score Matching

We used propensity score matching to identify a comparison group of nonusers of EPI services. This method mimics some of the characteristics of a randomized trial when using observational data (26). A propensity score is defined as the probability that a person is “exposed,” conditional on a set of observed variables. In a randomized trial, the probability of exposure is 0.5, and this can be estimated in an observational study by regressing exposure status on observed baseline characteristics (26). Matching on the propensity score functions to achieve balance between exposed and unexposed groups on the variables included in the propensity score model (26) but does not achieve balance on unmeasured confounding factors as in a randomized trial.

We used logistic regression to model the exposure variable (EPI service user, EPI service nonuser) as a function of observed covariates to yield a probability of EPI admission for each person. The propensity score model included baseline covariates representing sociodemographic characteristics, clinical factors, and previous service use (for further details, see Table S1 in the online data supplement). We used a “greedy match” technique without replacement to match EPI service users with nonusers based on the year of index diagnosis and the caliper of the propensity score (27). Matching was conducted with a variable ratio of up to 1:4, and each EPI service nonuser was assigned the same admission date as each EPI service user in the matched set.

Statistical Analyses

Baseline characteristics of EPI service users and nonusers were compared using standardized differences, and values >10% were indicative of significant differences (28). There were minimal missing data (<1%) for covariates of interest, and persons with missing data were excluded.

The Cox proportional hazards model with robust variance estimators was used to account for clustering within matched sets to model access to care, suicide, and mortality indicators (binary). Modified Poisson regression was used to model indicators of intensity of service use (counts). We adjusted for any covariates that were not well balanced by the propensity score matching. Observations were censored at the end of the follow-up period, at termination of Ontario Health Insurance Plan eligibility, or in the event of death.

Results were consistent across the indicators for access to care and intensity of service use. Therefore, only the results of the access (binary) indicators are presented. All results are presented as hazard ratios with corresponding 95% confidence intervals. Analyses were conducted with SAS Version 9.4 (SAS Institute, Cary, N.C.).

Results

Study Cohort

Details on creation of the study cohort are presented in Figure 1. An additional seven matched sets were removed from the analysis due to missing admission dates, which resulted in a final matched sample of 530 EPI service users (70% of the linked sample) and 992 EPI service nonusers.

The distribution of baseline characteristics of EPI service users and nonusers is summarized in Table 1. There were substantial differences across most indicators. After matching, the groups were well balanced on sociodemographic and clinical characteristics but not on several health service indicators, and we adjusted for these indicators in the analyses.

Impact of EPI Services

Admission to the 2-year follow-up.

Data on outcome events and results from the Cox proportional hazards models are presented in Table 2. In the first 2 years following admission, EPI service users had used primary care at half the rate of EPI service nonusers (hazard ratio=0.46, 95% CI=0.41–0.52). Additionally, EPI service users were six times more likely to have had contact with a psychiatrist (hazard ratio=6.05, 95% CI=5.30–6.91) and had a more rapid time to the first psychiatric contact after program admission (user median days=13, compared with nonuser median days=78). EPI service users also had lower emergency department use (hazard ratio=0.71, 95% CI=0.60–0.83) compared with nonusers but higher rates of hospitalization (hazard ratio=1.42, 95% CI=1.18–1.71) (Figure 2). We did not find differences between the two groups regarding rates of involuntary admission (hazard ratio=1.04, 95% CI=0.88–1.22). There were no differences in rates of self-harm behavior (hazard ratio=0.86, 95% CI=0.18–4.24) or deaths by suicide (hazard ratio=0.73, 95% CI=0.29–1.80). However, EPI service users had lower rates of all-cause mortality compared with nonusers (hazard ratio=0.24, 95% CI=0.11–0.53) (Figure 3). This translates to an absolute risk reduction of 2.5% (95% CI=1.3%−3.7%) and a number needed to treat of 40 (95% CI=27–77).

2–5 years postadmission.

Data on outcome events and results from the Cox proportional hazards models are presented in Table 2. In the period between 2 and 5 years after admission, when care is typically stepped down to medical management, EPI service users continued to have an increased likelihood of contact with a psychiatrist (hazard ratio=3.24, 95% CI=2.85–3.67) compared with nonusers. EPI service users also continued to have lower rates of primary care use (hazard ratio=0.52, 95% CI=0.46–0.60) and higher rates of hospitalization (hazard ratio=2.07, 95% CI=1.58–2.72), as well as higher rates of involuntary admissions (hazard ratio=1.64, 95% CI=1.30–2.06). Although rates of emergency department use among EPI service users remained relatively stable over this period (2 years: 26.2%; 5 years: 22.6%), rates among nonusers declined substantially (from 33.8% to 18.3%), resulting in no difference in emergency department use between EPI service users and nonusers (hazard ratio=1.21, 95% CI=0.99–1.48). Rates of all-cause mortality increased among EPI services users (from <0.5% to 1.5%) and decreased among nonusers (from 3.9% to 1.7%), such that there were no longer differences between the groups (hazard ratio=1.01, 95% CI=0.48–2.12) (Table 2).

Discussion

Our analysis of population-based health administrative data suggests that users of EPI services have better outcomes than nonusers across several indicators in the larger health care system context—in particular, rapid access to psychiatric services, less use of the emergency department, and lower rates of all-cause mortality. However, EPI service users had higher hospitalization rates and lower rates of primary care visits, which has implications for service planning and collaborations between primary and specialty care providers. Additionally, many of the observed benefits did not persist in the period from 2 to 5 years postadmission, when care is typically stepped down to medical management. These changes in patterns of service use are likely a consequence of both reductions in the intensity of EPI services and improvements in the clinical status of nonusers, due to the natural trajectory of psychosis. Groups were matched on the year of index diagnosis, and for EPI service users the median time to EPI admission was 11 days. Therefore, individuals were expected to have been in the early stages of psychosis at the start of the follow-up period.

Our findings demonstrate a substantially lower risk of all-cause mortality among EPI service users compared with nonusers during the first 2 years following program admission, translating to an absolute risk reduction of 2.5% and a number needed to treat of 40 to prevent one death. It is unclear why engagement with an EPI program would lead to reductions in all-cause mortality. Increased mortality rates among persons with psychosis are well established (29), and excess mortality is even present during the early phases of psychosis. Recent U.S. data suggest that among this patient population, mortality in the first year after an incident diagnosis of psychotic disorder is at least 24 times higher than mortality in the general population (30). All mortality events in the EPI service users group were attributed to suicide, whereas only 24% of mortality events in the EPI service nonusers group were attributed to suicide. Privacy policies of the Institute for Clinical Evaluative Sciences prohibit the release of more detailed information on causes of death, and further investigation of the mechanisms by which EPI services reduce early excess mortality is warranted. Given the small number of events regarding self-harm and suicide, our analyses likely did not have sufficient power to detect significant effects for these more specific outcome measures.

Given the young age range of our study sample—nearly 90% were under the age of 40—it is likely that some of the excess mortality we observed was due to misattributed deaths by suicide. The challenges associated with the manner of death certifications for unnatural deaths are well recognized, and factors such as type of injury, evidence of suicidal intention, presence of natural disease, and time lapse between injury and death influence how a death is certified (25). The initial period following the first contact with services for psychosis is a high-risk time for patients with suicidal behavior, and death by suicide during this phase is 1.5–2 times higher than during later phases of psychosis (31). The large reduction in all-cause mortality that we observed among nonusers of EPI services after 2 years is consistent with this course.

We found that EPI service users had higher rates of hospitalization throughout the 5-year follow-up period. This contrasts with findings from a meta-analysis suggesting that EPI service use reduces both hospitalizations and total hospital days (32). Hospitalization rates have been proposed as a quality benchmark for EPI services (33), given the ease of reliable measurement and concordance with other clinical measures, such as relapse and quality of life (34, 35). However, use of hospitalization as an outcome measure has been criticized because it denotes negative connotation toward often-needed therapeutic intervention (36). Indeed, engagement with EPI services should lead to more rapid admissions for patients who need it but avoidance or delay for those who do not. In our post hoc analysis, we found a higher proportion of involuntary hospitalizations among persons who did not use EPI services (84% compared with 54%), suggesting that the ongoing monitoring and treatment of EPI service users may enable inpatient care to be provided when it is needed. Additionally, inpatient status at EPI service entry is associated with a greater number of hospital days and shorter time to rehospitalization (37), and a large proportion of EPI service users received their index diagnosis during an inpatient admission (23.6% compared with 9.0%). Therefore, we cannot rule out the possibility that residual confounding by factors such as symptom severity or willingness to accept treatment persisted between the study comparison groups. Finally, we examined a hospital-based EPI program, and this affiliation may facilitate access and increase the propensity for clinicians to initiate an inpatient admission relative to community-based service providers.

It is concerning that less than one-half of EPI service users had contact with a family physician during the first 2 years of the EPI program, and even fewer had such contact during the 2- to 5-year follow-up period. Ongoing contact with primary care is important for people with first-episode psychosis in order to manage medical comorbidities and monitor risk factors associated with psychotic disorders and antipsychotic treatment (38), such as weight gain, smoking, and sedentary behavior. Conversely, more than two-thirds of those in the nonusers group had contact with a primary care physician, indicating that family physicians likely assumed some responsibility for psychiatric care in this group. It has been argued that the EPI services sector needs to be more actively engaged with the primary care sector to ensure success (39), and efforts should be made by EPI programs to ensure that patients have regular access to a family physician and that there is ongoing collaboration and support from primary care providers.

Strengths and Limitations

Our findings are strengthened by data from a well-established EPI program spanning nearly two decades, thus providing information on the effect of EPI services on important health care system indicators in a real-world context. We used propensity score matching to improve balance in the distribution of confounding factors between groups, and we used concurrent control subjects matched on the year of diagnosis to ensure equivalency in temporal and contextual factors.

It is noteworthy that propensity score matching allows for the estimation of the average treatment effect among those who are treated (26)—that is, the average treatment effect among persons who have a similar probability of admission to EPI services. Additionally, it is important to point out that a large proportion of individuals with nonaffective psychosis in the catchment area do not come into contact with EPI services, which we have discussed in detail elsewhere (40). Furthermore, propensity score matching enables the ability to achieve balance on observed covariates and cannot account for unmeasured confounding factors. We cannot rule out the possibility that our findings are due to baseline differences in factors such as illness severity and level of family involvement in treatment. Moreover, we assessed whether the protective effect of EPI services on all-cause mortality could be due to survivor treatment-selection bias (also referred to as immortal time bias) by modeling the use of EPI services as a time-dependent exposure (41) with negligible impact on our findings (data available on request from the authors).

Although we mirrored the inclusion criteria of the Prevention and Early Intervention Program for Psychoses as closely as possible, information on some criteria—specifically the duration of previous treatment with antipsychotic medication (<1 month) and outstanding legal issues—was not available in the administrative databases. In a previous study conducted in the United States, data from electronic medical records were used to confirm cases of first-episode psychosis identified from health administrative data (41). The proportion of putative cases that were true cases of first-episode psychosis varied by clinical setting and age, with confirmation rates ranging from 78% for those from inpatient or emergency department admissions to 34% for those from primary care settings (41). In our study, approximately 40% of EPI service nonusers did not have their diagnosis confirmed by a psychiatrist or during an inpatient admission at any point in their medical records. Consequently, a subset of our comparison group likely would not meet inclusion criteria for EPI programs. We attempted to reduce the effect of this by propensity score matching to select control subjects who had a similar probability of admission to an EPI program.

We did not have information pertaining to the date on which a patient was discharged or disengaged from the program, and thus misclassification of exposure was possible. However, this would underestimate the benefits of EPI services on the outcomes of interest. We were limited by information available in the health administrative data and were unable to capture use of nonmedical services, such as community-based mental health organizations, private psychologists, and other social services.

Finally, we lost approximately 30% of our sample due to an inability to find a suitable match, and this affects the generalizability of our findings. Unmatched EPI service users differed on several sociodemographic factors but did not differ on outcome measures relative to the matched EPI service users (for further details, see Table S3 in the online data supplement).

Conclusions

Using population-based health administrative data spanning nearly two decades, we demonstrated that users of EPI services had substantially lower mortality rates than persons with first-episode psychosis who did not use these services. EPI service use was also associated with better outcomes across several health care system indicators in the first 2 years after admission. However, there was no significant advantage for EPI service users in the period between 2 and 5 years postadmission, when care is typically stepped down to medical management. Overall, our findings support the effectiveness of EPI services for the treatment of first-episode psychosis in the larger health care system context.