Validity of reaction to severe stress and adjustment disorder diagnoses in the Danish Psychiatric Central Research Registry

Introduction

Since the population-based medical registries, including the Danish Psychiatric Central Research Registry (DPCRR),¹ were established primarily for administrative purposes, understanding the validity of the diagnostic data included within them is critical to the interpretation of epidemiological research using these data.^2,3 Various diagnoses within the DPCRR have been the subject of validity studies;^4–6 studies of affective disorders, autism, and schizophrenia have all been able to confirm over 90% of registry cases using various methods for validation including review of case records and computerized diagnosis.^4–6 Although this evidence regarding the validity of the registry diagnoses is promising, investigation of additional diagnoses contained in the register is warranted.² In fact, a recent editorial highlighted the need for more research aimed at examining the validity of diagnoses in this widely used registry.⁷

Previously, we reported on the creation of a national longitudinal cohort of 101,663 Danish-born adult residents of Denmark who received a reaction to severe stress and adjustment disorder diagnosis between 1995 and 2011,⁸ and the corresponding incidence, as recorded in the DPCRR.¹ Reaction to severe stress and adjustment disorders (as classified by the International Classification of Diseases, 10th Edition [ICD-10])⁹ is associated with psychiatric comorbidities, negative physical health consequences, and suicidal behaviour.^10–15 There have been few population-based registry studies of reaction to severe stress and adjustment disorders, with the exception of posttraumatic stress disorder (PTSD). Objections to the use of registries for the study of these disorders include the difficulty of assessing the associated stressor or trauma and the possible inclusion of only severe cases in the registries. Thus, the aims of the current study were to assess the validity of the severe stress and adjustment disorder diagnoses registered in the DPCRR, to examine the documentation of stressful and traumatic events in the medical records, and to investigate the occurrence of stress diagnoses in primary practice among persons not registered in the DPCRR.

Materials and methods

Denmark provides tax-funded universal access to primary and secondary health care, with no out-of-pocket expenses. Individual-level data from all Danish registries can be linked via the unique personal identifier (CPR number) assigned at birth by the Danish Civil Registration System (CRS).¹⁶

We identified all patients with incident reaction to severe stress or adjustment disorder diagnoses between January 1st, 1995 and December 31st, 2011 based on the DPCRR to create the stress diagnosis cohort. The DPCRR includes data on psychiatric inpatient diagnoses since 1977, and outpatient diagnoses since 1994.¹ Since 1970, it has been mandatory for all psychiatric departments and mental hospitals to report all psychiatric admissions to the registry. The records contain the CPR number, treatment dates (start and stop), and up to 20 diagnoses. The diagnoses have been coded according to ICD-10 since 1994.⁹

The diagnoses we studied include: acute stress reaction (ICD-10 code: F43.0); PTSD (ICD-10 code: F43.1); adjustment disorder (ICD-10 code: F43.2); other reactions to severe stress (ICD-10 code: F43.8); and reaction to severe stress, unspecified (ICD-10 code: F43.9/F43).

The characteristics of the patients in the cohort are described elsewhere,⁸ but, in brief, the sex composition of the stress diagnosis cohort was 60% females, and diagnoses were most frequent in the late teens through early 30s. Based on the CRS,¹⁶ an age- and sex-matched comparison cohort of Danish-born residents of Denmark who had not received a reaction to severe stress and adjustment disorder diagnosis in the DPCRR at the time of their matched stress cohort case’s diagnosis was also created. The characteristics of the comparison cohort are described elsewhere.¹⁷

Validation process in DPCRR

From the stress diagnosis cohort, we selected 50 cases each from two hospitals in the Northern Denmark region (100 patients total) over 15 years of age for validation. Consistent with the larger stress cohort, the sex distribution of the selected validation cases was 60% females and 40% males. For hospital 1 an outpatient clinic was chosen, while in hospital 2 we used an inpatient department. We used a computer-generated random selection to choose one patient below and one above median age for each diagnosis (F43.0, F43.1, F43.2, F43.8, F43.9/F43) for five time periods within the study period (1995–1997, 1998–2000, 2001–2003, 2004–2006, 2007–2011), resulting in the validation of 20 medical records for each diagnosis. We found 43 patients fulfilling these criteria in the outpatient clinic at hospital 1; the remaining seven patients were selected from an inpatient clinic at that same hospital. Forty-four patients fulfilled the criteria at the inpatient clinic at hospital 2; the remaining six patients were selected from the outpatient clinic at that hospital.

A postgraduate epidemiologist (ES) with training in psychiatric epidemiology systematically reviewed the medical records. All symptoms, based on the ICD-10 Diagnostic Criteria for Research manual,⁹ were translated to Danish and further recorded in a standardized form in EpiData (EpiData Association, Odense, Denmark). EpiData is an open source program for data entry and documentation, which allows for programming of acceptable responses, making the entry of invalid values impossible. All forms and the content were back-translated to English. A postgraduate epidemiologist (JLG) with training in psychiatric epidemiology went through all records, and determined whether 1) the diagnosis in the registry and the medical records were consistent and 2) whether the symptoms recorded in the medical records met criteria for the given diagnosis, according to ICD-10 diagnostic criteria.⁹ For each patient, we also entered up to ten stressful or traumatic events and the date they occurred.

Occurrence of stress diagnoses in primary care

Each person in Denmark chooses or is assigned to the list of a Danish general practitioner (GP)’s practice, registered in the Danish National Health Services Register.¹⁸ To investigate the occurrence of stress diagnoses among persons not registered in the DPCRR, we identified two single-practice GPs having over 50 patients in the age- and sex-matched comparison cohort registered on their list through the Danish National Health Services Register.¹⁸ We randomly selected 50 patients who had been on the GPs list over 5 years, and sent a form to the GPs that inquired about the history of these specific patients with regard to stress diagnoses (see Table S1).

Statistical analysis

We calculated the positive predictive value (PPV) and corresponding 95% confidence intervals of the reaction to severe stress and adjustment disorder diagnoses registered in the DPCRR, using medical records as a reference standard. The PPV was calculated separately for each of the five diagnoses and defined as the number of patients with a confirmed diagnosis in both the stress cohort and in the medical records divided by the total number of patients with a given diagnosis in the stress cohort that were able to be validated against medical records. To examine variability in diagnoses based on location of the validation (ie, inpatient versus outpatient records), we also examined PPV stratified by hospital.

For the patients with consistent diagnosis in the DCPRR and in the medical records, we further determined the proportion of patients who have symptomatology noted in the records that is consistent with the full diagnostic criteria for the assigned diagnosis. Because the ICD-10 Diagnostic Criteria for Research⁹ does not outline specific diagnostic symptom criteria for other reactions to severe stress or reaction to severe stress, unspecified, these analyses were restricted to acute stress reaction, PTSD, and adjustment disorder.

We examined the frequency of traumatic or stressful life events described in the medical records by diagnosis, overall and for the patients with concordant diagnosis in the medical records and in DCPRR. The current paper reports on the most severe life event with the closest temporal proximity to diagnosis. Finally, to assess the completeness of the stress diagnosis cohort with regard to diagnoses that were not recorded in the DPCRR due to patients receiving their diagnosis and treatment from a GP, private psychologist or psychiatrist only, and not a public psychiatric specialist, we examined patients with reaction to severe stress and adjustment disorder diagnosis as reported by the GPs.

This study was approved by the Institutional Review Board at Boston University, the Danish Health and Medicines Authority (journal number 3-3013-244/1/), and the Danish Data Protection Agency (journal number 2012-41-0841).

Results

Validity of F43 diagnoses in the DCPRR

In total, we were able to locate medical records for 91 of 100 of the selected validation cases. We could not retrieve five records from hospital 1 (one acute stress reaction patient, one PTSD patient, one patient with adjustment disorder, and two cases with other reactions to severe stress). From hospital 2, we could not retrieve four records diagnosed primarily before 2005 (one patient with PTSD, one patient with adjustment disorder, one patient with other reactions to severe stress, and one person with reaction to severe stress, unspecified). At hospital 1, all of the medical records were in paper form for the years under study, and at hospital 2, the records were in paper form before 2005. Therefore, missing records were attributed primarily to misplacement of paper records.

Table 1 displays the overall results of the examination of PPV by disorder, and stratified by validation hospital. Of the 19 medical records we reviewed for patients with acute stress reaction, we were able to confirm that eleven had this diagnosis described in their record, yielding an overall PPV of 58%. Among the eleven patients whose acute stress reaction diagnosis could be confirmed with the available data in the medical records, eight were determined to have at least some symptoms consistent with acute stress reaction (Table 2). With regard to PTSD, we were able to confirm the diagnosis in 15 of the 18 records we reviewed, for a PPV of 83%. Of these, 13 patients had at least some symptoms consistent with PTSD. Eighteen records were reviewed for adjustment disorder diagnoses, and, of those, we were able to confirm 17 diagnoses, yielding a PPV of 94%. Sixteen of these patients had symptoms consistent with an adjustment disorder diagnosis noted in their medical record. We reviewed 17 records for patients diagnosed with other reactions to severe stress, and, of these, 12 were able to be confirmed (PPV =71%). Finally, with regard to reaction to severe stress, unspecified, we reviewed 19 records and were able to confirm the diagnoses in 13, resulting in a PPV of 68%.

Table 1 Validity of ICD-10 reactions to severe stress and adjustment disorder diagnoses in the DPCRR as compared with medical records, 1995–2011 Abbreviations: CI, confidence interval; DPCRR, Danish Psychiatric Central Research Register; ICD-10, International Classification of Diseases, 10th Edition; PPV, positive predictive value; PTSD, posttraumatic stress disorder.

Table 2 Fulfillment of ICD-10 criteria for reaction to severe stress and adjustment disorder symptom diagnosis noted in the medical records* Note: *Among the records with concordant journal and registry entry. Abbreviations: ICD-10, International Classification of Diseases, 10th Edition; PTSD, posttraumatic stress disorder.

We identified some cases of registration error (n=9), where a different diagnosis than the reaction to severe stress and adjustment disorder was assigned by the treating psychiatrist, and should have been reported to the DPCRR instead of F43x. All of these participants had a mood disorder diagnosis recorded in their medical record, with the most common being depressive episode (F32.x). The majority of participants with a diagnosis outside of the category of reaction to severe stress and adjustment disorder in their medical record had a diagnosis of reaction to severe stress, unspecified in the DPCRR.

Traumatic/stressful events observed

Table 3 lists the stressful and traumatic events recorded in the medical records among the 91 validation cases, by diagnosis. Overall, 81% had a stressful event noted. The corresponding figure was 79% (15/19) for patients with acute stress reaction, 100% for PTSD, 83% (15/18) for adjustment disorder, 71% (5/17) for other reactions to severe stress, and 74% (5/19) for reaction to severe stress, unspecified.

Table 3 Traumatic/stressful life events experienced by the 91 patients with reaction to severe stress or adjustment disorder according to the Danish Psychiatric Central Research Registry Abbreviations: ICD-10, International Classification of Diseases, 10th Edition; PTSD, posttraumatic stress disorder.

The most common stressful life event among the patients with acute stress reaction was a divorce or the ending of a relationship, experienced by 53% (10/19). For patients with PTSD, the most common stressful life events were abuse, neglect, and other maltreatment (including incest and rape) (39%, 7/17), and professional hazards (including firefighters, burglary at work, and military personnel) (28%, 5/17). Over half the patients with adjustment disorder had experienced either divorce or end of a relationship (28%, 5/19) or death of a close relative, including traumatic deaths (28%, 5/19). The traumatic events observed for patients with the unspecified diagnoses (F43.8 and F43.9) spanned from relationship end and friend-related problems (including loss of friends and being bullied) to severe events, such as a car accident resulting in death, and abuse, neglect, and other maltreatment. In examining only the medical records where a concordant diagnosis with registry data was found, 90% had a stressful life event noted. The types of events were consistent with overall results (Table 3).

Occurrence of diagnoses in primary care

Of the 100 patients from the comparison cohort reviewed by a GP for a diagnosis of reaction to severe stress and adjustment disorder, we observed seven patients with this diagnosis. The first GP reported one patient with adjustment disorder (diagnosed 2007, treated by private psychologist/psychiatrist) and one with acute stress disorder (diagnosed 2011, treated by private psychologist/psychiatrist). The second GP reported diagnosing and treating three patients with adjustment disorder (in 2005, 2006, and 2011). One additional patient was diagnosed with reaction to severe stress, unspecified (2008), and treated by a private psychologist. Information on the last patient was not available, because the patient had changed his or her GP.

Discussion

This study is the first to assess the validity of all diagnoses among this ICD-10 category of reaction to severe stress and adjustment disorder in the DCPRR by comparing registry data and medical records. The more severe diagnoses (eg, PTSD) showed good validity when compared to medical records at two hospitals, while the more transient and/or unspecified diagnoses (eg, acute stress reaction, other reactions to severe stress, and reaction to severe stress, unspecified) had poorer results, particularly at the hospital where mostly outpatient records were reviewed.

We have only identified one other validation study on this group of diagnoses. The study by Gravely et al assessed the validity of PTSD diagnoses in US Department of Veterans Affairs administrative data, and found a PPV of 75%.¹⁹ We found a higher PPV within the Danish registry data, which may be explained by comparing registry data with medical records rather than self-report, as done in the Gravely et al study. Our finding of high validity for adjustment disorder and PTSD in the DPCRR fits with previous reports of high validity in this registry for other disorders.^4–6

When examining symptomatology noted in the medical records among those with a consistent diagnosis in both the registry and the medical record, we found that patients with adjustment disorder frequently had symptomatology reported in the records that fulfilled the criteria for the diagnosis. Interestingly, for acute stress reaction and PTSD, most patients had symptoms noted in their record that were consistent with their assigned diagnosis, but not enough symptoms were noted to meet full diagnostic criteria. Psychiatric diagnosing involves complex symptomatology, potentially experienced over long periods of time, and symptoms that result in diagnoses may be observed by clinicians or reported by patients but not fully documented in the medical record. These results, particularly with regard to acute stress reaction and PTSD, highlight the need for validation studies of these disorders that examine medical records across multiple sessions of assessment and treatment to fully capture all symptomatology that may result in a given diagnosis. The current study also found that, in over 80% of the medical records, a stressful or traumatic event was noted – a useful resource for future work.

Based on validation of 100 patients without a severe stress or adjustment disorder diagnosis in the DPCRR who had been on a GP list for more than 5 years during 1995 to 2011, we only found seven cases of the milder or transient forms of these disorders. This highlights the comprehensive nature of the DPCRR, particularly with regard to more severe diagnoses following a stressful or traumatic event. However, it is important to be aware that there is a small proportion of the milder or transient forms of these disorders that are treated in general practice or by private psychologists/psychiatrists only.

A major strength of the study is the ability to link information on the entire Danish population by using the Danish CPR number, including DRCPP, medical records, GP records, and other population-based patient information. Bearing in mind practicality and feasibility and the limited resources of the project, we randomly selected medical records at two public psychiatric hospitals in Denmark, selecting cases according to sub-diagnosis and time of diagnosis, in one of Denmark’s five regions. We were able to select medical records from 1995 to 2011, encompassing the entire timeframe of the established research cohorts. All in- and outpatient psychiatric hospital visits are registered in the DPCRR. We assumed that treatment was comparable across regions, thus the representativeness depends on whether our assumption holds. Future studies should include a more expansive validation, encompassing the entire country.

A few limitations should be kept in mind when interpreting our results. Clinicians make varying determinations regarding the importance of documenting specific symptoms in a medical record, which may have inhibited our ability to confirm some diagnoses, stressful events, or traumas. Considerable variation in detail among the different medical records was noted. Additionally, reviewers of the medical records and of the reliability and accuracy were not blinded to the relevant register data, possibly introducing bias in the process of documenting evidence in the chart review and in the symptom evaluation. We were unable to report specificity and sensitivity, because we sampled on the basis of the registered diagnosis. We were also unable to assess validity over time, because we only had four records per diagnosis per time. Two GPs were selected to minimize reporting bias. Due to limitations in resources, we were only able to examine 100 cases in primary practice in the examination of occurrence of the diagnoses not registered in the DPCRR; thus, future large-scale studies are needed to add to the knowledge base about this.

Conclusion

The thorough evaluation of medical records, stressful and traumatic events identified, and the high PPV found in the current study support the DPCRR as a valid and comprehensive resource for future epidemiologic research on reactions to severe stress and adjustment disorders, particularly research aimed at examining PTSD and adjustment disorder.

Acknowledgments

The authors acknowledge the assistance of medical secretaries at the two hospitals, and the two general practitioners for their thorough evaluation of records. This work was funded by a National Institute of Mental Health grant: “Constructing a Danish Reaction to Severe Stress Cohort” (1R21MH094551-01A1), Jaimie Gradus, Principal Investigator.

Disclosure

The authors report no conflicts of interest in this work.

---

References

1.	Mors O, Perto GP, Mortensen PB. The Danish Psychiatric Central Research Register. Scand J Public Health. 2011;39(7 Suppl):54–57.
2.	Byrne N, Regan C, Howard L. Administrative registers in psychiatric research: a systematic review of validity studies. Acta Psychiatr Scand. 2005;112:409–414.
3.	Allebeck P. The use of population based registers in psychiatric research. Acta Psychiatr Scand. 2009;120:386–391.
4.	Kessing L. Validity of diagnoses and other clinical register data in patients with affective disorder. Eur Psychiatry. 1998;13:392–398.
5.	Lauritsen MB, Jørgensen M, Madsen KM, et al. Validity of childhood autism in the Danish Psychiatric Central Register: findings from a cohort sample born 1990–1999. J Autism Dev Disord. 2010;40:139–148.
6.	Uggerby P, østergaard SD, Røge R, Correll CU, Nielsen J. The validity of the schizophrenia diagnosis in the Danish Psychiatric Central Research Register is good. Dan Med J. 2013;60:A4578.
7.	Mors O. [Validity of diagnoses in the Danish Psychiatric Central Research Register]. Ugeskr Laeger. 2013;175(20):1399. Danish.
8.	Gradus JL, Bozi I, Antonsen S, et al. Severe stress and adjustment disorder diagnoses in the population of Denmark. J Trauma Stress. 2014;27:370–374.
9.	World Health Organization. The ICD-10 Classification of Mental and Behavioural Disorders: Diagnostic Criteria for Research. Geneva: World Health Organization; 1993.
10.	Gradus JL, Qin P, Lincoln AK, Miller M, Lawler E, Lash TL. The association between adjustment disorder diagnosed at psychiatric treatment facilities and completed suicide. Clin Epidemiol. 2010;2:23–28.
11.	Gradus JL, Qin P, Lincoln AK, et al. Acute stress reaction and completed suicide. Int J Epidemiol. 2010;39:1478–1484.
12.	Gradus JL, Qin P, Lincoln AK, et al. Posttraumatic stress disorder and completed suicide. Am J Epidemiol. 2010;171:721–727.
13.	Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:617–627.
14.	Qureshi SU, Pyne JM, Magruder KM, Schulz PE, Kunik ME. The link between post-traumatic stress disorder and physical comorbidities: a systematic review. Psychiatr Q. 2009;80:87–97.
15.	Casey P. Adjustment disorder: epidemiology, diagnosis and treatment. CNS Drugs. 2009;23:927–938.
16.	Schmidt M, Pedersen L, Sørensen HT. The Danish Civil Registration System as a tool in epidemiology. Eur J Epidemiol. 2014;29(8):541–549.
17.	Gradus JL, Antonsen S, Svensson E, Lash TL, Resick P, Hansen JG. Trauma, comorbidity, and mortality following severe stress and adjustment disorder diagnoses: a nationwide cohort study. AJE. In press 2015.
18.	Andersen JS, Olivarius Nde F, Krasnik A. The Danish National Health Service Register. Scand J Public Health. 2011;39(7 Suppl):34–37.
19.	Gravely AA, Cutting A, Nugent S, Grill J, Carlson K, Spoont M. Validity of PTSD diagnoses in VA administrative data: comparison of VA administrative PTSD diagnoses to self-reported PTSD Checklist scores. J Rehabil Res Dev. 2011;48:21–30.

---

Supplementary material

Table S1 The validation form assessing the presence of reaction to severe stress and adjustment disorder in 100 patients in two general practitioner practices Abbreviation: PTSD, posttraumatic stress disorder.