https://orcid.org/0000-0003-0351-6216
Figures
Abstract
Currently health care pathways (the combination and order of services that a patient receives to manage their injury) following a mild traumatic brain injury vary considerably. Some clinicians lack confidence in injury recognition, management and knowing when to refer. A clinical expert group developed the Brain Injury Screening Tool (BIST) to provide guidance on health care pathways based on clinical indicators of poor recovery. The tool aims to facilitate access to specialist services (if required) to improve longer term prognosis. The tool was developed using a three-step process including: 1) domain mapping; 2) item development and 3) item testing and review. An online retrospective survey of 114 adults (>16 years) who had experienced a mild brain injury in the past 10 years was used to determine the initial psychometric properties of the 15-item symptom scale of the BIST. Participants were randomised to complete the BIST and one of two existing symptom scales; the Rivermead Post-concussion Symptom Questionnaire (RPQ) or the Sports Concussion Assessment Test (SCAT-5) symptom scale to determine concurrent validity. Participant responses to the BIST symptom scale items were used to determine scale reliability using Cronbach’s alpha. A principal components analysis explored the underlying factor structure. Spearman’s correlation coefficients determined concurrent validity with the RPQ and SCAT-5 symptom scales. The 15 items were found to require a reading age of 6–8 years old using readability statistics. High concurrent validity was shown against the RPQ (r = 0.91) and SCAT-5 (r = 0.90). The BIST total symptom scale (α = 0.94) and the three factors identified demonstrated excellent internal consistency: physical/emotional (α = 0.90), cognitive (α = 0.92) and vestibular-ocular (α = 0.80). This study provides evidence to support the utility, internal consistency, factor structure and concurrent validity of the BIST. Further research is warranted to determine the utility of the BIST scoring criteria and responsiveness to change in patients.
Citation: Theadom A, Hardaker N, Bray C, Siegert R, Henshall K, Forch K, et al. (2021) The Brain Injury Screening Tool (BIST): Tool development, factor structure and validity. PLoS ONE 16(2): e0246512. https://doi.org/10.1371/journal.pone.0246512
Editor: Firas H. Kobeissy, University of Florida, UNITED STATES
Received: November 2, 2020; Accepted: January 20, 2021; Published: February 4, 2021
Copyright: © 2021 Theadom et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: There are ethical restrictions on sharing of de-identified data for this study. The ethics committee has not agreed to the public sharing of data as we do not have the participants’ permission to share their anonymous data. Further, the dataset includes sensitive injury cases and descriptions of the mechanism of injury. It is likely given the nature of the dataset that patients may still be identifiable despite efforts to anonymise the data. Qualified and interested researchers may request access to the data by contacting Professor Kate Diesfield of the Auckland University of Technology Ethics Committee (Phone +64 9 921 9999 extn: 7837, Email ethics@aut.ac.nz) ethics reference number 20/121.
Funding: The TBI Network at the Auckland University of Technology provided reimbursement to participants. No additional external funding was received for this study.
Competing interests: "The authors of this paper have read the journal’s policy and have the following competing interests: Kris Fernando is a paid employee at Active+ (https://www.activeplus.co.nz/). Mark Fulcher is a paid employee at Axis Sports Medicine (https://www.axissportsmedicine.co.nz/). Sam Jewell is a paid employee at Wellington Sports Medicine (https://wellingtonsportsmed.co.nz/). There are no patents, products in development or marketed products associated with this research to declare. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Traumatic brain injury occurs where there is an external force causing an alteration in brain function [1]. One of the critical acute decisions clinicians need to make, is to determine the risk of bleeding and swelling in the brain following injury. This includes establishing if any of a range of clinical indicators for further diagnostic imaging are present. Clinical indicators include repeated vomiting, worsening symptoms, age over 65 years, suspicion of skull fracture, severe headache, more than brief loss of consciousness, use of coagulation impacting therapies and post-injury seizure [2].
Many people recover well in the days to weeks following mild traumatic brain injury (mTBI), however, over 40% can experience persistent symptoms that impact on daily living for many years [3–5]. Research has shown that risk factors for poor recovery outcome include a history of (previous) TBI, a pre-existing mental health condition, delayed medical attention (including patient delays in seeking treatment), older age, maladaptive coping, and increased severity of acute symptoms [6, 7]. There is increasing evidence that early identification and good recovery advice improves longer term recovery [8–10]. However, currently access to specialist services requires the patient to know to go back to their General Practitioner (GP) if they do not recover spontaneously. Further the GP needs to be aware of available services and processes to support a patient’s recovery. The current pathway can create significant barriers or delays in treatment.
Given up to 95% of TBIs are classified as being mild in severity, many injuries can be managed outside of the hospital context. However, currently treatment can vary widely both within and between countries and can often depend on the experience and expertise of the treating clinician on initial medical presentation [11]. There are a number of clinical indicators and risk factors for clinicians to consider as part of decision making on the appropriate health care pathway for a patient. It can be challenging for clinicians to be aware of all the specific indicators particularly if a clinician is less experienced or does not see many mTBI’s within their clinical practice. Indeed, studies have shown that many primary care practitioners lack confidence in recognising and managing mTBI effectively and knowing when to make referrals to specialist services [12]. There are also increasing calls to support other health professionals such as physiotherapists, occupational therapists and nurse practitioners who frequently assist in recognition of possible mTBI’s [13].
Tools designed to try and subclassify mTBI’s based on injury type, level of consciousness and alterations in mental state post-injury have not been found to adequately discriminate mTBI’s or to predict how a person will recover [14]. Consequently, the acute assessment now focuses on a more comprehensive assessment of acute symptoms such as the severity of headaches, dizziness and disturbed vision. Two of the most commonly utilised symptom scales include the Sports Concussion Assessment tool (SCAT 5) [15] and the Rivermead Post Concussion Scale (RPQ) [16]. The SCAT-5 was designed specifically for the sports context to assist in concussion recognition and diagnosis. For example, the memory questions ask ball sport-specific questions such as “which half is it now” which need to be adapted for different sports. The tool is widely used across different sports across the globe and has continued to be refined over five versions to improve clinical utility. However, there is less evidence for applicability of use outside of the sports context. An international review revealed that 40/67 (60%) countries have primary care consultations with an average duration of 10 minutes or less [17]. Yet it is acknowledged that the SCAT-5 cannot be performed correctly in less than 10 minutes [15] making its utility within a primary care context challenging. The psychometric properties of the SCAT-5 have also been found to be poor [18] and its authors have acknowledged its limited role in tracking recovery and assisting the return to play/sport decision [15]. Additionally, there is little support for the SCAT-5 as a unidimensional measure of severity [19]. Symptoms scales, such as the RPQ [16] were designed for the research context to assist in determining the type and frequency of symptoms experienced following a TBI. However, the underlying factor structure of the RPQ has been found to vary considerably making it difficult to use total or subscale scores in outcome prediction and it has limited utility on clinical pathway decision making [19, 20].
There is also evidence that dominant symptom clusters, such as high physiological symptom reporting, may be additional predictors of prolonged recovery [21]. Particular symptom clusters may also indicate where specific professional input is needed early in treatment (e.g. a referral for neurophysiotherapy) [21–23]. Use of different tools and concussion policies across different situational contexts and sports is confusing and inefficient for clinicians and patients alike increasing variation in health care delivery. Consequently, there is a need for consistent, equitable access to concussion recognition and best practice management.
The aims of this study were to develop and validate a short, brain injury screening tool for use at initial assessment that could be utilised:
- across a diverse an age range;
- for injuries sustained through many situational contexts (e.g. sport, school, work, violence or everyday living activities);
- across varied health care settings (e.g. general practice, urgent care or Accident and Medical Setting, allied health clinics, physiotherapy);
- without the need for specialist training; and
- to provide guidance to clinicians on initial health care pathways.
Materials and methods
A clinical expert group (CEG) was established comprised of general practitioners, sport and exercise medicine physicians, physiotherapists, trauma nurses, psychologists, TBI researchers and service funding representatives. A three step process was undertaken to develop the BIST drawing on the methods used by Seela et al [24] and outlined in Table 1.
Following the CEG’s review of the existing literature and assessment tools against the identified needs and purpose (Table 1), a number of key limitations were identified. Current symptom scales often include complex terms (e.g. nausea and noise sensitivity) that may make accurate reporting for patients challenging. This is a particular issue for people whose first language is not English. Existing tools also do not support clinical decision making as to how the patient should best be managed across all health care pathways. For example, determining whether a patient should be referred to the emergency department, refer for early specialist treatment (if they are at risk of long-term problems) and those who can be monitored and managed in primary care. The time taken to administer some tools such as the SCAT-5 was also highlighted as a limitation on search for a measure for use. Based on the identified constraints the CEG determined the need for development of a new symptom measure called the Brain Injury Screening Tool (BIST).
To develop the BIST, the CEG debated the most important symptoms that would need to be considered clinically at the initial stage to determine care pathways for the patient e.g. to determine their level of risk of acute complications/sequelae (high risk patients requiring urgent hospital care), those at risk of longer-term problems (medium risk) needing early support and those likely to recover with monitoring and initial advice (low risk). One of the operational considerations identified was that as many symptoms cannot be observed until at least 24 hours after an injury (e.g. sleep quality), these symptoms should be made optional depending on the time since injury that a person presents for medical treatment. There were 11 items identified by the CEG that could be assessed within 24 hours of injury and an additional four items that could be assessed if the person presented 24 hours post-injury. The additional items were included to allow a more comprehensive symptom assessment and to monitor recovery. The draft BIST was taken out for consultation with practitioners outside of the CEG working in clinical settings, to obtain initial feedback and to identify and operational barriers to potential implementation. Feedback included the need to ensure that national criteria for neuroimaging and hospital review was captured by the tool, which was subsequently undertaken, and additional screening questions added. The final tool is available from https://tbin.aut.ac.nz/.
Procedure for the validity study
To determine initial validity of the tool a cross sectional study of people following a mild TBI was completed. Permission to conduct this study was obtained from the Auckland University of Technology Ethics Committee (AUTEC reference number– 20/121). Adults aged over 16 years, who had experienced at least one TBI in the past 10 years were recruited via social media, concussion clinics and sports organisations over a 4-month period between May and September 2020. Adults interested in taking part were able to access a weblink that took them to a Qualtrics website that provided information about the study and what would be involved. Participants were asked to read the consent form and then if they wished to take part to indicate their consent in the study by ticking a box agreeing to take part in the study (online consent). The participants were then able to proceed with answering the questionnaire online including sociodemographic characteristics and about their brain injury history. They were then randomised within the Qualtrics survey to receive one of the following combinations of assessments; the BIST and the symptom scale of the SCAT-5, or the BIST and the RPQ to determine concurrent validity. At the end of the survey, participants were asked to comment on how they found completing the BIST. Those who completed the questionnaire were then invited to enter a prize draw for one of three $100 fuel vouchers. If interested, participants were taken to a separate questionnaire to enter in their contact details to protect their confidentiality, which was drawn following completion of data completion. Contact details were then deleted.
Measures
The BIST consists of two components. The first component comprises of nine questions used to determine if a patient is at ‘high risk’ via a description of what happened and specific questions aiming to identify any ‘red flags’ or clinical indicators suggesting that the person may need an urgent referral to hospital (e.g. repeated vomiting). The second component includes a 15-item symptom report scale. People are asked to rate how much they now experience the symptoms listed on a scale of 0 (not at all) to 3 (severe). The BIST was initially designed for those aged 8 years and over and to have a clinical conceptual framework of five subscales: physical, vestibular, cognitive, emotional and sleep.
The SCAT-5 [15] was designed to support the assessment of suspected concussion within the sports context in those aged over 13 years. A child version of the SCAT-5 is available for those aged 5–12 years old. It can also be used as a baseline measure with athletes to enable comparison with post-injury responses. The SCAT-5 consists of a short relevant medical history, observable signs, a cognitive and neurological screen, and symptom evaluation and is repeated over time to monitor recovery in sport to support diagnosis of concussion. It can only be administered by a trained healthcare professional. The symptom scale consists of 22 symptoms rated on a scale of 0 (none) to 6 (severe). The SCAT-5 has demonstrated high test retest reliability [25] and adequate internal consistency [26].
The RPQ [16] assesses the presence and severity of 16 symptoms commonly experienced following a brain injury compared to pre-injury. Participants rate each symptom using a five-point scale ranging from 0 (absent) to 4 (severe problem). The RPQ was designed for adults following mild to moderate brain injury aged 18 years plus. The RPQ has demonstrated excellent test re-test reliability [16] and internal consistency [27].
Statistical analysis
Individual participant feedback comments were explored, and any suggested modifications identified and listed. Frequencies were used to determine proportion of missing data on each of the 15 symptom items. Scale reliability was assessed using Cronbach’s alpha (α) with a value of 0.70 considered the minimally acceptable level of reliability. The number of factors underpinning the BIST were determined by a scree plot and principal components exploratory factor analysis. As the factors were expected to be correlated, a direct oblimin rotation was used that only included items with eigenvalues >1. Spearman’s correlation coefficients were used to determine concurrent validity between responses on the BIST and the RPQ and SCAT-5.
Results
Of the 163 adults who consented to take part in the study, 118 (72.4%) completed the questionnaire. Data were then checked against the inclusion criteria to determine eligibility; four participants were excluded as the date of the last TBI was >10 years ago. Data were analysed for the final sample of 114 participants. A minimum of 75 participants were required to ensure sufficient sample size for this analysis (e.g. a minimum of five people per item of the BIST—which consisted of 15 items) [28]. Average time to complete the full questionnaire (including both the BIST and one other measure) was 7.4 (± 115.5) minutes.
Participants ranged in age between 16 and 72 years, with a mean age of 32.4 years (± 13.6). The average time since last TBI was 2.1 years (± 2.29). Table 2 summarises the characteristics of the participants included in the analysis.
Usability of the BIST
There were only five items (33%) for which at least one data point was missing. There were no specific questions identified with high levels of missing data indicating a problem with that item. Only one or two people omitted data on these five items and the data were therefore deemed to be missing at random. Feedback from adult participants supported the readability and applicability of the items with comments including “easy for people to understand”, “very easy to follow”, “easy to complete” “nice and simple”, and “good questions that felt relevant”.
One clarification was highlighted by participants about whether they should report their symptom experience for right now (at this point in time), when the injury happened or when symptoms were at their worst? The instructions on the tool were amended from “Compared to before your accident, do you have any of the following?” to “Compared with before the accident, please rate how much you experience the following right now (at this point in time)”. One participant highlighted that changes in her emotional reactions were not captured by the existing item focusing on being annoyed “my main issue is that I anger quicker and have less control over my emotions.” Discussion of this item with clinicians also highlighted that term “annoyed” may be too broad and could reflect a number of issues not necessarily related to the injury. The item “I get easily annoyed” was changed to “I get angry quicker”.
Discussions with clinicians and service providers on the 0–3 response scale indicated that a 0–10 scale would be better due to it being in line with clinical advice which advocates monitoring of symptoms on a 0–10 scale and to enable smaller changes in symptoms to be detected over time. It was also suggested there was a need to be clearer about which criteria put patients as high, medium, or low risk. An additional question on the impact of symptoms on the person and their life was also proposed as a more global measure of impact, and as symptoms do not always correlate with the impact on a person’s life.
Readability statistics
Based on a combination of seven readability statistics (including a Flesch Reading Ease score = 96.5, Gunning Fog = 3.7, Flesch-Kincaid Grade Level: 1.2 First Grade, The Coleman-Liau Index: 5, Fifth Grade, The SMOG Index: Third Grade, Automated Readability Index: -0.4, 3–5 yrs. old (Preschool) and Linsear Write Formula: 2, Second Grade) the 15 BIST items were classified overall as ‘very easy to read” with a reading age of 6–8 years old [29].
Internal consistency
Total scores on the BIST symptom scale ranged between 0–44, with a median score of 15 (interquartile range of 15.75). The Cronbach’s alpha for the 15-item symptom scale of the BIST was 0.94, indicating excellent scale reliability. This suggests that the 15-items of the BIST are closely related and are measuring the same construct. This compares favourably with previous evidence for both the RPQ (0.95) [27] and the SCAT (0.94) [18].
Underlying factor structure of the BIST
The data were found to be suitable for factor analysis with a Kaiser-Meyer-Olkin measure of sampling adequacy of 0.88. As shown in Fig 1, the scree plot indicated a two or three factor structure of the BIST.
Looking at the output from the principal component analysis, an item was assumed to load on a given factor if the factor loading was at least 0.40. There was moderate correlation between the items supporting a link to the same underlying construct but limited overlap. There were no items identified as not contributing significantly to the overall measure as shown in Table 3.
As shown in Table 4, the proposed component structure of the BIST based on clinical relevance was well supported. The relevant items in the proposed vestibular-ocular and cognitive components strongly loaded on to the equivalent factors. The items in the physical and emotional and sleep components were found to load onto the same single factor (renamed physical and emotional). Loadings below 0.30 were removed from the table.
A reliability index was calculated to assess the internal consistency of the identified three factors. The scale reliability of each of the components/factors was supported, with a Cronbach’s alpha of α = 0.90 for the physical and emotional component, α = 0.92 for the cognitive component and α = 0.80 for the vestibular-ocular component. The lower alpha of the vestibular factor can be attributed in part to the small number of items within this subscale. There were moderate correlations between the three factors physical/emotional: cognitive r = 0.71, physical/emotional:vestibular r = 0.69, cognitive:vestibular r = 0.64.
Discussion
The findings of this study provide initial support for readability, scale reliability and use of the BIST as a total symptom scale and subscales. However, some modifications were highlighted including changes in wording from “I get annoyed” to “I get angry or irritated more easily”. The response scale was also changed from 0–3 to 0–10, clarifying the instructions and including an overall impact rating.
Time to complete the BIST and readability of the symptom items support the potential utility of the BIST as an initial assessment tool that could be used within busy clinical environments to support consistency of care. There was support that patients could also easily self-complete the symptom scale (second component of the BIST). This finding supports the potential for the symptom scale to be used by patients to monitor their symptoms over time and support the identification of symptom exacerbation triggers. Automation of the BIST could assist clinicians with scoring of the measure and facilitate the symptom monitoring process through enabling patients to complete the symptom items over time and providing graphs to track the recovery journey. This would assist clinicians in determining whether a patient’s symptoms had plateaued or were deteriorating indicating a need for specialist treatment and for patients to see progress even if it may be small. There is evidence that having a way to monitor progress would assist patients in their recovery journey [30]. Automation of the tool may also allow for large amounts of data to be collected that is easily accessible, enabling novel machine learning and statistical methods to be applied to build predictive models to aid in cut off scores for further refinement of care pathways and the factors affecting prognosis.
The excellent scale reliability of the total scale and the three factors of the BIST supports its potential use in adults exploring the predictive value of symptom clusters on recovery. There is increasing evidence that symptom clusters may be useful in identifying people who may need early additional support and specifically the type of support they need [21–23]. Evidence for use of the BIST total score will assist in determining a cut off score that may help distinguish people needing early specialist intervention and those who can be given early advice and monitored in primary care to assist with resource allocation, particularly given a high proportion of people do recover naturally [4, 5].
Limitations of this study
To determine initial feasibility and validity of the scale and to enable comparison with other measures, only adults were included in this analysis. Specific studies are needed to determine utility of the measure for those aged 8–15 years. There appeared to be a lower proportion of males within the current analysis than would be expected in the mTBI population, where males are at higher risk of mTBI. Most participants in the current analysis were several years post-injury and had a good level of education. Piloting of the measure within an acute setting is now warranted. Responsiveness, predictive validity, and test-retest reliability were unable to be explored within this study.
Acknowledgments
The authors would like to thank the organisations who supported recruitment for this study and the participants who took part. We would also like to acknowledge other members of the Clinical Expert Group who developed the BIST but were not involved in this study.
References
- 1. Menon DK, Schwab K, Wright DW, Maas AI, Demographics and Clinical Assessment Working Group of the International Health and Interagency Initiative toward Common Data Elements for Research on Traumatic Brain Injury and Psychological Health. Position Statement: Definition of Traumatic Brain Injury. Arch Phys Med Rehabil. 2010;91:1637–40.
- 2.
Ontario Neurotrauma Foundation. Guidelines for concussion/mild traumatic brain injury and persistent symptoms (3rd Edition). 2018: Toronto, Canada.
- 3. Theadom A, Barker-Collo S, Jones K, Kahan M, Te Ao B, McPherson K, et al. Work limitations four years following mild traumatic brain injury: A cohort study. Arch Phys Med Rehabil. 2017;98(8):1560–66. pmid:28188778
- 4. Theadom A, Starkey N, Barker-Collo S, Jones K, Ameratunga S, Feigin V, et al. Population-based cohort study of the impacts of mild traumatic brain injury in adults four years post-injury. PLoS One. 2018;13(1):e0191655. pmid:29385179
- 5. McMahon P, Hricik A, Yue JK, Puccio AM, Inoue T, Lingsma HF, et al. Symptomatology and Functional Outcome in Mild Traumatic Brain Injury: Results from the Prospective TRACK-TBI Study. J Neurotrauma. 2014;31 (1):26–33. pmid:23952719
- 6. Lingsma HF, Yue JK, Maas AIR, Steyerberg EW, Manley GT on behalf of the Track-TBI Investigators. Outcome Prediction After Mild and Complicated Mild Traumatic Brain Injury: External Validation of Existing Models and Identification of New Predictors Using the TRACK-TBI Pilot Study. J Neurotrauma. 2014. 32(2):83–94. pmid:25025611
- 7. Forrest RHJ, Henry JD, McGarry PJ, Marshall RN. Mild traumatic brain injury in New Zealand: factors influencing post-concussion symptom recovery time in a specialised concussion service. J Prim Health Care. 2018;10(2):159–66. pmid:30068471
- 8. Ponsford J, Willmott C, Rothwell A, Cameron P, Kelly AM, Nelms R, et al. Impact of early intervention on outcome following mild head injury in adults. J Neurol, Neurosurg Psychiatr. 2002;73(3):330–2. pmid:12185174
- 9. Gravel J, Ledoux AA, Tang K, Yeates KO, Craig W, Osmond M, et al. Early versus delayed emergency department presentation following mild Traumatic Brain Injury and the presence of symptom at 1, 4 and 12 weeks in children. Emerg Med J. 2020;37:338–43. pmid:32139516
- 10. Kontos AP, Jorgensen-Wagers K, Trbovich AM, Ernst N, Emami K, Gillie B, et al. Association of Time Since Injury to the First Clinic Visit With Recovery Following Concussion JAMA Neurol. 2020;77(4):435–40. pmid:31904763
- 11. Cnossen MC, Polinder S, Lingsma HF, Maas AIR, Menon D, Steyerberg EW, et al. Variation in Structure and Process of Care in Traumatic Brain Injury: Provider Profiles of European Neurotrauma Centers Participating in the CENTER-TBI Study. PloSOne. 2016;Aug 29, pmid:27571205
- 12.
Reid D, Stuart C, Fulcher M, Hume P, Theadom A, Whatman C, et al. Knowledge and attitudes surveys on concussion in sport. Doctors Report #4 to the Accident and Compensation Corporation of New Zealand. Auckland, NZ; 2018. https://sprinz.aut.ac.nz/areas-of-expertise/rugby-codes/brain-health-publications
- 13. Maxtone S, Bishop M, Chappie C, Tumilty S, Quinn D, Kennedy E. Physiotherapist Involvement in Concussion Services in New Zealand: A National Survey. NZJP. 2020;48(2):70–9.
- 14. Theadom A, Barker-Colllo SL, Greenwood A, Parmar P, Jones K, Starkey N, et al. Does sub-classifying mild traumatic brain injuries help to identify people who go on to experience long-term symptoms? Brain Impair. 2017;1–14.
- 15. Echemendia RJ, Meeuwisse W, McCrory P, Davis GA, Putukian M, Leddy J, et al. The Sport Concussion Assessment Tool 5th Edition (SCAT5): Background and rationale. Br J Sports Med. 2017;51:848–50. pmid:28446453
- 16. King NS, Crawford S, Wenden FJ, Moss NE, Wade DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability. J Neurol. 1995;242(9):587–92. pmid:8551320
- 17. Irving G, Neves AL, Dambha-Miller H, Oishi A, Tagashira H, Verho A, et al. International variations in primary care physician consultation time: a systematic review of 67 countries. BMJ Open. 2017:e017902. pmid:29118053
- 18.
Robinson M. Evaluation of the Psychometric and Measurement Properties of the SCAT5 and Child SCAT5 the SCAT5 and Child SCAT5. Electronic Thesis and Dissertation Repository. 6687. https://ir.lib.uwo.ca/etd/6687 Universty of Western Ontario; 2019.
- 19. Potter S, Leigh E, Wade D, Fleminger S. The Rivermead Post Concussion Symptoms Questionnaire: a confirmatory factor analysis. J Neurol. 2006;253(12):1603–14. pmid:17063314
- 20. Thomas M, Skilbeck C, Cannan P, Slatyer M. The structure of the rivermead post-concussion symptoms questionnaire in Australian adults with traumatic brain injury. Brain Impair. 2018;19(2):166–82.
- 21. McGeown J, Kara S, Fulcher M, Crosswell H, Borotkanics R, Hume PA, et al. Predicting sports-related mTBI symptom resolution trajectory using initial clinical assessment findings: a retrospective study. Sports Med. 2020;50(6):1191–202. pmid:31845203
- 22. Ellis MJ, Leddy JL, Willer B. Physiological, vestibulo-ocular and cervicogenic post-concussion disorders: an evidence-based classification system with directions for treatment. Brain Inj. 2015;29(2):238–48. pmid:25314613
- 23. Kontos AP, Sufrinko A, Sandel N, Emami K, Collins MW. Sport-related Concussion Clinical Profiles: Clinical Characteristics, Targeted Treatments, and Preliminary Evidence. Curr Sports Med Rep. 2019;18(3):82–92. pmid:30855306
- 24. Seela RT, Macciocchia S, Velozoc CA, Sharia K, Thompsona N, Heinemannf AW, et al. The Safety Assessment Measure for persons with traumatic brain injury: Item pool development and content validity. NeuroRehabilitation. 2016;39(3):371–87. pmid:27497470
- 25. Hänninen T, Parkkari J, Tuominen M, Öhman J, Howell DR, Iverson GL, et al. Sport Concussion Assessment Tool: Interpreting day-of-injury scores in professional ice hockey players. J Sci Med Sport. 2018;21(8):794–9. pmid:29254676
- 26.
Sikkema JA. Examination of the Psychometric Properties of the SCAT3 and SCAT5 Baseline Testing Data on NCAA Division III College Athletes. https://digitalcommons.georgefox.edu/edd/117 George Fox University; 2018.
- 27. Medvedev ON, Theadom A, Barker-Collo S, Feigin V, On behalf of the BIONIC research group. Distinguishing between enduring and dynamic concussion symptoms: applying Generalisability theory to Rivermead post-concussion symptoms questionnaire (RPQ). PeerJ. 2018;e5676. pmid:30280043
- 28. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis recommendations for getting the most from your analysis. Practical Assessment, Research and Evaluation. 2005;10(Article 7).
- 29.
https://readabilityformulas.com/free-readability-formula-tests.php.
- 30. McPherson K, Fadyl J, Theadom A, Channon A, Levack W, Starkey N, et al. Living life after traumatic brain injury (TBI): phase 1 of a longitudinal qualitative study. J Head Traum Rehabil. 2018;33(1):E44–52. pmid:28520671