Figures
Abstract
While considerable knowledge has been gained through the use of established cognitive and motor assessment tools, there is a considerable interest and need for the development of a battery of reliable and validated assessment tools that provide real-time and remote analysis of cognitive and motor function in the elderly. Smartphones appear to be an obvious choice for the development of these “next-generation” assessment tools for geriatric research, although to date no studies have reported on the use of smartphone-based applications for the study of cognition in the elderly. The primary focus of the current study was to assess the feasibility, reliability, and validity of a smartphone-based application for the assessment of cognitive function in the elderly. A total of 57 non-demented elderly individuals were administered a newly developed smartphone application-based Color-Shape Test (CST) in order to determine its utility in measuring cognitive processing speed in the elderly. Validity of this novel cognitive task was assessed by correlating performance on the CST with scores on widely accepted assessments of cognitive function. Scores on the CST were significantly correlated with global cognition (Mini-Mental State Exam: r = 0.515, p<0.0001) and multiple measures of processing speed and attention (Digit Span: r = 0.427, p<0.0001; Trail Making Test: r = −0.651, p<0.00001; Digit Symbol Test: r = 0.508, p<0.0001). The CST was not correlated with naming and verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming) or memory tasks (Logical Memory Test). Test re-test reliability was observed to be significant (r = 0.726; p = 0.02). Together, these data are the first to demonstrate the feasibility, reliability, and validity of using a smartphone-based application for the purpose of assessing cognitive function in the elderly. The importance of these findings for the establishment of smartphone-based assessment batteries of cognitive and motor function in the elderly is discussed.
Citation: Brouillette RM, Foil H, Fontenot S, Correro A, Allen R, Martin CK, et al. (2013) Feasibility, Reliability, and Validity of a Smartphone Based Application for the Assessment of Cognitive Function in the Elderly. PLoS ONE 8(6): e65925. https://doi.org/10.1371/journal.pone.0065925
Editor: Christian Lovis, University Hospitals of Geneva, Switzerland
Received: January 11, 2013; Accepted: April 30, 2013; Published: June 11, 2013
Copyright: © 2013 Brouillette et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by Hibernia National Bank/Edward G. Schlieder Chair, and the supporters of the Institute for Dementia Research and Prevention. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The number of elderly individuals in the United States is growing at an unprecedented rate, with nearly 10,000 individuals a day turning 65 in the United States [1]. This rapid expansion in the number of elderly is also being accompanied by an ever increasing demand for healthcare needs in the geriatric population [1]. Finding strategies that allow for the most efficient mechanism(s) of monitoring the health status of the elderly and identifying the most efficient mechanism(s) for the delivery of remote health surveillance, disease prevention, and medical support will be crucial to adequately addressing the healthcare needs of the elderly in the coming years. Recent technological advances such as telehealth and telemedicine products have proven to be very useful in a variety of settings [2]–[6]. Each of these mechanisms promotes remote patient, physician, and/or student interactions, which are critical features for the next generation of clinical research/care [2]–[8].
Smartphones are specialized mobile phones, which contain additional computing capacity/capabilities, that are likely going to be the principal platforms for the development of the next generation of clinical care/research [9], [10]. In addition to being mobile, and increasingly ubiquitous in the general population, smartphones have an ever increasing number of medical-related applications which can potentially be used for clinical research/care. Several recent papers have described the potential for smartphone-based applications to assist in a wide range of clinical research areas and the delivery of medical care related to falls, medication adherence, food intake, depression, stroke, diabetes management, rehabilitation, medication adherence, and patient point-of-care services [9]–[18].
Despite the numerous advances made in conducting smartphone research, almost nothing is known in terms of the feasibility and validity of using smartphone-based technologies for clinical research/interventions in the elderly. Development of smartphone-based assessments for cognition, mobility, and frailty research in the geriatric population is a crucial area of current research. The goal of the current study was to develop a new cognitive assessment test (processing speed) utilizing touch screen technology, with remote testing capabilities, that is suitable for use in an elderly non-demented population. In this paper, we report for the first time on the development of a new smartphone-based application to measure cognitive function in the elderly. This Color-Shape Test (CST) was found to be a reliable and valid tool for the measurement of cognitive processing and attention in the elderly. The importance of this finding to the development of smartphone-based assessment batteries for cognitive and motor function in the geriatric population is discussed.
Methods
Participants
All 57 participants for this study were recruited from the Louisiana Aging Brain Study (LABrainS) [19]–[24]. Volunteers for the LABrainS study are recruited through regular outreach efforts of the Institute for Dementia Research and Prevention (IDRP) throughout Louisiana, with LABrainS participants consisting of non-demented elderly individuals willing to undergo annual cognitive assessments. The cognitive assessments for these participants are conducted utilizing the Uniform Data Set (UDS) neuropsychological battery established by the National Alzheimer’s Coordinating Center (NACC). Participants in LABrainS are individuals over the age of 60 with no existing diagnosis of dementia or cognitive impairment. All participants have a Clinical Dementia Rating (CDR) of 0 and a Mini-Mental State Exam >25, consistent with the absence of dementia. Additionally, they have no history of other neurological condition (e.g., cerebrovascular disease, Parkinson’s disease, brain injury) that could cause cognitive sequelae. Individuals for this study used the smartphone provided by IDRP staff and had a wide range of experience in smartphone usage ranging from no previous experience to daily use of a smartphone.
Equipment
The assessment software was developed specifically for use with touch screen smartphones and was compatible with all Apple iPhones and Android phones. The software used for development was a Microsoft ASP.Net application which was programmed using C#, jQuery Mobile, and javaScript. Data from test administrations were captured using a Microsoft SQL Server database along with a time stamp indicating when the data were recorded.
Measures
a. Uniform data set.
The neuropsychological test battery from the Uniform Data Set (UDS) of the Alzheimer’s Disease Centers’ (ADC) program of the National Institute on Aging consists of brief measures of attention, processing speed, executive function, episodic memory, and language [25]. The battery consists of measures of dementia severity (Mini-Mental Status Examination) [26], attention (Digit Span Forward and Backward from the Wechsler Memory Scale-Revised) [27], processing speed (Digit Symbol Subtest from the Wechsler Adult Intelligence Scale-Revised [28], Trails Making Test Part A), executive function (Trails Making Test Part B [29]), memory (Logical Memory Story A Immediate Recall from the Wechsler Memory Scale-Revised, Logical Memory Story A Delayed Recall from the Wechsler Memory Scale-Revised [29]), Verbal Fluency (Animal List Generation [30] and Vegetable List Generation), and naming (Boston Naming Test - 30 odd items [31]). In addition to the UDS neuropsychological test battery, participants were also administered the Geriatric Depression Scale (GDS, 15-item version) [32] as a measure of depressive symptomatology.
b. Color-Shape test.
As part of their annual visit, study participants were administered the Color-Shape Test (CST), a test of cognitive processing speed designed at Pennington Biomedical Research Center. The test design is as follows: The test interface (see figure 1) consists of paired colors and shapes which appear at the top of the screen and serve as a “legend” for the task. At the bottom of the screen are colored “pads” which correspond to colors in the legend. Participants use these pads to respond. When the task is initiated, random shapes from the legend appear one at a time in the center of the screen. The shape remains on the screen until the subject taps the color pad (at the bottom of the screen) that corresponds to the one paired with the shape in the legend (top of the screen). Once a response (correct or not) is made, a new shape is presented and the task repeats. Participants were allowed a 30-second practice test prior to the initiation of the actual evaluation. The actual test records the number of attempts and number correct over a 2-minute testing interval.
A representative image of the screen shot for the Color-Shape Test is presented.
Following completion of the neuropsychological testing battery, the CST was administered to participants in the clinic using the examiner’s iPhone 4. The test was administered to each participant utilizing the following standardized instructions:
“This is a test of processing speed. When the test begins, you will see the following screen appear on the smartphone.”
Participants were then shown a screen shot of the interface to facilitate proper instruction of the task. Proper test-taking procedure was demonstrated on the screen shot along with the following verbal instructions:
“The object of this test is to correctly match as many shapes to their corresponding color as possible. Please try to respond as quickly as you can.”
Additionally, as a means of measuring reliability, participants who had been in possession of a smartphone for more than 6 months were emailed a link to the CST for re-administration 2 weeks later. Of the 57 participants who performed the clinic administration of CST, 18 were identified as having been in possession of their personal smartphone for at least 6 months. These 18 participants were emailed a link to retake the CST 2 weeks later. Of the 18 who were emailed, 11 responded by retaking the CST off-site under uncontrolled conditions on their personal phone.
Results
Participant Profile
A total of 57 volunteers from the longitudinal Louisiana Aging Brain Study (LABrainS) participated in this study. The demographics for the age, sex, and education of the participants are provided in Table 1. Additionally, the global cognitive status and GDS measures for the study group are provided in Table 1. Together, these data suggest that the study participants in the current study did not exhibit evidence for dementia or depression. The mean performance scores of study participants on all cognitive assessments are provided in Table 2.
Reliability and Validity of the CST for Processing Speed
Study participants were analyzed for their ability to complete the CST smartphone application as described in the Methods section. Study participants exhibited a mean of 63.77±2.35 attempts on the task and answered correctly on 96.8% of the attempts. The test re-test reliability for the CST was observed to be significant (r = 0.726, p = 0.02). Re-tests were all conducted under uncontrolled off-site conditions. In order to begin validating the CST, we next examined the correlation between the CST and established measures of cognition including cognitive processing speed (Digit Symbol Test, Trails Making Test) (Table 3). In these analyses, we observed that the CST was significantly correlated with Digit Symbol Test (r = 0.508, p<0.00008), Trails Making Test A (r = –0.651, p<0.000001), and Trails Making Test B (r = −0.384, p<0.004). Measures of attention were also significantly correlated with the CST including the Digit Span Forward (r = 0.427, p<0.001) and Digit Span Backward (r = 0.434, p<0.001) tests. The CST was also found to be significantly correlated with the Mini-Mental State Exam (r = 0.515, p<0.00004). The CST was not correlated with naming or verbal fluency tasks (Boston Naming Test, Vegetable/Animal Naming) or memory tasks (Logical Memory Test) (Table 3). Age was significantly correlated with CST performance (r = −0.647, p<0.0000001) (Table 3). The CST was not correlated with education (Table 3).
Discussion
These data demonstrate the feasibility of using smartphone-based applications for the assessment of cognitive function in the elderly. Importantly, these data were collected in participants who had a range of experience with the use of smartphones, ranging from no experience to daily use of smartphones. While we did not collect data on the actual experience of individual study participants, we were aware of the range of experience based on statements made by the individuals ranging from no previous experience to those with extensive experience. Additionally, it is important to point out that the study was not designed to select for people with moderate to high proficiency with smartphone technology. Within this study sample, the CST was found to be a reliable and valid measure of cognitive function, supporting the concept that smartphone-based assessments are likely to provide a useful means of assessing cognition in the elderly, despite the relatively low use of smartphones in the geriatric population. While the relatively small screen of the smartphone was not an insurmountable obstacle to conducting assessments, it was clearly one of the largest experimental obstacles observed in the current study. The screen of the smartphone made for some inadvertent errors with regards to the color-shape matching, as well as premature screen advancement. It is likely that the use of tablet devices, which have a much larger screen and higher computing capacity, may be able to overcome some of the limitations associated with the smartphone use in the present study (i.e., larger surface for manipulating objects, larger screen to accommodate larger objects). Recent surveys indicate that elderly subjects purchase significantly more tablets as compared to smartphones, suggesting that tablet-based cognitive assessments may have a larger population base for conducting cognitive assessments/studies in the geriatric population.
Based on the design of the CST, we hypothesized that the CST would correlate significantly with other established measures of processing speed. The CST application in the current study was observed to be a reliable and valid measure of processing speed and attention in the elderly and suggested that this cognitive assessment tool may be a useful and mobile mechanism for assessing cognitive function in the elderly. The majority of previous studies developing computerized cognitive assessment tools have focused on the use of desktop-based technologies, despite the fact that desktop computers have multiple limitations that affect their potential usage for clinical care/research. For example, desktop computers do not lend themselves readily for mobile/real-time assessments and require dedicated space and Internet availability, both of which may limit patient access. Smartphones/tablets offer a nearly unlimited number of applications that can be seamlessly combined for the comprehensive assessment of cognition, health status, and mobility under free-living conditions. For example, studies have reported on the use of smartphones in a non-geriatric population for monitoring daily activities, rehabilitation, mobility, heart rate, glucose levels, blood pressure, and food intake [11]–[18]. The ability of smartphones to be worn on the body makes them particularly attractive for geriatric studies [9]–[11]. For example, by coupling existing smartphone-based clinical tools (i.e., EKG, EEG, accelerometer, blood pressure recorders) with smartphone-based measures of cognition, researchers could elucidate the interactions between changes in multiple health outcomes and cognitive decline. This real-time comprehensive assessment of overall health status in the elderly could rapidly transform geriatric research, particularly in the areas of dementia, falls, and frailty research. Studies are currently under way by our group to develop a comprehensive battery of cognitive assessments that can be delivered via smartphone/tablet-based technologies to monitor cognitive status in a longitudinal manner within the elderly. This cognitive status battery will then be coupled with additional smartphone-based clinical tools to determine the feasibility of using smartphones for the real-time assessment of frailty in the elderly.
In addition to monitoring the health status of elderly individuals, smartphone/tablet-based technologies may ultimately be used for the delivery and surveillance of clinical interventions in the elderly. For example, smartphone-based applications have been used to deliver clinical care in a non-geriatric population for a variety of conditions including rehabilitation, depression, and falls [6], [12], [15]. Smartphones were found to be an efficient means of delivering interventions, as well as to provide an effective means for monitoring adherence, and measuring outcomes, for each of the interventions. It is important to point out that smartphone-based studies will also likely transform the way individuals are recruited for clinical research trials by decreasing the impact of geography, clinical capacity, patient diversity, and recruitment costs [12]. Such advances will allow for more studies, larger studies, and more diverse forms of studies to be conducted by incorporating the use of smartphones in geriatric research.
Author Contributions
Conceived and designed the experiments: RMB RA CKM ABK JNK. Performed the experiments: RMB HF SF AC. Analyzed the data: RMB ABK JNK. Contributed reagents/materials/analysis tools: RA. Wrote the paper: RMB CKM ABK JNK.
References
- 1. Olshansky SJ, Goldman DP, Zheng Y, Rowe JW (2009) Aging in America in the twenty-first century: demographic forecasts from the MacArthur Foundation Research Network on an Aging Society. Milbank Q 87: 842–862.
- 2. Levine SR, Gorman M (1999) “Telestroke”: the application of telemedicine for stroke. Stroke 30: 464–469.
- 3. Rosser JC Jr, Gabriel N, Herman B, Murayama M (2001) Telementoring and teleproctoring. World J Surg 25: 1438–1448.
- 4.
Rajan R (2012) The promise of wireless: an overview of a device-to-cloud mHealth solution. Biomed Instrum Technol Fall Suppl: 26–32.
- 5. Clifford GD, Clifton D (2012) Wireless technology in disease management and medicine. Annu Rev Med 63: 479–492.
- 6. Rogante M, Grigioni M, Cordella D, Giacomozzi C (2010) Ten years of telerehabilitation: A literature overview of technologies and clinical applications. NeuroRehabilitation 27: 287–304.
- 7. Panayides A, Pattichis MS, Pattichis CS, Schizas CN, Spanias A, et al. (2010) An overview of recent end-to-end wireless medical video telemedicine systems using 3G. Conf Proc IEEE Eng Med Biol Soc 2010: 1045–1048.
- 8. Koch S (2006) Home telehealth–current state and future trends. Int J Med Inform 75: 565–576.
- 9. Fortney JC, Burgess JF Jr, Bosworth HB, Booth BM, Kaboli PJ (2011) A re-conceptualization of access for 21st century healthcare. J Gen Intern Med 26 Suppl 2639–647.
- 10. Boulos MN, Wheeler S, Tavares C, Jones R (2011) How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online 10: 24.
- 11. Doherty ST, Oh P (2012) A multi-sensor monitoring system of human physiology and daily activities. Telemed J E Health 18: 185–192.
- 12. Dufau S, Dunabeitia JA, Moret-Tatay C, McGonigal A, Peeters D, et al. (2011) Smart phone, smart science: how the use of smartphones can revolutionize research in cognitive science. PLoS One 6: e24974.
- 13. Gill PS, Kamath A, Gill TS (2012) Distraction: an assessment of smartphone usage in health care work settings. Risk Manag Healthc Policy 5: 105–114.
- 14. Martin CK, Correa JB, Han H, Allen HR, Rood JC, et al. (2012) Validity of the remote food photography method (RFPM) for estimating energy and nutrient intake in near real-time. Obesity 20: 891–899.
- 15. Mellone S, Tacconi C, Schwickert L, Klenk J, Becker C, et al. (2012) Smartphone-based solutions for fall detection and prevention: the FARSEEING approach. Z Gerontol Geriatr 45: 722–727.
- 16. Mellone S, Tacconi C, Chiari L (2012) Validity of a smartphone-based instrumented timed up and go. Gait Posture 36: 163–165.
- 17. Klasnja P, Pratt W (2012) Healthcare in the pocket: mapping the space of mobile-phone health interventions. J Biomed Inform 45: 184–198.
- 18.
Worringham C, Rojek A, Stewart I (2011) Development and feasibility of a smartphone, ECG and GPS based system for remotely monitoring exercise in cardiac rehabilitation. PLoS One 6: e 14669.
- 19. Schuna JM Jr, Brouillette RM, Foil HC, Fontenot SL, Keller JN, et al. (2013) Steps per day, peak cadence, body mass index, and age in community-dwelling older adults. Med Sci Sports Exerc 45: 914–919.
- 20.
Tudor-Locke C, Barreira TV, Brouillette RM, Foil HC, Keller JN (2012) Preliminary comparison of clinical and free-living measures of stepping cadence in older adults. J Phys Act Health In press.
- 21.
Barreira TV, Brouillette RM, Foil HC, Keller JN, Tudor-Locke C (2012) Comparison of older adults' steps/day using NL-1000 pedometer and two GT3X+ accelerometer filters. J Aging Phys Act In press.
- 22. Bruce-Keller AJ, Brouillette RM, Tudor-Locke C, Foil HC, Gahan WP, et al. (2012) Relationship between cognitive domains, physical performance, and gait in elderly and demented subjects. J Alzheimers Dis 30: 899–908.
- 23. Bruce-Keller AJ, Brouillette RM, Tudor-Locke C, Foil HC, Gahan WP, et al. (2012) Assessment of cognition, physical performance, and gait in the context of mild cognitive impairment and dementia. J Am Geriatr Soc 60: 176–177.
- 24. Brouillette RM, Martin CK, Correa JB, Davis AB, Han H, et al. (2011) Memory for names test provides a useful confrontational naming task for aging and continuum of dementia. J Alzheimers Dis 23: 665–671.
- 25. Weintraub S, Salmon D, Mercaldo N, Ferris S, Graff-Radford NR, et al. (2009) The Alzheimer’s Disease Centers’ Uniform Data Set (UDS): The neuropsychologic test battery. Alzheimer Dis Assoc Disord 23: 91–101.
- 26.
Folstein MF, Folstein SE, McHugh PR (2004) Mini-mental state examination. Lutz, FL: Psychological Assessment Resources.
- 27.
Wechsler D (1987) Wechsler memory scale – Revised. San Antonio, TX: The Psychological Corporation.
- 28.
Wechsler A (1987) Wechsler adult intelligence scale – Revised. San Antonio, TX: The Psychological Corporation.
- 29.
Reitan R, Wolfson D (1993) The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation. 2nd ed. Tucson, AZ: Neuropsychology Press.
- 30. Morris JC, Heyman A, Mohs RC, et al. (1989) The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology 39: 1159–1165.
- 31.
Kaplan E, Goodglass H, Weintraub S (1983) The Boston naming test. Philadelphia: Lea and Febiger.
- 32.
Sheikh JI, Yesavage J (1986) Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version. In: Brink TL, editor. Clinical gerontology: A guide to assessment and intervention. New York: Haworth Press. 165–173.