Skip to main content

Validation of a patient satisfaction survey of the Teleneurology program in Chile

Abstract

Objective

Telemedicine arises as an attractive intervention for reducing the waiting time for appointments with medical specialists in Chile. Successful implementation of this technology requires safeguarding the patient/clinician trust relationship; however, no studies have been conducted to evaluate quality perception of a telemedicine program in Chile. To assess patient satisfaction with the Teleneurology program at the Hospital Higueras Talcahuano (HHT), addressing patient/clinician trust relationship.

Results

A perception survey was constructed with 23 questions, distributed into 5 key areas (items) of user satisfaction. Its face validity was performed by five neurology specialists from the Teleneurology unit of HHT. The survey was applied to 167 patients of the HHT, recruited between 2018 and 2019, for conducting a pilot cross-sectional descriptive study to assess internal consistency (Cronbach alpha) and reliability (factorial analysis of main components). The survey showed an internal consistency of 0.88. Removing any of the items maintained its reliability in values over 0.8. All items showed point biserial correlations greater than 0.30. Overall, the survey constructed and evaluated in this study showed high internal consistency and reliability values, which will allow its application in further studies of quality assessment of the Teleneurology unit of HHT.

Introduction

The lack of timely access to health care is a growing issue worldwide, particularly in the population with a disability, such as neurological patients [1] The deficit in the care of neurological patients can be explained by: (i) lack of resources to improve the primary care system [2], (ii) deficiencies in access to health care due to disability of patients or the distances they must travel to obtain adequate health care [3], and (iii) a deficit in the number and distribution of neurologist specialists. Telemedicine, which involves the remote access of a patient with a medical practitioner via the internet, has the potential to address some of these deficits. Thus, Teleneurology interventions are being implemented and evaluated in several regions of the globe [4].

The Teleneurology program implemented in 2015 at the Hospital Las Higueras of Talcahuano (HHT), belonging to the Healthcare Unit of Talcahuano (SST, Spanish acronym for Servicio de Salud Talcahuano) of the Ministry of Health, has pioneered addressing the access to adult neurology care with specialists in the country, through a technology that combines the simultaneous live care by a general practitioner in situ, and the remote primary care office by a neurology specialist in specifically designed HHT facilities. All patients have an active participation, making them permanently involved in the delivery of information, complete evaluation and resolution of the health problem. For each video session in the Teleprocess unit (synchronous), a referral and reference web platform are used, which is integrated with the hospital electronic clinical record with an HL7 standard and has an SSL (Secure Sockets Layer) security server for encryption.

Although telemedicine can address the issue of timely access to health care, it is possible that this type of interventions can adversely impact on the quality of the specialist’s relationship with his patient. However, very few tools evaluating the perception of patients for their health care in Teleneurology programs exist, and all of them are in English [4], leaving the entire Spanish speaking population without an opportunity to evaluate quality of care. Given that the HHT Teleneurology program has been in operation for 4 years, it became necessary to assess its patient satisfaction, in order to investigate possible deficiencies that may exist in the generation of the patient-specialist physician trust bond [5,6,7,8]. To this end, a perception survey was elaborated for the first time in Chile to evaluate the patient satisfaction of a Teleneurology unit. A questionnaire of 23 items was constructed to assess 5 dimensions of patient perception: general perception of satisfaction, perception of telemedicine, perception of the patient’s active participation in their own well-being, perception of the convenience of the system and perception of interaction with the doctor specialist. The present study shows the results of the evaluation of this survey, applied in the form of a pilot cross-sectional study in a cohort of 167 patients of the Teleneurology program of the HHT, recruited from September 2018 to February 2019. The survey was evaluated in terms of content, reliability, discrimination and difficulty, for future evaluation of this and other Teleneurology programs in the region, in terms of patient perception of quality.

Main text

Methods

Design and population

Cross-sectional study in a cohort of 167 patients of the HHT Teleneurology program. Participants were selected among the patients who were treated within this program between September 2018 and February 2019, through a non-probabilistic sampling (by volunteering). Participants had to meet the inclusion and not meet the exclusion criteria described in the reference protocol of the Teleneurology program (exempt resolution number 4889 of December 31, 2015), issued by the SST of the Ministry of Health (Additional file 1: Table S1). It should be noted that persons with mental disabilities or who were minors were also excluded, as specified in the Chilean Law 28584, article 28. The research protocol was approved by the Scientific Ethics Committee of the SST of the Ministry of Health. All participants signed a written informed consent.

Survey design

A patient satisfaction questionnaire was designed and constructed in Spanish, consisting of a total of 23 questions with closed responses on a single Likert scale (totally disagree, disagree, neither agree nor disagree, agree, totally agree). This survey was aimed to evaluate the general perception of patient satisfaction (questions 1 and 2), perception of telemedicine care (questions 3 to 5), perception of the patient’s active participation in their own well-being (questions 6 to 13), perception of convenience of the system (questions 14 to 18) and perception of interaction with the medical specialist (questions 19 to 23). Most questions (20 out of 23) were designed based on published English language satisfaction surveys in telemedicine [9,10,11,12,13,14]. On the other hand, three questions were specifically designed for the Teleneurology program of HHT, given that the medical care of this program involves the participation of a general practitioner who accompanies the patient during the remote medical appointment with the neurology specialist (questions 13, 22 and 23) (Additional file 1: Table S2). The survey questionnaire, with a maximum score of 115 points, was graded in terms of satisfaction: very low (under or equal to 23 points), low (24 to 46 points), moderate (47 to 69 points), high (70 to 92 points), and very high (93 to 115 points). The questionnaire was self-administered in order to safeguard the anonymity of the study participant.

Survey evaluation and statistical analysis

Face validity of the survey, in terms of language and particulars of the neurology practice was assessed by five neurology specialists from the Neurology Unit at the HHT. A descriptive analysis of normality of the sample (Kolmogorov–Smirnov) was conducted. The internal consistency was evaluated by Cronbach’s alpha test, which suggests the following scale for alpha coefficients: excellent (> 0.9), good (> 0.8), acceptable (> 0.7), questionable (> 0.6), poor (> 0.5), and unacceptable (< 0.5) [11]. Difficulty and discrimination of the instrument were evaluated by index of difficulty and specific biserial correlation, respectively. Finally, suitability of the data for structure detection was conducted using Kaise–Meyer Olkin measure of sample adequacy and Bartlett’s sphericity test. All analyses were carried out in SPSS, version 25.0. Statistical significance was established at p < 0.05.

Results

Patient cohort description

Of the total number of respondents, 71.86% were women, while the rest were men. Over 97% of men and women showed a very high level of satisfaction with respect to the care received through the Teleneurology program of the HHT. According to Kolmogorov–Smirnov’s analysis, the sample did not exhibit a normal distribution (p < 0.05). Statistically significant differences were also found in the distribution of the population according to sex (p < 0.05).

Survey evaluation

Face validity of the survey was conducted by five local neurology specialists at the HHT. This allowed for the wording of each question to conform both the Spanish language and the pertaining to the neurology practice of medicine (not shown). The questionnaire applied to the voluntary patients was already in its revised form (see Additional file 1: Table S2 for the original Spanish questions).

The reliability analysis of the questionnaire (Table 1) showed that internal consistency was high, with a Cronbach’s alpha of 0.88. Only the consistency of the item on the “general perception of satisfaction” was unacceptable (Cronbach’s alpha 0.47), while those of the items “perception about the active participation of the patient in their own well-being”, “perception about the interaction with the specialist”, “perception of Telemedicine”, and “convenience of the system” were acceptable (Cronbach’s alphas 0.79, 0.75, 0.73, and 0.79, respectively).

Table 1 Analysis of internal consistency per question of the user satisfaction survey (n = 167)

The mean of patient satisfaction score in the total of the surveyed population in this pilot study (Table 2) was 110.4 ± 5.98 points; for men it was 111.1 ± 4.84 and for women of 110.1 ± 6.37. The elimination of any of the questionnaire items led to maintaining its reliability as high (Cronbach’s alpha > 0.8), regardless of the element that is removed (Additional file 1: Table S3).

Table 2 Descriptive data (mean and SD) for total population and by gender; and reliability analysis (α-Cronbach) of the patient the satisfaction survey

Difficulty of the instrument’s questions was assessed by using the difficulty index, it should be noted that none of the items showed an index lower than 0.10; therefore, no items was considered for discarding (Additional file 1: Table S4).

The patient satisfaction survey analysed in this pilot study showed very high levels of patient satisfaction (> 97%, n = 167). The use of Cronbach’s alpha coefficient allowed for evaluation of the reliability of the elaborated instrument. Results showed that the patient satisfaction survey of the HHT Teleneurology program shows a good general reliability (Alpha = 0.88, Additional file 1: Table S5), with a good coherence between its questions.

The analysis of discrimination and difficulty showed that the instrument presents a high index of discrimination, except for questions 22 and 23, which could be revised and rewritten for future analysis, as shown in Additional file 1: Table S5.

Finally, a factorial analysis of the originally designed survey suggested a new questionnaire order in 5 item areas as well as the elimination of one outlier question (Table 3). The new suggested final questionnaire has 22 questions broken down into the following item areas: (i) quality of care; (ii) patient satisfaction; (iii) patient’s relationship with the medical team; (iv) patient’s relationship with the specialist doctor and (v) trust in the Teleneurology unit, whose reliability indexes were 0.80, 0.75, 0.82, 0.70, and 0.71, respectively.

Table 3 New order proposed for the areas in the user satisfaction survey, according to biserial analysis

Discussion

Teleneurology arises as an attractive tool to address some of the deficits in the access to timely healthcare for adult neurological patients. In Chile, the Hospital Las Higueras Talcahuano (HHT) has pioneered the implementation of a Teleneurology program with the goal of increasing access of adult patients to remote neurological care in out-patient procedures since 2015. Since there was no survey evaluating patient perception of healthcare quality for the specific area of Teleneurology in the Spanish language, the present 23-item questionnaire was originally designed based on published reports of surveys in the English language and some guidelines for Spanish surveys in the telemedicine field [9,10,11,12,13,14,15].

Following refining this survey for face validity by five neurology medical specialists, it was applied to 167 patients enrolled in the HHT Teleneurology program between September 2018 and February 2019. Statistical analyses of data resulting from this study showed high reliability and internal consistency of the designed questionnaire. In addition, the biserial correlation analysis showed that 22 of the 23 items can be more appropriately rearranged to assess five different areas related to quality of care, patient satisfaction, and patient relationship with and trust in the medical personnel involved during the Teleneurology consultation (Additional file 2).

In agreement with the results found on the present study, previously published reports indicate that telemedicine interventions do not hinder the development of the link between the patient and his/her specialist doctor, with high patient satisfaction [4, 16,17,18]. However, no definitive conclusions should be drawn regarding the quality of care of patients enrolled in the HHT Teleneurology program until a greater scale study can be performed.

In the specific field of Teleneurology, patient satisfaction studies have been conducted Australia, Canada, the United States, France, Israel, Japan, Norway and the United Kingdom [4, 16, 19,20,21], all non-Spanish speaking countries. Overall, the present pilot study shows that the survey questionnaire in its final form of 22 questions (Table 3) is a reliable tool to evaluate the perception of quality of the HHT Teleneurology program. Validation of the present questionnaire allows for a larger-scale study assessing the impact of this Teleneurology program and its permanent implementation for quality of care vigilance. It also represents the starting point for replicating this experience in other Spanish speaking regions.

Limitations

  1. 1.

    Survey bias We are the only Chilean group in Teleneurology working within the Chilean Public Health system, allowing for no reliable comparison with other programs in the country at the current time.

  2. 2.

    Size bias The limited cohort of this pilot study does not allow for comparison of this program user satisfaction with that of the Chilean Public Health system or other telemedicine programs until a larger-scale study is conducted.

Availability of data and materials

The dataset supporting the conclusions of this article is included within the article and its additional file.

Abbreviations

HHT:

Hospital las Higueras of Talcahuano

SST:

Health Service of Talcahuano (SST) of the Ministry of Health

References

  1. GBD 2015 Neurological Disorders Collaborator Group VL, Abajobir AA, Abate KH, Abd-Allah F, Abdulle AM, Abera SF, et al. Global, regional, and national burden of neurological disorders during 1990–2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet Neurol. 2017;16(11):877–97.

    Article  Google Scholar 

  2. Ray KN, Chari AV, Engberg J, Bertolet M, Mehrotra A. Disparities in time spent seeking medical care in the United States. JAMA Intern Med. 2015;175(12):1983.

    Article  Google Scholar 

  3. Willis AW, Schootman M, Evanoff BA, Perlmutter JS, Racette BA. Neurologist care in Parkinson disease: a utilization, outcomes, and survival study. Neurology. 2011;77(9):851–7.

    Article  CAS  Google Scholar 

  4. Dorsey ER, Glidden AM, Holloway MR, Birbeck GL, Schwamm LH. Teleneurology and mobile technologies: the future of neurological care. Nat Rev Neurol. 2018;14(5):285–97. https://0-doi-org.brum.beds.ac.uk/10.1038/nrneurol.2018.31.

    Article  PubMed  Google Scholar 

  5. Müller KI, Alstadhaug KB, Bekkelund SI. A randomized trial of telemedicine efficacy and safety for nonacute headaches. Neurology. 2017;89(2):153–62. https://0-doi-org.brum.beds.ac.uk/10.1212/WNL.0000000000004085.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Beck CA, Beran DB, Biglan KM, Boyd CM, Dorsey ER, Schmidt PN, et al. National randomized controlled trial of virtual house calls for Parkinson disease. Neurology. 2017;89(11):1152–61.

    Article  Google Scholar 

  7. Finlayson M, Preissner K, Cho C, Plow M. Randomized trial of a teleconference-delivered fatigue management program for people with multiple sclerosis. Mult Scler J. 2011;17(9):1130–40. https://0-doi-org.brum.beds.ac.uk/10.1177/1352458511404272.

    Article  Google Scholar 

  8. Chua R, Craig J, Wootton R, Patterson V. Randomised controlled trial of telemedicine for new neurological outpatient referrals. J Neurol Neurosurg Psychiatry. 2001;71(1):63–6.

    Article  CAS  Google Scholar 

  9. Bakken S, Holzemer WL, Brown M-A, Powell-Cope GM, Turner JG, Inouye J, et al. Relationships between perception of engagement with health care provider and demographic characteristics, health status, and adherence to therapeutic regimen in persons with HIV/AIDS. AIDS Patient Care STDS. 2000;14(4):189–97.

    Article  CAS  Google Scholar 

  10. Dick PT, Filler R, Pavan A. Participant satisfaction and comfort with multidisciplinary pediatric telemedicine consultations. J Pediatr Surg. 1999;34(1):137–41 (discussion 141–2).

    Article  CAS  Google Scholar 

  11. Demiris G, Speedie SM, Hicks LL. Assessment of patients’ acceptance of and satisfaction with teledermatology. J Med Syst. 2004;28(6):575–9.

    Article  Google Scholar 

  12. Demiris G, Speedie SM, Finkelstein S. Change of patients’ perceptions of TeleHomeCare. Telemed J e-Health. 2001;7(3):241–8.

    Article  CAS  Google Scholar 

  13. Bakken S, Grullon-Figueroa L, Izquierdo R, Lee N-J, Morin P, Palmas W, et al. Development, validation, and use of english and spanish versions of the telemedicine satisfaction and usefulness questionnaire. J Am Med Informatics Assoc. 2006;13(6):660–7.

    Article  Google Scholar 

  14. Telemedicine Forms-The Source. 2017. http://hub.americantelemed.org/thesource/resources/telemedicine-forms.

  15. Demiris G, Speedie S, Finkelstein S. A questionnaire for the assessment of patients’ impressions of the risks and benefits of home telecare. J Telemed Telecare. 2000;6(5):278–84.

    Article  CAS  Google Scholar 

  16. Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. BMJ. 2000;320(7248):1517–20.

    Article  CAS  Google Scholar 

  17. Powell RE, Henstenburg JM, Cooper G, Hollander JE, Rising KL. Patient perceptions of Telehealth primary care video visits. Ann Fam Med. 2017;15(3):225–9.

    Article  Google Scholar 

  18. Reed ME, Parikh R, Huang J, Ballard DW, Barr I, Wargon C. Real-time patient-provider video telemedicine integrated with clinical care. N Engl J Med. 2018;379(15):1478–9. https://0-doi-org.brum.beds.ac.uk/10.1056/NEJMc1805746.

    Article  PubMed  Google Scholar 

  19. Channer D. Telemedicine technology. In: Teleneurology in practice. New York: Springer; 2015. p. 39–46. https://0-doi-org.brum.beds.ac.uk/10.1007/978-1-4939-2349-6_4.

  20. Tsao JW, Demaerschalk BM, eds. Teleneurology in practice. New York: Springer; 2015. https://0-doi-org.brum.beds.ac.uk/10.1007/978-1-4939-2349-6.

  21. Weinstein RS, Lian F, Bhattacharyya AK. Teleneuropathology. In: Teleneurology in practice. New York: Springer; 2015. p. 113–29. https://0-doi-org.brum.beds.ac.uk/10.1007/978-1-4939-2349-6_11.

Download references

Acknowledgements

We thank all the staff personnel of the Neurology Unit at the HHT, for their invaluable contributions to this work. Project approved by Comité Ético Científico del Servicio de Salud Talcahuano (Acta No. 97 from 12.12.2017).

Funding

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

FC: Study conception and design, supervision of informed consent signing and survey application, and manuscript elaboration. PA-S: Manuscript elaboration and English editing. JPH: Statistical analysis. MM: Statistical analysis. GV: Supervision of Teleprocesses for the Teleneurology program at HHT. CA: Coordination of researchers and manuscript elaboration. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Freddy Constanzo.

Ethics declarations

Ethics approval and consent to participate

The research protocol was approved by the Scientific Ethics Committee of the SST of the Ministry of Health. All participants signed a written informed consent.

Consent for publication

Not applicable.

Competing interests

The authors declared that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1.

Supplementary tables.

Additional file 2.

Raw data.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Constanzo, F., Aracena-Sherck, P., Hidalgo, J.P. et al. Validation of a patient satisfaction survey of the Teleneurology program in Chile. BMC Res Notes 12, 359 (2019). https://0-doi-org.brum.beds.ac.uk/10.1186/s13104-019-4358-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13104-019-4358-1

Keywords