This paper is in the following e-collection/theme issue:

mHealth for Treatment Adherence (377) Engagement with and Adherence to Digital Health Interventions, Law of Attrition (446) Methods and Feasibility Studies (365) HIV/AIDS/STI Prevention and Care (806)

Published on in Vol 8, No 1 (2020): January

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14557, first published .
The Florida Mobile Health Adherence Project for People Living With HIV (FL-mAPP): Longitudinal Assessment of Feasibility, Acceptability, and Clinical Outcomes

The Florida Mobile Health Adherence Project for People Living With HIV (FL-mAPP): Longitudinal Assessment of Feasibility, Acceptability, and Clinical Outcomes

The Florida Mobile Health Adherence Project for People Living With HIV (FL-mAPP): Longitudinal Assessment of Feasibility, Acceptability, and Clinical Outcomes

Authors of this article:

César Escobar-Viera1 Author Orcid Image ;   Zhi Zhou2 Author Orcid Image ;   Jamie P Morano3 Author Orcid Image ;   Robert Lucero4 Author Orcid Image ;   Spencer Lieb5 Author Orcid Image ;   Sean McIntosh5 Author Orcid Image ;   Kevin A Clauson6 Author Orcid Image ;   Robert L Cook7 Author Orcid Image
Article Authors Cited by (10) Tweetations (22) Metrics

    Original Paper

    1Center for Research on Media, Technology, and Health, University of Pittsburgh, Pittsburgh, PA, United States

    2College of Public Health and Health Professions, University of Florida, Gainesville, FL, United States

    3Division of Infectious Disease and International Medicine, Morsani College of Medicine, University of South Florida, Tampa, FL, United States

    4College of Nursing, University of Florida, Gainesville, FL, United States

    5The AIDS Institute, Tampa, FL, United States

    6College of Pharmacy & Health Sciences, Lipscomb University, Nashville, TN, United States

    7Southern HIV and Alcohol Research Consortium Center for Translational HIV Research, University of Florida, Gainesville, FL, United States

    Corresponding Author:

    César Escobar-Viera, MD, PhD, MPH

    Center for Research on Media, Technology, and Health

    University of Pittsburgh

    230 McKee Place

    Suite 600

    Pittsburgh, PA, 15213

    United States

    Phone: 1 412 692 4297

    Email: escobar-viera@pitt.edu


    Background: For people living with HIV (PLWH), antiretroviral therapy (ART) adherence is crucial to attain better health outcomes. Although research has leveraged consumer health information technologies to enhance ART adherence, no study has evaluated feasibility and clinical outcomes associated with the usage of a commercially available, regularly updated mobile health (mHealth) app for improving ART adherence among PLWH.

    Objective: This study aimed to assess the feasibility, acceptability, and clinical outcomes of Care4Today, an existing, free, biprogrammatic mHealth app for improving ART adherence among PLWH.

    Methods: The Florida mHealth Application Adherence Project (FL-mAPP) was a 90-day longitudinal pilot study conducted in 3 public HIV clinics in Florida, United States. After obtaining informed consent, 132 participants completed a survey and then were given the option to try an existing mHealth app to help with ART adherence. Of these, 33.3% (44/132) declined, 31.1% (41/132) agreed but never used the app, and 35.6% (47/132) used the app. All were asked to complete follow-up surveys at 30 days and 90 days after enrollment. Usage data were used to assess feasibility. Clinical outcomes of self-reported ART adherence and chart-obtained HIV viral load and CD4+ T-cell counts were compared among those who used the platform (users) versus those who did not (nonusers). Participants and HIV care providers also provided responses to open-ended questions about what they liked and did not like about the app; comments were analyzed using thematic analysis.

    Results: Of 132 participants, 47 (35.6%) and 85 (64.4%) were categorized as users and nonusers, respectively. Among users, a Kaplan-Meier plot showed that 25 persons (53%) continued using the app after the 90-day follow-up. At 30-day follow-up, 13 (81.3%) of those who used the mHealth app reported ≥95% ART adherence, compared with 17 (58.6%) nonusers (P=.12). Overall, 39 (82%) users liked or somewhat liked using the platform. Participants’ favorite features were medication reminders, ability to create custom reminders, and adherence reports.

    Conclusions: This longitudinal study found that a commercially available medication adherence mHealth app was a feasible and acceptable intervention to improve ART adherence among PLWH and engaged in clinical care across 3 public HIV clinics in the state of Florida. Overall, participants liked the Care4Today app and thought the medication reminders were their favorite feature. Generally, self-reports of ART adherence were better among users than nonusers, both at 30- and 90-day follow-ups. Further clinical research needs to address user fatigue for improving app usage.

    JMIR Mhealth Uhealth 2020;8(1):e14557

    doi:10.2196/14557

    Keywords

    mHealth; HIV; ART adherence; feasibility; acceptability 


    Background

    For people living with HIV (PLWH), antiretroviral therapy (ART) adherence is crucial to attain both viral load suppression and optimal health outcomes. Although up to 85% of PLWH in the United States report >95% ART adherence [1], concerns remain about the dynamic nature of and potential decline in adherence over time [2-10]. Although research has leveraged consumer health information technologies to enhance ART adherence, such as SMS [11-25] and mobile health (mHealth) apps [26-31], no study has evaluated the longitudinal feasibility and acceptability of an existing, commercially available, free, and regularly updated mHealth app for improving ART adherence among PLWH [32].

    Although 3 meta-analyses [33-35] found SMS reminders to be efficacious for improving ART adherence across diverse populations, the effects were modest and declined over time [34]. Given most studies were descriptive, evidence for mHealth apps that improve HIV medication adherence is limited [36,37]. For example, previous research showed mobile apps to improve ART adherence are feasible and acceptable [28,38], and reminders seem to be effective in helping initiate a medication adherence routine. Data-guided counseling may help PLWH to identify strategies to overcome adherence barriers [39]. However, mHealth interventions seem to have a limited scope during the efficacy trial phase, failing to incorporate functions identified as useful by PLWH [40]. In addition, the traditional research approach of developing and testing homegrown mHealth apps may no longer be practical, given the rapid pace of innovation in mHealth technologies and the budgetary requirements of maintaining, debugging, updating, and upgrading a mobile app [41,42].

    Care4Today Mobile Health Manager (Johnson & Johnson) is a commercially available mHealth app designed to improve medication adherence and has been described in detail elsewhere [43]. The app is biprogrammatic (ie, designed for both smartphones [iPhone or Android] and SMS interfaces). Previously, we reported that among PLWH recruited from public HIV clinics in Florida, United States, over 70% of participants were interested in using an app to monitor their medication adherence. We found that perceived need and availability of someone at the clinic to assist with the app were strongly associated with willingness to try it [43]. These findings informed the implementation of the study we report here.

    Objectives

    The objective of this study was to assess the feasibility and acceptability of Care4Today among PLWH. More specifically, we sought to evaluate (1) app usage pattern; (2) changes in self-reported ART adherence, HIV viral load, and CD4+ T-cell count among PLWH who used the app compared with those who did not; and (3) preferences and suggestions for app features for PLWH. As a secondary goal, we sought to assess the interest of HIV providers and clinical staff in receiving app-generated adherence data.


    Design, Participants, and Setting

    The Florida mHealth Application Adherence Project (FL-mAPP) was a 90-day feasibility and acceptability longitudinal pilot study of a commercially available mHealth app for treatment adherence among PLWH, conducted from 2015 to 2016. As previously described, participants were recruited from public HIV clinics (2 urban and 1 rural) affiliated with the Florida Department of Health [43]. Eligibility criteria included (1) having an HIV positive laboratory result and having been prescribed ART, (2) documented viral load or scheduled viral load test within 4 weeks before or after enrollment, (3) 18 years of age or older, (4) capable of reading and speaking English, (5) owner of a smartphone or feature phone, and (6) able and willing to incur the cost of receiving and sending SMS messages. Potential participants were excluded if they were visually impaired or if they shared their phone with another person.

    After providing informed consent, participants completed a survey questionnaire. They were then able to choose by trying the app in 2 forms: (1) SMS reminders only (for flip phone owners) or (2) full app (for smartphone owners). Clinic personnel assisted participants as needed with downloading the app, creating their account, and configuring the app or SMS reminders. All participants were asked to complete 2 follow-up surveys at 30 and 90 days after enrollment. Participants who chose to try the app or SMS reminders were asked if we could submit a dashboard-generated adherence report to be reviewed by their provider as part of ongoing clinical care. All of these participants accepted the request.

    Participants received gift cards totaling US $30 over the course of the study in compensation for their time completing the surveys regardless of the decision to use the app. Follow-up clinical measures included 30- and 90-day self-reported ART adherence, HIV viral load, and CD4+ T-cell count.

    Data and Measures

    Surveys were conducted at baseline, interim (day 30), and endpoint (day 90). Baseline paper surveys were completed in person at the clinics, and follow-ups were conducted either in person or over the phone by research staff, as described previously [43]. The baseline survey assessed demographic characteristics, self-reported ART adherence, and mobile phone and mHealth app use. Interim and endpoint surveys assessed self-reported ART adherence only.

    Demographic Variables

    Participants reported their age in years (categorized in 4 groups [ie, 18-29, 30-35, 37-39, 43-49, and 50 and older]), gender assigned at birth, race and ethnicity (categorized as Hispanic; white, non-Hispanic; black, non-Hispanic; and other, non-Hispanic), highest education level attained (less than high school, high school graduate or general education diploma, and more than high school), and marital status (single/never married, divorced/widowed/separated, and married/living with a long-time partner).

    Feasibility

    We assessed usage of the app at 30 and 90 days after enrollment among those who used the SMS or app at least once. For each participant, continued use was defined as number of days elapsed from enrollment until the last time the participant reported either a taken or a missed dose via the app or SMS. For analysis, persons who declined to try the app and those who agreed to try but never actually used it were categorized together as nonusers, and those who used the app at least once were defined as users.

    Clinical Outcomes: Antiretroviral Therapy Adherence, HIV Viral Load, and CD4+ T-Cell Count

    We measured self-reported ART adherence using a previously developed item that was validated for PLWH [44]. Participants were asked, Over the last 30 days, how many days did you forget or miss your HIV medications, even partially? We categorized participants’ ≥95% adherence if they reported missing doses on less than 2 days in the previous month. We also attempted to define ART adherence according to the responses from the app, but if participants did not report on a specific day, it was not possible to tell if they were nonadherent or just forgot to enter information into the system.

    We collected HIV viral load and CD4+ T-cell count at 2 distinct intervals: (1) baseline viral load and CD4+ T-cell count at enrollment day or the 30 days preceding enrollment and (2) the next available medical record laboratory results after enrollment. We used a plasma level cut-off point of less than 40 copies/ml of HIV-1 RNA to define optimal viral suppression.

    Acceptability

    At study day 30, participants who chose to try the app answered open-ended questions that assessed acceptability [45]. Items included the following: How did you like using the Care4Today system? What stopped you from using the system more? Which features did you like the most? What did you not like about Care4Today? How can the system be improved?

    Provider Survey

    Participant providers were invited to participate in the study by receiving printed copies of app-generated adherence reports at day 30, 60, and 90. After day 90 of the study, consenting providers completed brief surveys regarding their interest in receiving the app-generated adherence data. Items included the following: How interested would you be in continuing to receive information on patients’ self-reported adherence with this dashboard?How often would you like to see these type of data about your patients? and How interested would you be in learning how to access the dashboard online to look at adherence reports of your patients who are using the “Care4Today” app?

    Data Analyses

    For quantitative data, proportions and chi-square tests compared baseline demographic characteristics, ART adherence, and clinical variables between users versus nonusers [43], both at baseline and follow-up. A Kaplan-Meier plot curve described the proportion of persons who continued to use the app over time. To measure ART adherence, we compared ≥95% versus <95% ART adherence at follow-up among users and nonusers with poor adherence at baseline (n=57). Then, chi-square tests were used to determine the difference in adherence between those who used the mHealth platform and those who did not, at days 30 and 90 after enrollment.

    For responses to open-ended questions, we used an inductive thematic analysis [46] to unveil barriers and facilitators of app use. Participant answers were organized in 2 separate spreadsheets. In total, 2 of the authors (RLC and CGEV) created initial codes and labeled the data using these codes. Next, the codes were categorized into themes and interpreted. Throughout the process, discrepancies were resolved among the qualitative research team.

    Ethics and Consent Process

    All participants were provided with written informed consent, and Health Insurance Portability and Accountability Act required health authorization to use personal health information for research. The institutional review boards of the Florida Department of Health, the University of South Florida, and the University of Florida reviewed and approved all procedures and data management processes.


    Participant Characteristics and Mobile Health Platform Feasibility

    As previously reported by this study group [43], of the total 132 participants at baseline, 47 (47/132, 35.6%) agreed to use the app and used it at least once (users). Nonusers (85/132, 64.4%) comprised 44 participants (44/132, 33.3%) who declined to try the app and 41 (41/132, 31.1%) who agreed to try the app but did not use it. Of 47 Users, 37 (78%) who chose to use the full smartphone app and 10 (21%) who chose SMS. We conducted an analysis to compare these 2 groups and found no significant differences other than smartphone ownership, which was significantly more frequent among those who chose to use the full app. At baseline and follow-up, no significant differences were found in ART adherence, HIV viral load, or CD4+ T-cell count between users and nonusers (Table 1). Among users, more than 50% continued to use the app for 90 days or more, with 6% (3/47) using the app for less than 30 days, 19% (9/47) between 30 and 59 days, 21% (10/47) between 60 and 89 days, and 53% (25/47) for over 90 days (Figure 1).

    From baseline, 81% (38/47) of app users and 64% (54/85) of nonusers completed 90-day follow-up surveys. Table 2 shows the proportion of persons who reported >95% adherence at 30- and 90-day follow-up among participants that reported <95% ART adherence at baseline. At enrollment, 57/132 persons (44.5%) reported <95% ART adherence. At 30-day follow-up, we found a nonsignificant difference (P=.12) between ≥95% adherence reported by app users (81%, 13/16) compared with just over half of nonusers (58%, 17/29). Similarly, at 90-day follow-up, 7/15 users (46%) reported ≥95% ART adherence compared with 9/22 nonusers (40%), although this difference was not significant.

    Table 1. Baseline and follow-up of antiretroviral adherence and clinical parameters of Care4Today users and nonusers, Florida Mobile Health Application Adherence Project study, 2015 to 2016 (N=132).
    CharacteristicsTotal (N=132)a, n (%)Users (n=47), n (%)Nonusers (n=85), n (%)P valueb
    Antiretroviral therapy adherence ≥95%
     Baseline71 (55.5)26 (57.8)45 (54.2)0.7
     30-day follow-up85 (79.4)35 (85.4)50 (75.8)0.23
     90-day follow-up65 (70.7)25 (65.8)40 (74.1)0.39
    HIV RNA ≤200 copies/mL
     Baseline91 (71.0)37 (78.7)54 (66.7)0.15
     90-day follow-up97 (88.2)60 (85.7)37 (92.5)0.37
    CD4 count >500 cells/µL
     Baseline69 (54.3)25 (54.4)44 (54.3)0.99
     90-day follow-up61 (56.0)23 (59.0)38 (54.3)0.64
    Mean CD4 count (cells/µL), mean (SD)
     Baseline575 (311)560 (329)583 (280)0.68
     90-day follow-up629 (346)635 (352)626 (345)0.88

    aSome variables might not total 132 owing to missing data.

    bP value derived using chi-square analysis comparing proportions in each category.

    ‎
    Figure 1. Proportion of persons who continued to use the app to input antiretroviral therapy adherence information after enrollment (N=47).
    View this figure
    Table 2. Self-reported antiretroviral adherence and clinical parameters at 30- and 90-day follow-up among participants with <95% adherence at baseline (N=57).
    ParameterUsers (n=19)aNonusers (n=38)bP valuec
    Self-reported antiretroviral therapy adherence, n (%)

    		30-day follow-up.12


    		≥95%13 (81)17 (58)

    		90-day follow-up.73


    		≥95%7 (46)9 (40)
    HIV viral load, n (%)

    		Baseline.30


    		<40 copies/mL12 (63)17 (48)

    		Follow-up.74


    		<40 copies/mL12 (75)20 (69)
    CD4+ T-cell count, n (%)

    		Baseline.13


    		0-200 cells/mL35 (26)2 (5)


    		201-500 cells/mL36 (31)14 (40)


    		>500 cells/mL38 (42)19 (54)

    		Follow-up.36


    		0-200 cells/mL34 (26)3 (10)


    		201-500 cells/mL35 (33)10 (34)


    		>500 cells/mL36 (40)16 (55)

    aSome variables might not be total 19 owing to missing data.

    bSome variables might not be total 38 owing to missing data.

    cP value derived using chi-square analysis comparing proportions in each category.

    Acceptability

    Overall, at 30-day follow-up, there were 47 users who used the app at least once, of whom 39/47 (82%) liked or somewhat liked using the app, 1/47 (2%) did not like it, and 7/47 (15%) did not respond the question. Favorite features were medication reminders (n=25), ability to create custom reminders (n=5), adherence reports (n=5), and multiple features (n=4). Open-ended questions identified app features and technical issues that participants found challenging. The qualitative findings are available in Multimedia Appendix 1. Users reported that these issues kept them from using the app and could be improved for future use. Among these, lack of consistency in receiving reminders, characteristics of these reminders (eg, frequency, sound type, and duration), and inability to use all the features when not connected to the internet were main complaints and barriers to use. In terms of the features, users wanted more flexibility to report their medication taking, both if medication was taken a little early or a little late. Suggestions for improvement focused on enhancing the reminders in frequency, recurrence, or volume. Nevertheless, 36/47 users (80%) reported that they intended to keep using the Care4Today app.

    Provider Survey

    A total of 10 clinicians completed the survey: 5 nurse practioners and 5 physicians. The average age was 47.8 years (SD 15.6) with the majority being female (80%, 8/10) and white non-Hispanic (70%, 7/10). Of the 10 providers, 6 had over 10 years of experience working with PLWH, 1 had 6 to 10 years, and 3 had 5 years or less. Although 90% (9/10) of providers were either very or somewhat interested in reviewing adherence reports through the app, 80% (8/10) wanted to see these reports either monthly or quarterly (as opposed to daily or weekly). Finally, 5 providers (3 nurse practitioners and 2 physicians) had low to no interest in logging in to the provider interface to access the reports.


    Principal Findings

    This longitudinal study found that a commercially available medication adherence mHealth app was a feasible and acceptable intervention to improve ART adherence among PLWH engaged in clinical care across 3 public HIV clinics in the state of Florida. To our knowledge, this is the first feasibility pilot study that used a free, commercially available mHealth app to improve medication adherence among PLWH. Overall, participants liked the Care4Today app and favored the medication reminders feature. Generally, self-reports of ART adherence were better among users than nonusers, both at 30- and 90-day follow-ups, although the difference between groups was not statistically significant. Given the main goal of this study was to assess acceptability and feasibility of the intervention, we did not intend to demonstrate statistical significance of our findings. Rather, we sought to identify trends and factors that might inform a larger efficacy study. Importantly, although our findings do not support larger implementation of commercially available medication adherence apps for PLWH at this time, we did identify promising results in clinical outcomes that are commonly accepted proxies for efficacy (ie, self-reported ART adherence).

    Among participants who had less than optimal ART adherence at baseline, over half of them started and kept using the intervention for the entire duration of the study. Although the drop-in usage between 30- and 90-day follow-ups might seem considerable when compared with previous research [30,47], some clarifications might help to interpret these findings. On one hand, study participants were not financially compensated to use the platform, and all usage was completely voluntary. On the other hand, almost half of the platform users already reported an ART adherence of ≥95% at baseline; therefore, some participants might have found the intervention less critical for their compliance than others. It would be appropriate to have future research on commercially available adherence apps looking more specifically at usage behavior of a larger group of persons with less than optimal ART adherence at the start of the study.

    Of our participants with less than optimal ART adherence at baseline, the proportion of persons who reported good ART adherence at follow-up was higher among app users than nonusers. Although this difference was not statistically significant, this finding suggests that use of the app did improve medication taking, and this could be assessed with a larger clinical trial. Although self-reported ART adherence has been consistently associated with improved clinical outcomes, including viral suppression [48,49], most self-report measures are a proxy of true adherence and present some limitations. To address this limitation, we used a carefully validated measure of self-report ART adherence that is both simple and convenient for PLWH [44].

    Participants’ feedback focused on suggestions to improve features and technical aspects of the app. Among these, suggestions about personalized and customized reminder characteristics were paramount. Moreover, users wanted an app that would work even if they were off a wireless internet connection. These recommendations speak about the user’s desire of more independence when using mHealth apps. However, these requests need to be balanced with the need to keep apps efficient and fun to use. Potentially, these objectives can be met with commercially available apps that are developed and maintained by dedicated personnel in charge of updating and enhancing app databases and features. On the contrary, providers’ feedback showed clinicians see value in accessing self-reported adherence data from their patients during clinical visits. This seems to fit well with the clinical workflow; visits provide the opportunity to address adherence inconsistencies that may be reported through the mHealth app reports. However, in order for providers to retrieve these data on their own, mHealth apps should be able to easily communicate with information systems already being used by health professionals. Clinicians had less interest in viewing these reports on a daily or weekly basis. One potential way to link these reports into a workflow could be to establish a critical number of missing doses that would trigger an increased frequency of medication reminders, or suggestions for tele-consultations with clinical staff in advance of the next visit to address adherence inconsistencies promptly.

    Limitations and Summary

    Our work has several limitations. This study was conducted at 3 HIV clinics affiliated to the Florida Health Department. Thus, our findings cannot be generalized to sites that operate with different procedures for patient engagement and follow-up. However, our work is of interest given that these clinics draw from a distinct socioeconomic group of diverse backgrounds. Almost two-thirds of study participants were categorized as app nonusers, either because they declined to try the app at all, had problems downloading and using the app, or did not use it after downloading. As reported previously [43], interest in using the app was associated both with perceived benefits and with perceptions that someone at the clinic would assist them on how to use it. Although a detailed description of our study participants (both users and nonusers) is available elsewhere [43], we did not collect information on nonparticipants, who may have not participated because the study was not available in Spanish, they did not have the correct type of phone, or may have been concerned about costs for SMS services. Given the sample size of both app users and nonusers, we were limited in our ability to conduct subgroup analyses at this time. These would be valuable to further contextualize the app’s utility in subpopulations of PLWH, and future research on commercially available medication adherence apps should consider assessing these differences. We noticed that almost 50% of participants had user fatigue throughout this 90-day pilot study. Active updates and app maintenance and a more robust contingency system (such as gamification) might help to sustain novelty and address this concern. ART adherence data relied on self-report. Future research should include not only a larger sample and experimental design, but also an objective measure of medication adherence.

    In summary, Care4Today Mobile Health Manager, a commercially available, biprogrammatic medication adherence app that operates with both iOS and Android operating systems, as well as SMS, was found feasible and acceptable for improving ART adherence among PLWH during a 3-month longitudinal study. Further clinical research needs to address user fatigue to improve platform usage.

    Acknowledgments

    This research was supported by an Educational and Innovation Grant from Janssen Healthcare Innovations. The first author was supported by the National Institute of Health Disparities and Minority Health (K99-MD012813). Administrative support was provided by the Southern Alcohol HIV Research Consortium (SHARC) U24AA022002 and the Florida Consortium for HIV/AIDS Research. The authors would like to thank Michelle Woods for editorial assistance.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Summary of responses to open-ended questions regarding the acceptability of the mobile health platform, Florida Mobile Health Adherence Project for People Living With HIV Study, 2016.

    DOCX File , 14 KB
    1. Kelso-Chichetto NE, Okafor CN, Harman JS, Canidate SS, Cook CL, Cook RL. Complementary and alternative medicine use for HIV management in the state of Florida: medical monitoring project. J Altern Complement Med 2016 Nov;22(11):880-886 [FREE Full text] [CrossRef] [Medline]
    2. Byakika-Tusiime J, Crane J, Oyugi JH, Ragland K, Kawuma A, Musoke P, et al. Longitudinal antiretroviral adherence in HIV+ Ugandan parents and their children initiating HAART in the MTCT-Plus family treatment model: role of depression in declining adherence over time. AIDS Behav 2009 Jun;13(Suppl 1):82-91. [CrossRef] [Medline]
    3. Liu H, Miller LG, Hays RD, Golin CE, Wu T, Wenger NS, et al. Repeated measures longitudinal analyses of HIV virologic response as a function of percent adherence, dose timing, genotypic sensitivity, and other factors. J Acquir Immune Defic Syndr 2006 Mar;41(3):315-322. [CrossRef] [Medline]
    4. Mannheimer S, Friedland G, Matts J, Child C, Chesney M. The consistency of adherence to antiretroviral therapy predicts biologic outcomes for human immunodeficiency virus-infected persons in clinical trials. Clin Infect Dis 2002 Apr 15;34(8):1115-1121. [CrossRef] [Medline]
    5. Lima VD, Harrigan R, Bangsberg DR, Hogg RS, Gross R, Yip B, et al. The combined effect of modern highly active antiretroviral therapy regimens and adherence on mortality over time. JAIDS J Acquir Immune Defic Syndr 2009;50(5):529-536. [CrossRef]
    6. Tesoriero J, French T, Weiss L, Waters M, Finkelstein R, Agins B. Stability of adherence to highly active antiretroviral therapy over time among clients enrolled in the treatment adherence demonstration project. J Acquir Immune Defic Syndr 2003 Aug 1;33(4):484-493. [CrossRef] [Medline]
    7. Gross R, Bilker WB, Friedman HM, Strom BL. Effect of adherence to newly initiated antiretroviral therapy on plasma viral load. AIDS 2001 Nov 09;15(16):2109-2117. [CrossRef] [Medline]
    8. Lazo M, Gange SJ, Wilson TE, Anastos K, Ostrow DG, Witt MD, et al. Patterns and predictors of changes in adherence to highly active antiretroviral therapy: longitudinal study of men and women. Clin Infect Dis 2007 Nov 15;45(10):1377-1385. [CrossRef] [Medline]
    9. Lucas GM, Gebo KA, Chaisson RE, Moore RD. Longitudinal assessment of the effects of drug and alcohol abuse on HIV-1 treatment outcomes in an urban clinic. AIDS 2002 Mar 29;16(5):767-774. [CrossRef] [Medline]
    10. Carrieri P, Cailleton V, Le Moing V, Spire B, Dellamonica P, Bouvet E, APROCO study group. The dynamic of adherence to highly active antiretroviral therapy: results from the French National APROCO cohort. J Acquir Immune Defic Syndr 2001 Nov 1;28(3):232-239. [CrossRef] [Medline]
    11. Abdulrahman SA, Rampal L, Ibrahim F, Radhakrishnan AP, Kadir Shahar H, Othman N. Mobile phone reminders and peer counseling improve adherence and treatment outcomes of patients on ART in Malaysia: A randomized clinical trial. PLoS One 2017;12(5):e0177698 [FREE Full text] [CrossRef] [Medline]
    12. Dowshen N, Kuhns LM, Johnson A, Holoyda BJ, Garofalo R. Improving adherence to antiretroviral therapy for youth living with HIV/AIDS: a pilot study using personalized, interactive, daily text message reminders. J Med Internet Res 2012 Apr 5;14(2):e51 [FREE Full text] [CrossRef] [Medline]
    13. Ruan Y, Xiao X, Chen J, Li X, Williams AB, Wang H. Acceptability and efficacy of interactive short message service intervention in improving HIV medication adherence in Chinese antiretroviral treatment-naïve individuals. Patient Prefer Adherence 2017;11:221-228 [FREE Full text] [CrossRef] [Medline]
    14. Ingersoll KS, Dillingham RA, Hettema JE, Conaway M, Freeman J, Reynolds G, et al. Pilot RCT of bidirectional text messaging for ART adherence among nonurban substance users with HIV. Health Psychol 2015 Dec;34S:1305-1315 [FREE Full text] [CrossRef] [Medline]
    15. Garofalo R, Kuhns LM, Hotton A, Johnson A, Muldoon A, Rice D. A randomized controlled trial of personalized text message reminders to promote medication adherence among HIV-positive adolescents and young adults. AIDS Behav 2016 May;20(5):1049-1059 [FREE Full text] [CrossRef] [Medline]
    16. Mao L, Buchanan A, Wong HT, Persson A. Beyond mere pill taking: SMS reminders for HIV treatment adherence delivered to mobile phones of clients in a community support network in Australia. Health Soc Care Community 2018 Jul;26(4):486-494. [CrossRef] [Medline]
    17. King E, Kinvig K, Steif J, Qiu AQ, Maan EJ, Albert AY, et al. Mobile text messaging to improve medication adherence and viral load in a vulnerable Canadian population living with human immunodeficiency virus: a repeated measures study. J Med Internet Res 2017 Jun 1;19(6):e190 [FREE Full text] [CrossRef] [Medline]
    18. Georgette N, Siedner MJ, Zanoni B, Sibaya T, Petty CR, Carpenter S, et al. The acceptability and perceived usefulness of a weekly clinical SMS program to promote HIV antiretroviral medication adherence in Kwazulu-Natal, South Africa. AIDS Behav 2016 Nov;20(11):2629-2638 [FREE Full text] [CrossRef] [Medline]
    19. Hardy H, Kumar V, Doros G, Farmer E, Drainoni M, Rybin D, et al. Randomized controlled trial of a personalized cellular phone reminder system to enhance adherence to antiretroviral therapy. AIDS Patient Care STDS 2011 Mar;25(3):153-161 [FREE Full text] [CrossRef] [Medline]
    20. Hirsch-Moverman Y, Daftary A, Yuengling KA, Saito S, Ntoane M, Frederix K, et al. Using mHealth for HIV/TB treatment support in Lesotho: enhancing patient-provider communication in the start study. J Acquir Immune Defic Syndr 2017 Jan 1;74(Suppl 1):S37-S43 [FREE Full text] [CrossRef] [Medline]
    21. Orrell C, Cohen K, Mauff K, Bangsberg DR, Maartens G, Wood R. A randomized controlled trial of real-time electronic adherence monitoring with text message dosing reminders in people starting first-line antiretroviral therapy. J Acquir Immune Defic Syndr 2015 Dec 15;70(5):495-502. [CrossRef] [Medline]
    22. Pop-Eleches C, Thirumurthy H, Habyarimana JP, Zivin JG, Goldstein MP, de Walque D, et al. Mobile phone technologies improve adherence to antiretroviral treatment in a resource-limited setting: a randomized controlled trial of text message reminders. AIDS 2011 Mar 27;25(6):825-834 [FREE Full text] [CrossRef] [Medline]
    23. Rana AI, van den Berg JJ, Lamy E, Beckwith CG. Using a mobile health intervention to support HIV treatment adherence and retention among patients at risk for disengaging with care. AIDS Patient Care STDS 2016 Apr;30(4):178-184 [FREE Full text] [CrossRef] [Medline]
    24. Wagner N, Ouedraogo D, Artavia-Mora L, Bedi A, Thiombiano BA. Protocol for a randomized controlled trial evaluating mobile text messaging to promote retention and adherence to antiretroviral therapy for people living with HIV in Burkina Faso. JMIR Res Protoc 2016 Aug 17;5(3):e170 [FREE Full text] [CrossRef] [Medline]
    25. Rodrigues R, Shet A, Antony J, Sidney K, Arumugam K, Krishnamurthy S, et al. Supporting adherence to antiretroviral therapy with mobile phone reminders: results from a cohort in South India. PLoS One 2012;7(8):e40723 [FREE Full text] [CrossRef] [Medline]
    26. Molton JS, Pang Y, Wang Z, Qiu B, Wu P, Rahman-Shepherd A, et al. Prospective single-arm interventional pilot study to assess a smartphone-based system for measuring and supporting adherence to medication. BMJ Open 2016 Dec 20;6(12):e014194 [FREE Full text] [CrossRef] [Medline]
    27. Perera AI, Thomas MG, Moore JO, Faasse K, Petrie KJ. Effect of a smartphone application incorporating personalized health-related imagery on adherence to antiretroviral therapy: a randomized clinical trial. AIDS Patient Care STDS 2014 Nov;28(11):579-586 [FREE Full text] [CrossRef] [Medline]
    28. Anglada-Martínez H, Martin-Conde M, Rovira-Illamola M, Sotoca-Momblona JM, Sequeira E, Aragunde V, et al. Feasibility and preliminary outcomes of a web and smartphone-based medication self-management platform for chronically ill patients. J Med Syst 2016 Apr;40(4):99. [CrossRef] [Medline]
    29. Przybyla SM, Eliseo-Arras RK, Krawiec G, Gower E, Dermen K. Feasibility and acceptability of a smartphone app for daily reports of substance use and antiretroviral therapy adherence among HIV-infected adults. AIDS Res Treat 2016;2016:9510172 [FREE Full text] [CrossRef] [Medline]
    30. Himelhoch S, Kreyenbuhl J, Palmer-Bacon J, Chu M, Brown C, Potts W. Pilot feasibility study of Heart2HAART: a smartphone application to assist with adherence among substance users living with HIV. AIDS Care 2017 Jul;29(7):898-904. [CrossRef] [Medline]
    31. Sabin LL, DeSilva M, Gill CJ, Zhong L, Vian T, Xie W, et al. Improving adherence to antiretroviral therapy with triggered real-time text message reminders: the China adherence through technology study. J Acquir Immune Defic Syndr 2015 Aug 15;69(5):551-559 [FREE Full text] [CrossRef] [Medline]
    32. Dayer L, Heldenbrand S, Anderson P, Gubbins PO, Martin BC. Smartphone medication adherence apps: potential benefits to patients and providers. J Am Pharm Assoc (2003) 2013;53(2):172-181 [FREE Full text] [CrossRef] [Medline]
    33. Finitsis DJ, Pellowski JA, Johnson BT. Text message intervention designs to promote adherence to antiretroviral therapy (ART): a meta-analysis of randomized controlled trials. PLoS One 2014;9(2):e88166 [FREE Full text] [CrossRef] [Medline]
    34. Kanters S, Park JJ, Chan K, Socias ME, Ford N, Forrest JI, et al. Interventions to improve adherence to antiretroviral therapy: a systematic review and network meta-analysis. Lancet HIV 2017 Jan;4(1):e31-e40. [CrossRef] [Medline]
    35. Amankwaa I, Boateng D, Quansah DY, Akuoko CP, Evans C. Effectiveness of short message services and voice call interventions for antiretroviral therapy adherence and other outcomes: A systematic review and meta-analysis. PLoS One 2018;13(9):e0204091 [FREE Full text] [CrossRef] [Medline]
    36. Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV treatment & prevention: a systematic review of the literature. Open AIDS J 2013;7:17-41 [FREE Full text] [CrossRef] [Medline]
    37. Anglada-Martinez H, Riu-Viladoms G, Martin-Conde M, Rovira-Illamola M, Sotoca-Momblona JM, Codina-Jane C. Does mHealth increase adherence to medication? Results of a systematic review. Int J Clin Pract 2015 Jan;69(1):9-32. [CrossRef] [Medline]
    38. Hightow-Weidman L, Muessig K, Knudtson K, Srivatsa M, Lawrence E, LeGrand S, et al. A gamified smartphone app to support engagement in care and medication adherence for HIV-positive young men who have sex with men (AllyQuest): development and pilot study. JMIR Public Health Surveill 2018 Apr 30;4(2):e34 [FREE Full text] [CrossRef] [Medline]
    39. Sabin LL, Mansfield L, DeSilva MB, Vian T, Li Z, Wubin X, et al. Why it worked: participants' insights into an mHealth antiretroviral therapy adherence intervention in China. Open AIDS J 2018;12:20-37 [FREE Full text] [CrossRef] [Medline]
    40. Cooper V, Clatworthy J, Whetham J, Consortium E. mHealth interventions to support self-management in HIV: a systematic review. Open AIDS J 2017;11:119-132 [FREE Full text] [CrossRef] [Medline]
    41. Pham Q, Wiljer D, Cafazzo JA. Beyond the randomized controlled trial: a review of alternatives in mHealth clinical trial methods. JMIR Mhealth Uhealth 2016 Sep 9;4(3):e107 [FREE Full text] [CrossRef] [Medline]
    42. Baker TB, Gustafson DH, Shah D. How can research keep up with eHealth? Ten strategies for increasing the timeliness and usefulness of eHealth research. J Med Internet Res 2014 Feb 19;16(2):e36 [FREE Full text] [CrossRef] [Medline]
    43. Morano J, Clauson K, Zhou Z, Escobar-Viera C, Lieb S, Chen I, et al. Attitudes, beliefs, and willingness toward the use of mHealth tools for medication adherence in the Florida mHealth adherence project for people living with HIV (FL-mAPP): pilot questionnaire study. JMIR Mhealth Uhealth 2019 Jul 3;7(7):e12900 [FREE Full text] [CrossRef] [Medline]
    44. Wilson IB, Lee Y, Michaud J, Fowler FJ, Rogers WH. Validation of a new three-item self-report measure for medication adherence. AIDS Behav 2016 Nov;20(11):2700-2708 [FREE Full text] [CrossRef] [Medline]
    45. Sekhon M, Cartwright M, Francis JJ. Acceptability of healthcare interventions: an overview of reviews and development of a theoretical framework. BMC Health Serv Res 2017 Jan 26;17(1):88 [FREE Full text] [CrossRef] [Medline]
    46. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    47. Horvath KJ, Lammert S, MacLehose RF, Danh T, Baker JV, Carrico AW. A pilot study of a mobile app to support HIV antiretroviral therapy adherence among men who have sex with men who use stimulants. AIDS Behav 2019 Jul 15. [CrossRef] [Medline]
    48. Da W, Li X, Qiao S, Zhou Y, Shen Z. Evaluation of self-report adherence measures and their associations with detectable viral load among people living with HIV (PLHIV) in China. PLoS One 2018;13(8):e0203032 [FREE Full text] [CrossRef] [Medline]
    49. Arnsten J, Demas P, Farzadegan H, Grant R, Gourevitch M, Chang C, et al. Antiretroviral therapy adherence and viral suppression in HIV-infected drug users: comparison of self-report and electronic monitoring. Clin Infect Dis 2001 Oct 15;33(8):1417-1423 [FREE Full text] [CrossRef] [Medline]

    ‎
    ART: antiretroviral therapy
    FL-mAPP: Florida mHealth Application Adherence Project
    mHealth: mobile health
    PLWH: people living with HIV

    Edited by CL Parra-Calderón; submitted 02.05.19; peer-reviewed by A Fialho, A Eaton; comments to author 15.06.19; revised version received 11.08.19; accepted 28.09.19; published 08.01.20

    Copyright

    ©César G Escobar-Viera, Zhi Zhou, Jamie P Morano, Robert Lucero, Spencer Lieb, Sean McIntosh, Kevin A Clauson, Robert L Cook. Originally published in JMIR mHealth and uHealth (http://mhealth.jmir.org), 08.01.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR mHealth and uHealth, is properly cited. The complete bibliographic information, a link to the original publication on http://mhealth.jmir.org/, as well as this copyright and license information must be included.