http://orcid.org/0000-0003-1914-4891
Figures
Abstract
To improve health outcomes in people living with HIV, adoption of evidence-based interventions (EBIs) using effective and transferable implementation strategies to optimise the delivery of healthcare is needed. ViiV Healthcare’s Positive Pathways initiative was established to support the UNAIDS 90-90-90 goals. A compendium of EBIs was developed to address gaps within the HIV care continuum, yet it was unknown whether efforts existed to adapt and implement these EBIs across diverse clinical contexts. Therefore, this review sought to report on the use of implementation science in adapting HIV continuum of care EBIs. A systematic literature review was undertaken to summarise the evaluation of implementation and effectiveness outcomes, and report on the use of implementation science in HIV care. Ten databases were reviewed to identify studies (time-period: 2013–2018; geographic scope: United States, United Kingdom, France, Germany, Italy, Spain, Canada, Australia and Europe; English only publications). Studies were included if they reported on people living with HIV or those at risk of acquiring HIV and used interventions consistent with the EBIs. A broad range of study designs and methods were searched, including hybrid designs. Overall, 118 publications covering 225 interventions consistent with the EBIs were identified. These interventions were evaluated on implementation (N = 183), effectiveness (N = 81), or both outcomes (N = 39). High variability in the methodological approaches was observed. Implementation outcomes were frequently evaluated but use of theoretical frameworks was limited (N = 13). Evaluations undertaken to assess effectiveness were inconsistent, resulting in a range of measures. This review revealed extensive reporting on implementation science as defined using evaluation outcomes. However, high variability was observed in how implementation outcomes and effectiveness were defined, quantified, and reported. A more specific and consistent approach to conducting and reporting on implementation science in HIV could facilitate achievement of UNAIDS 90-90-90 targets.
Citation: Cox J, Gutner C, Kronfli N, Lawson A, Robbins M, Nientker L, et al. (2019) A need for implementation science to optimise the use of evidence-based interventions in HIV care: A systematic literature review. PLoS ONE 14(8): e0220060. https://doi.org/10.1371/journal.pone.0220060
Editor: Wisit Cheungpasitporn, University of Mississippi Medical Center, UNITED STATES
Received: February 6, 2019; Accepted: July 7, 2019; Published: August 19, 2019
Copyright: © 2019 Cox et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and Supporting Information files.
Funding: This research was funded by ViiV Healthcare (https://www.viivhealthcare.com/). ViiV Healthcare provided support in the form of salaries for authors CG, AL, MR and EF, and consultancy fees for authors LN and AO. The specific roles of these authors are articulated in the ‘author contributions’ section. The costs of the development of the publication were funded by ViiV Healthcare. Editorial assistance was provided under the direction of the authors by MedThink SciCom and was funded by ViiV Healthcare. CG, AL and MR work full-time at ViiV Healthcare and EF works as a part-time service provider for ViiV Healthcare. The funders had a role in study design, data collection and analysis, decision to publish and preparation of the manuscript through the provision of expert decision-making input from clinical and management teams.
Competing interests: JC, NK, KA, TB, DC, DH, HJ, CK, GR and MW received honoraria for advisory boards from ViiV Healthcare. JM received honoraria for advisory boards, lectures or research grants from Gilead, ViiV Healthcare, Roche, Janssen and Merck. CG, AL and MR work full-time at ViiV Healthcare and EF works as a part-time service provider for ViiV Healthcare. LN and AO received consulting fees from ViiV Healthcare for the conduct of the review. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
To accelerate progress toward ending acquired immunodeficiency syndrome (AIDS) as a public health threat by 2030, the Joint United Nations Program on HIV/AIDS (UNAIDS) established the 90-90-90 targets [1]. These ambitious targets aim to diagnose 90% of all people living with HIV (PLHIV), provide antiretroviral therapy (ART) for 90% of those diagnosed and achieve virological suppression in 90% of those treated with ART by 2020. In 2017, an estimated 75% of PLHIV knew their HIV-positive status, of which an estimated 79% were receiving ART among whom 81% were virologically suppressed [1]. Recent epidemiological estimates and programme data from 168 countries in all regions reveal progress but persistent gaps across the HIV care continuum remain [1, 2]. The HIV care continuum constitutes sequential steps of medical care from HIV awareness and prevention to the achievement of virological suppression [2]. To achieve virological suppression, PLHIV need to know their HIV-infection status, be linked and engaged in care, and receive and adhere to the prescribed ART regimen. Effective evidence-based interventions (EBIs) are available for all steps along the HIV care continuum and have been implemented in different geographic settings and contexts with success [2]. Despite a global downward trend in the epidemic, progress along the continuum is variable and several regions are experiencing increases in new infections and a lack of progress toward the UNAIDS 90-90-90 targets [1]. Globally, as of 2017 only 47% of all PLHIV achieved virological suppression, which is far lower than the target of 73%, suggesting many regions are not on track to meet the 2020 target [1].
In order to support the UNAIDS 90-90-90 initiative across diverse contexts, it is essential to identify appropriate EBIs (i.e. relevant for settings given local epidemiology and health infrastructure), understand which EBIs are effective and how these can be implemented, scaled and replicated from single trials of local innovations to broad-scale use [3–5]. This is a recognized goal of implementation science [6]. Poorly specified and evaluated implementation strategies present challenges to those who seek to reproduce or scale up the intervention in different settings and contexts and potentially impede real-world adoption of the EBIs [6].
ViiV Healthcare’s Positive Pathways initiative was developed with the overall objective to support the achievement of the UNAIDS 90-90-90 targets [7], by understanding current evidence-based practice in HIV care in real-world settings [7, 8]. This initiative set out to identify current EBIs along the HIV care continuum in centres across multiple geographies. The first phase of the Positive Pathways initiative was to map EBIs in high-income countries and develop a compendium of EBIs across the HIV care continuum as well as a self-assessment questionnaire. EBIs were thematically grouped and prioritised in terms of potential impact and practicality. A final compendium of 21 EBIs across six key themes of current HIV practice was established (Fig 1; for more details on the development of the Positive Pathways initiative refer to S1 Fig). The aim was to share the compendium and questionnaire with other HIV care centres to support the delivery of EBIs to increase prevention, diagnosis, linkage to care and retention in care.
From the compendium of 21 interventions, 12 were prioritized by an expert panel across six key themes of current HIV practice (interventions shaded under each of the six themes). Prioritization was based on a consideration of feasibility/perceived ease for care centres to trial the EBI. These EBIs are expected to be more widely used, investigated and reported. These 12 EBIs from the Positive Pathways initiative were included in the scope of the review. For details on the development of the compendium, refer to S1 Fig. ART, antiretroviral therapy; EBI, evidence-based intervention; HIV, human immunodeficiency virus; PrEP, pre-exposure prophylaxis.
During the development stage of the Positive Pathways initiative, it became apparent that effective knowledge transfer to share and embed EBIs in real-world settings would benefit from an implementation science approach. Implementation science is defined in the HIV Lancet as a ‘multidisciplinary specialty that seeks generalisable knowledge about the behaviour of stakeholders, organisations, communities, and individuals to understand the scale of, reasons for, and strategies to close the gap between evidence and routine practice for health in real-world contexts’ [9].
Before expanding the Positive Pathways initiative to other geographical regions, we sought to understand the extent to which identified EBIs were evaluated using implementation science within the targeted geographical area. To better understand the current use of implementation science in HIV care, only a selected set of EBIs were considered for this review. From the compendium of 21 EBIs, 12 were prioritized by an expert panel across six key themes of current HIV practice (interventions shaded under each of the six themes). Prioritization was based on a consideration of feasibility/perceived ease for care centres to trial the EBI. These 12 EBIs are expected to be more widely used, investigated and reported (Fig 1).
This information was considered instrumental in engaging care centres in the choice and adaptation of EBIs to their respective settings. Practically, given the focus on evaluation, studies using implementation outcome measures and related study designs may provide a broad appreciation for the use of implementation science in HIV care. Also, as implementation outcomes are key intermediate results in relation to clinical effectiveness, measures of effectiveness are also important to considering the use of implementation science.
Therefore, using measures and methods aligned with implementation science, we set out to summarise and critically appraise the evidence to obtain a better understanding of the current state of implementation science in HIV in high-income countries.
Methods
The review focused on 12 EBIs (Fig 1; refer to S1 Fig for more information) and was conducted according to guidelines in the Cochrane Handbook for Systematic Reviews of Interventions [10] and Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses (PRISMA) [11] to obtain relevant information using a reproducible, robust and transparent methodology. In-line with these guidelines, we developed a study protocol in which the search strategy and study eligibility criteria were established prior to conducting the review. After this, searches were performed and retrieved publications were assessed for eligibility in a two-phase screening process based on predefined eligibility criteria. From the final list of publications considered relevant for this review, addata were extracted, the scope of which was also established a priori. As the final step, we synthesised key findings from the data. The review methodology is detailed below.
Search sources and strategy
Given the objective of this review, we searched the following 10 databases: Medline, Embase, ABI/INFORM, Adis Pharmacoeconomic & Outcomes News, Allied and Complementary Medicine, DH-DATA: Health Administration Medical Toxicology and Environmental Health, Gale Group Health Periodicals Database, Lancet titles, New England Journal of Medicine, and Cochrane Central Register of Controlled Trials [12–21]. In the literature search strategy, we included both free-text and Emtree/MeSH terms for HIV, the 12 EBIs (Fig 1) and implementation outcomes. The search terms for implementation science were identified from previously published literature [6, 22, 23]. Hickey et al. [6] was used as the basis for the development phase of this study whereas Proctor et al. [23] was used to derive relevant implementation and service outcome search terms. This taxonomy [23] was chosen to guide the review as it is a widely used evaluation framework in the field of implementation science. Also, given the clear link between these outcomes and the evaluation of implementation strategies, the review was expected to be sensitive to detecting any research potentially aligned with implementation science. Also, Curran et al. [22] was used to identify relevant study designs for inclusion. As this review focused both on implementation and service (e.g. effectiveness) outcomes, we considered it appropriate to include hybrid study designs to be able to capture relevant publications for this review [22].
We searched the databases simultaneously via ProQuest [24], with the exclusion of the Cochrane Central Register of Controlled Trials Database which was searched via the Cochrane Library [25] (Search date: 29 March 2018). We applied different limits to the searches. This included restricting the geographical scope of the review to studies conducted in the United States, United Kingdom, France, Germany, Italy Spain, Canada, Australia, and Europe. This was done because the EBIs resulting from the Positive Pathways initiative involved only high-income countries. Furthermore, the review was restricted to English only publications and publication year from 2013 to 2018. Considering the fact that implementation science is an emerging field within HIV with guidance published in 2011 and 2012 to advance the understanding of implementation science [22, 23], the search timeframe of five years was deemed appropriate by the authors to identify relevant publications (for full details on the search strategy refer to S1 Table).
Inclusion and exclusion criteria
Eligibility criteria for this review are provided in Table 1. In this review, publications were eligible if they reported on PLHIV or individuals at risk of being infected with HIV and who received an intervention that could be categorised into the 12 EBIs (Fig 1; refer to S1 Fig for more information). For inclusion it was required that the publication reported on outcomes related to the implementation, effectiveness or both, of the intervention. Including both implementation and effectiveness outcomes as per Proctor and colleagues [23] allowed for a review that was comprehensive in scope and could produce the most accurate overview of implementation science in the targeted countries. Implementation outcomes were determined using Proctor et al.’s taxonomy [23]. The effectiveness outcomes of interest (i.e. linkage to care, retention in care and medication adherence) were chosen because of their key role in achieving the UNAIDS 90-90-90 targets. We considered a broad range of study designs for inclusion including observational and experimental study designs (such as randomised controlled trial [RCT]), qualitative study designs (such as focus groups and interviews) as well as hybrid study designs which combine attributes of both quantitative and qualitative data collection.
Screening and selection
After the searches were performed, identified publications were screened in two phases, with reviews divided between three reviewers (LN, MB and NB). The first phase included screening of titles and abstracts of all publications based on the eligibility criteria followed by a second phase which included reviewing the full-texts of articles using the same criteria (Table 1).
Data extraction and descriptive analyses
After we identified the eligible publications for this review, one reviewer (AO, NB, MB) extracted the relevant data from these publications. A second reviewer (LN) quality checked the data extracted. Discrepancies were resolved through discussion and consensus between the reviewers. We determined data extraction parameters a priori and included intervention details (e.g. type of intervention, including category of EBI, location of intervention, target population), implementation outcomes (e.g. parameter assessed, methodology of assessment, use of a theoretical framework and reported values of the parameter assessed) and effectiveness outcomes (i.e. linkage to care, retention to care and medication adherence). Transferable implementation strategies are required to ensure the consistent use of evidence to change healthcare policy and practice, therefore theoretical frameworks play a key role in implementation science [26, 27]. Given the number of theoretical frameworks available to evaluate the implementation of EBIs, we considered it most appropriate to focus on the taxonomy of implementation outcomes as defined by Proctor et al. [23]. However, we included theoretical frameworks as a relevant parameter for data extraction.
In this paper we provide a descriptive overview of the types of EBIs identified using the extracted dataset. This is followed by a discussion on the distribution of EBIs across three possible categories of evaluation: evaluation of both implementation and effectiveness, evaluation of implementation, and evaluation of effectiveness. Also, we provide a description on the implementation outcomes and effectiveness outcomes that have been documented across the 12 EBI categories. Evaluation of the EBIs using implementation and effectiveness outcomes are reported separately in the results section. As this review aims to document and better understand the current state of implementation science in HIV, our results focus primarily on the identification of documented implementation and service outcomes and the methodologies used to evaluate the implementation of EBIs. As implementation outcomes are key intermediate results in relation to clinical effectiveness, effectiveness outcomes were included in this review as secondary outcomes of interest. The description on effectiveness outcomes only focuses on the identification of documented effectiveness outcomes.
Results
Included studies
A total of 4,241 publications were identified from the databases (Fig 2). After the removal of duplicates, the title and abstracts of 3,908 publications were screened for eligibility. After excluding 3,451 publications based on title and abstract screening, 457 full-text publications were assessed for full-text eligibility based on the pre-specified criteria (see Table 1). A total of 339 publications were excluded after full-text screening. Reasons for exclusion were due to the study population (n = 13), intervention (n = 106), outcomes (n = 141), study design (n = 34), geographic scope (n = 44) and language (n = 1). A total of 118 publications were included in the review [28–145] (refer to S1 File for the list of publications excluded after full-text screening).
PRISMA, Preferred Reporting Items for Systematic Literature Reviews and Meta-Analyses. †Comprises the records identified via Medline, Embase, ABI/INFORM, Adis Pharmacoeconomic & Outcomes News, Allied and Complementary Medicine, DH-DATA: Health Administration Medical Toxicology and Environmental Health, Gale Group Health Periodicals Database, Lancet Titles, and the New England Journal of Medicine.
Study and intervention characteristics
From a total of 118 publications, a total of 145 single and combination EBIs were identified. These 145 interventions were categorised into the 12 prioritised EBI categories resulting in a total of 225 EBIs (note: number is higher as several interventions involved more than one category of the prioritised EBIs). Results from this review are reported for 225 EBIs (N, number of EBIs).
Most of the 225 EBIs were implemented in the United States (N = 167, 74%) followed by Australia (N = 21, 9%), Canada (N = 13, 6%), United Kingdom (N = 8, 4%), France (N = 10, 4%), Italy (N = 1, <1%) and Spain (N = 5, 2%). No publications were identified for Germany. Of the 12 prioritised EBI categories “rapid access to testing services” was the most frequently implemented EBI (N = 66, 29%) followed by the “role of care navigators” (N = 63, 28%) and “structured follow up” (N = 35, 16%). A variety of study designs was used for the implementation and evaluation of EBIs. The majority of the 225 EBIs were implemented and evaluated using a hybrid study design (N = 94, 42%), followed by quantitative study design (N = 45, 20%), qualitative study design (N = 39, 17%), clinical observational study design (N = 25, 11%) and RCT study design (N = 22, 10%; Table 2). A detailed overview of study and EBI characteristics is provided in S2 Table.
Evaluation of EBIs
Fig 3 provides a distribution of the EBIs across the three evaluation categories. Of the 225 EBIs, 144 EBIs were evaluated only on their implementation. “Rapid access to testing services” (N = 50) and “role of care navigators” in HIV care and management (N = 44) were most often evaluated exclusively on their implementation. In total, 42 EBIs were evaluated only on their effectiveness, with the EBIs “structured follow-up” (N = 14) and “role of care navigators” (N = 7) most commonly documented. In addition, 39 EBIs were evaluated on both implementation and effectiveness. Of this latter group, “rapid access to testing services” (N = 13) and “role of care navigators” (N = 12) were most frequently assessed.
N represents the total number of EBIs included in this review. n represents the number of publications in which these EBIs are evaluated. ART: antiretroviral therapy; HIV: human immunodeficiency virus.
Of the 225 EBIs, the majority (N = 183, 81%) were evaluated on their implementation. In addition, a total of 81 EBIs were evaluated on their effectiveness (see Table 2).
Implementation outcomes.
Of the 183 EBIs where implementation was evaluated, 59 EBIs (32%) involved two implementation outcomes, resulting in a total of 242 documented implementation outcomes. A high level of variability was observed in the definitions of reported implementation outcomes. For example, one study used fidelity to evaluate whether the intervention was delivered as intended. Fidelity was defined by two components, exposure and engagement. The implementers defined exposure as the proportion of text messages sent successfully, and specified engagement with study text messages as the number of months in which a requested response to the study text message was received [30]. In another study, fidelity was used to evaluate the quality and adherence of a trained HIV therapist. The implementers defined quality as the competence of the trained HIV therapist to deliver the intervention and adherence was defined as the % of the session content provided by the HIV therapist being aligned with the study protocol [109].
Across the eight implementation outcomes of interest, acceptability (N = 100) and implementation costs (N = 55) were most often reported. The outcome acceptability was most commonly used for “rapid access to testing services” (N = 48) and “role of care navigators” (N = 22). The assessment of the implementation costs was primarily undertaken in the group of EBIs focused on the “role of care navigators” (N = 24), “individualised plan of care” (N = 12) and “structured follow-up” (N = 10; Table 3). The predominance of implementation cost as an outcome for these EBIs is evident due to the involvement of human resources and their potential impact on healthcare systems.
Of the 242 implementation outcomes reported, 23 implementation outcomes were evaluated by more than one methodological approach, resulting in a total of 265 reported methodologies to evaluate the implementation of the EBIs. The methods reported were classified into three categories: questionnaires, interviews, and frameworks.
A mix of quantitative and qualitative methods was used to assess the implementation of the EBIs (see Fig 4). For example, in a recent study, the acceptability of an opt-out inpatient HIV screening at an urban teaching hospital was evaluated by using a questionnaire. To determine the acceptability and to describe the predictors of acceptance or refusal of HIV opt-out inpatient testing, surveys were offered to two samples: a) adult patients admitted to the hospital who had been offered an HIV test upon admission over a 3-month period and b) the medical staff of the hospital who offered the HIV tests. The survey consisted of a 5-point Likert-scale and multiple-choice questions [55]. In another study, the acceptability of a mobile health intervention to improve HIV care coordination for PLHIV with co-morbidities was evaluated by applying an interview approach. The first 12 study participants and three peer navigators were asked for their perceptions about the usefulness of the intervention in a one-on-one, in-depth, semi-structured interview [61]. Another example of the use of quantitative methods is a retrospective medical record review of patient-level data in an urban academic medical centre that was used to determine the acceptance rate of HIV testing services and to identify reasons for declining [121].
N represents the total number of implementation outcomes reported. n represents the number of publications in which these implementation outcomes are reported. †The numbers reported do not add up to the total number of reported implementation outcomes (N = 242) as multiple methods could be used to evaluate the implementation outcome.
As presented in Fig 4, amongst all evaluations, only a small number of evaluations (N = 13) involved the use of a theoretical framework. Evaluations involving a theoretical framework were applied to five of the 12 prioritised EBIs, and included: “rapid access to testing services” (N = 2), “access to mental health services” (N = 2), “role of the care navigators” (N = 5), “individualised plan of care” (N = 1) and “structured follow-up” (N = 3). All the EBIs were implemented in the United States, primarily in HIV care clinics (N = 9) and were studies involving fewer than 200 study participants (N = 10). The EBIs were evaluated using the following frameworks: Theory of Reasoned Action and Social Cognitive Theory; ADAPTS framework, Information, Motivation, Behavioural Skills Model; Constant Comparative Method as described by Glaser and Strauss; Grounded theory analysis by Strauss & Corbin; Health Belief Model; Theory of Planned Behaviour and Reasoned Action; Trans-Theoretical Model and Precaution Adoption Process Model [35, 46, 61, 64, 104, 107, 133, 136]. All but one of these frameworks (ADAPTS) originates from disciplines external to implementation science [146] and provides guidance to researchers to study the implementation of an intervention (see Table 4) [136].
Effectiveness.
In total, 81 EBIs were evaluated on effectiveness outcomes (Table 5). Among the 81 EBIs, 17 EBIs were evaluated for two effectiveness outcomes and one EBI was evaluated for three effectiveness outcomes, resulting in a total of 100 effectiveness outcomes reported overall. Retention in care (N = 42) and linkage to care (N = 41) were more frequently considered for the evaluation of intervention effectiveness compared to medication adherence (N = 17). The effectiveness outcomes reported were consistent with the objectives of the identified EBIs. Evaluation of retention in care was most commonly documented for the following EBIs: “structured follow-up” (N = 14), “role of care navigators” (N = 12) and “individualised plan of care” (N = 6). The EBI “rapid access to testing services” (N = 16), “role of care navigators” (N = 10) and “structured follow-up” (N = 7) were most often evaluated for linkage to care. Medication adherence was evaluated for EBIs that focused on care management, namely: “role of care navigators” (N = 5), “individualised plan of care” (N = 4), “structured follow-up” (N = 4) and “access to mental health services” (N = 3).
Significant levels of variation were observed in the evaluation of effectiveness. This was due to the lack of standardised measures and definitions for the measurement of effectiveness. As a result, a wide range of reported measures of effectiveness were observed with too many variations and inconsistencies to report.
Discussion
With its focus on 12 of the EBIs identified within the Positive Pathways initiative, the findings of this review provide valuable insights into the current state of implementation science in real-world HIV care settings. As such, it provides a valuable context for consideration in the adaptation of EBIs identified in the Positive Pathways initiative, as well as highlighting the progress that remains in maximizing implementation science within HIV to obtain the biggest impact, especially with the 90-90-90 initiative.
In this review, we found 118 publications covering 225 EBIs spanning across the 12 prioritised EBI categories. Of these EBIs, “rapid access to testing services” was most frequently evaluated followed by “role of care navigators” and “structured follow up”. Of these 225 EBIs, 183 were evaluated on implementation. Significant variability was observed in the definitions of reported implementation outcomes. The variability in definitions represents a challenge for implementers to effectively evaluate and understand what EBI works where, how and with whom, as the reported outcomes are not comparable. Consequently, the challenge to bridge gaps in the HIV care continuum remains.
Very few implementation outcomes were being considered for the evaluation of EBIs, which may ultimately limit adaptation of EBIs in the real-world setting. Among the 183 EBIs assessed for implementation outcomes, acceptability and implementation costs were most commonly evaluated, whereas fidelity was rarely reported. This could be attributable to the underlying methods needed to assess these outcomes. Evaluations of acceptability and implementation costs use methods familiar to clinical settings, such as questionnaires and data analysis. In contrast, fidelity measures often require more complex methods, such as an audio and video recording, the development of tailored checklists, and related analyses to assess healthcare professionals’ adherence to study protocols. A mix of quantitative and qualitative methods were used to evaluate the EBIs, however, there was a lack of consistent use of methodologies to evaluate the implementation of EBIs. The variability in reported methodologies suggests that either researchers do not seem to use implementation science approaches, or this variability is perhaps a consequence of the large number of available frameworks. Of the reported evaluations, only 13/183 used a theoretical framework, indicating a knowledge gap in implementation science in HIV. As transferable implementation strategies are required to ensure the consistent use of evidence to change healthcare policy and practice, theoretical frameworks play a key role in implementation science [26, 27]. Only one out of the of the eight frameworks identified in this review provides guidance to researchers to study the implementation of an intervention [136]. The high level of variety in definitions and methodologies used, the disparity in reported implementation outcomes and the minimal use of theoretical frameworks reported in our review suggests that the evaluation of EBIs along the HIV care continuum is not yet aligned with implementation science principles.
Approximately one-third of EBIs were evaluated on effectiveness, most often on linkage and retention in care. Given the critical role that linkage to care plays after HIV diagnosis, it was to be expected that the evaluation of the EBI “rapid access to testing services” was largely measured using linkage to care. In addition, EBIs with a focus on improving care (such as the role of care navigators, individualised care plans and diagnosis and management of co-morbidities) were also evaluated for effectiveness. Substantial levels of variation were observed in the evaluation of effectiveness. Proctor et al. state that implementation outcomes are key intermediate results in relation to clinical effectiveness [23]. Given the inconsistent approaches to assessing implementation, it is not surprising that similar levels of variability were observed with the evaluation of effectiveness, both in terms of definition and methodology used.
Successful implementation of EBIs to support the HIV care continuum in real-world settings, is essential to achieve UNAIDS 90-90-90 targets. However, this review shows evaluations of EBIs in real-world settings, either on implementation or effectiveness outcomes, do not appear to be making optimal use of available implementation science approaches.
These findings corroborate the conclusions of a recent literature review by Hickey et al.[6] which suggested that researchers and implementers continue to face challenges to transfer effective EBIs from one setting to another, or scale up the intervention within the same setting. This could potentially undermine progress toward achieving the UNAIDS 90-90-90 targets. Implementation researchers need to be able to compare and prioritise effective interventions that contribute to achieving optimal health outcomes for PLHIV. It could be argued, that without the consistent use of implementation science, challenges remain to effectively close the gap between evidence and the effective use of EBIs in real-world settings.
We recognise there are limitations to this work. The first phase of the Positive Pathways initiative was to map EBIs in high-income countries and to develop the compendium and self-assessment questionnaire. Before expanding to other geographical regions, we wanted to contextualise the initial findings with a review to understand the extent to which identified EBIs were being evaluated using implementation science. Future reviews of this nature could involve more geographical regions, including low and middle-income countries. Secondly, we restricted our search to a period of five years (2013–2018) and publications written in English only which could potentially limit the generalisability of our findings. Thirdly, we did not conduct all screening activities with two independent reviewers and did not perform a risk of bias analysis for the publications included. As this review aims to obtain a better understanding of the current state of the use of implementation science in HIV and did not aim to evaluate the quality of reported implementation outcomes and methodologies used for evaluation, a detailed data analysis was not included. Therefore, having one researcher conduct screening activities was considered appropriate and a risk of bias analysis not necessary.
Furthermore, this review was not registered in a database for systematic reviews, which may have influenced the level of transparency of this review. However, this review was conducted according to the guidelines in the Cochrane Handbook for Systematic Reviews of Interventions [10] and PRISMA [11] which does minimise the risk of bias in the conduct of this review. In addition, we observed a high level of variability in the definitions used for commonly reported implementation outcomes which required us to interpret where the outcome was best-suited to fit within the taxonomy of the eight Proctor et al. [23] implementation outcomes. In addition, studies using implementation outcomes as a measurement of evaluation were considered eligible for inclusion. However, given the sensitive and non-specific nature of these outcomes, using these outcomes does not mean that implementation science principles were necessarily applied in a study. Given the objective of this review to identify the current state of implementation science in HIV, we qualified a broad range of study designs such as quantitative studies, qualitative studies and hybrid designs, which evaluated either an implementation or effectiveness outcome, or both to this review. It can be argued whether the eligible studies were designed according to accurate implementation science principles, but as both implementation and effectiveness outcomes were of interest for this review, the interrelated link between the two outcomes, and the limited HIV implementation science literature, publications that only reported effectiveness outcomes, without an implementation science focus were also included in this review. For example, an RCT that assessed the retention to care of HIV patients in a real-world setting was considered eligible for inclusion as it reported an effectiveness outcome of interest, even if it did not provide any details about the implementation of the EBI. Adding to this is the inclusion of all hybrid designs that met inclusion criteria for this review. Given that the hybrid methodology is relatively new, these types of study designs remain inconsistently reported by name in the literature. Therefore, studies that reported both on effectiveness and implementation of EBIs were labelled by the reviewers as hybrid design studies using the criteria from Curran et al. [22]. This self-labelling approach may have resulted in misclassification of effectiveness or implementation studies as they may not actually be that type of study design given the variability in interpretation and understanding of these types of studies. This approach is somewhat subjective, and therefore our assessments may be imperfectly reproducible which is not ideal but is reflective of the current state of implementation science in HIV care.
Lastly, this review focused on the taxonomy of implementation outcomes of Proctor et al. [23], as it is widely used and accepted in the field of implementation science for the evaluation of EBIs. The taxonomy provides a classification of implementation outcomes and is therefore often considered in theoretical frameworks that focus on the evaluation of implementation science strategies. Many other theoretical frameworks and models for determining feasible implementation strategies are available. Given the number of theoretical frameworks, it was considered more appropriate to use the taxonomy of Proctor et al. [23] for the search strategy. The inclusion of theoretical frameworks in the search strategy may have restricted the identification of theoretical frameworks and introduce bias. Therefore, in this review only theoretical frameworks that are aligned with Proctor et al. [23] are considered. Overall, the approach used in the current paper likely captures studies that do not meet the strict implementation science criteria. However, we have still captured several implementation studies as evidenced by the proportion of studies citing the use of a hybrid methodology. Regardless, the lack of consistency in labelling a study as implementation research and adhering to proper methodology and reporting in HIV studies in this review remains a large problem. This issue highlights the need for more capacity building in implementation science within HIV research.
Conclusion
This systematic literature review provides an empirical review of implementation science approaches used to evaluate 12 EBIs in support of the HIV continuum. The learnings from this review highlight the need for a robust implementation science approach to optimise the use of EBIs in HIV care. Variability in how implementation science is applied to HIV, as seen in the ways implementation and effectiveness are evaluated and inconsistency in reporting of measures, methods and outcomes, needs to be addressed if we are to scale up EBIs in support of achieving the UNAIDS 90-90-90 targets. The lack of consistency in application reporting of key implementation science elements found in this study is consistent with the work done in sub-Saharan Africa [147]. Notably, the field of HIV is behind many other fields with respect to utilizing implementation science to improve health outcomes. To successfully scale and replicate EBIs in different settings and contexts there is a need to ensure the use of theoretical frameworks and consistent approaches for the evaluation of implementation outcomes. This will improve understanding of what EBI works where, how and with whom and will bridge the gaps in the HIV care continuum. Importantly, the consistent and accurate utilization and reporting of implementation science components of HIV studies in the future is crucial in our ability to end the epidemic.
Supporting information
S1 Fig. Positive pathways initiative: Development of the compendium of EBIs.
Objective: The ViiV Healthcare Positive Pathways initiative was established to define best practice and interventions that can effectively close the gaps in the treatment continuum. Development methodology: The methodology used to develop the compendium was divided in the three phases. Phase 1 literature search: To develop a baseline of the scope and range of activities of current evidence-based practice, a literature search of key published evidence was undertaken from 2010 to 2016. This was a non-systematic literature review using PubMed and Google Scholar as research databases and focusing on geography in the scope of the project. We reviewed a range of articles and journals, from internationally approved guidelines (WHO, IAPAC, CDC, ECDC, NICE and DHHS) for evidence-based practice care and management as well as reviews and observational studies in single centres. An initial list of 66 EBIs was developed from this first phase of the work. Phase 2 site visits: In the next phase of the work, we visited eight established centres delivering HIV care to observe current practice and establish the extent and scope of EBIs in use. Centres were identified through the ViiV Healthcare network and selected based on their commitment to UNAIDS 90-90-90 targets and willingness to participate in the initiative including the dissemination of findings through data publication in a relevant peer-reviewed journal. Centres were in Western Europe and North America. Performance data relating to the 90-90-90 targets, were collected and we conducted over 100 interviews across a wide range of stakeholders. An interview guide was created to facilitate structured collection of quantitative and qualitative data on HIV care and management across the care continuum. This included centre and community involvement in HIV awareness and prevention through to disease diagnosis, linkage to and retention in care and clinical management and follow-up. Interview participants were selected by the lead experts within each centre. Selection criteria were based on a participant’s input and level of experience in care and management of HIV patients. We observed HIV care being delivered in a wide range of care settings that included Specialist HIV Centres, Infectious Disease Departments, Sexual Health Clinics and Primary Care Centres, each with specific features. Observations at site visits were cross-referenced against the findings of our secondary literature review to help develop the compendium and HIV care and management assessment questionnaire. Phase 3 development of compendium: In the final phase to develop the compendium three advisory boards with 12 experts were held to test and prioritise the key intervention categories and interventions resulting from the first and second phase. Participants included experts from the participating centre, a health economist, patient advocacy group representative and a healthcare systems manager. Within this programme, EBIs were thematically grouped into six categories and prioritised in terms of impact and practicality for implementation leading to the establishment of a final compendium of 21 EBIs. From the compendium of 21 interventions, 12 were prioritized by an expert panel across six key themes of current HIV practice (interventions shaded under each of the six themes). Prioritization was based on a consideration of feasibility/perceived ease for care centres to trial the EBI. These EBIs are expected to be more widely used, investigated and reported. These 12 EBIs from the Positive Pathways initiative were included in the scope of the review.
https://doi.org/10.1371/journal.pone.0220060.s001
(DOCX)
S1 Table. Medline, Embase, ABI/INFORM, Adis Pharmacoeconomic & Outcomes News, Allied and Complementary Medicine, DH-DATA: Health Administration Medical Toxicology and Environmental Health, Gale Group Health Periodicals Database, Lancet Titles, and New England Journal of Medicine (via ProQuest).
https://doi.org/10.1371/journal.pone.0220060.s002
(DOCX)
S2 Table. Overview of study and intervention characteristics of publications included in the review (N = 224; n = 118)†.
https://doi.org/10.1371/journal.pone.0220060.s003
(DOCX)
S1 File. List of Publications excluded after full-text screening [1–339].
https://doi.org/10.1371/journal.pone.0220060.s004
(DOCX)
Acknowledgments
The authors would like to thank Celine Aubin (ViiV Healthcare) for her role in the initial set-up of the systematic literature review, KPMG LLP UK for their contribution in the development of the compendium for the ViiV Healthcare Positive Pathways initiative, Nicolai Braun, and Martina Bianco (both employees of Pharmerit International) for their contribution to the review (screening, data extraction and preparation of tables and figures for this publication). Editorial assistance was provided under the direction of the authors by MedThink SciCom.
References
- 1. UNAIDS. Ending AIDS: Progress towards the 90-90-90 targets. 2017:1–196.
- 2. Gardner EM, McLees MP, Steiner JF, Del Rio C, Burman WJ. The spectrum of engagement in HIV care and its relevance to test-and-treat strategies for prevention of HIV infection. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2011;52(6):793–800.
- 3. Ben Charif A, Zomahoun HTV, LeBlanc A, Langlois L, Wolfenden L, Yoong SL, et al. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implementation Science: IS. 2017;12:139. pmid:29166911
- 4. Tafireyi M, Mark S, Shahin L, A ML, Tendani G, Refeletswe L, et al. A tale of two countries: progress towards UNAIDS 90‐90‐90 targets in Botswana and Australia. Journal of the International AIDS Society. 2018;21(3):e25090.
- 5. Bain LE, Nkoke C, Noubiap JJN. UNAIDS 90–90–90 targets to end the AIDS epidemic by 2020 are not realistic: comment on “Can the UNAIDS 90–90–90 target be achieved? A systematic analysis of national HIV treatment cascades”. BMJ Global Health. 2017;2(2):e000227. pmid:28589026
- 6. Hickey MD, Odeny TA, Petersen M, Neilands TB, Padian N, Ford N, et al. Specification of implementation interventions to address the cascade of HIV care and treatment in resource-limited settings: a systematic review. Implement Sci. 2017;12(1):102. pmid:28784155
- 7.
E Le Fevre JC, M Wolfheiler, D Hardy, C Katlama, G Rizzardini, J Beckett, H Jessen JM, K Alcorn, editors. Improving the HIV Care Continuum—Developing a Compendium of Observed Evidence-Based Practice. European AIDS Conference (EACS); 2017; Milan, Italy.
- 8. ViiV Healthcare. HIV best practice management report. Data on File. 2016.
- 9. Odeny TA, Padian N, Doherty MC, Baral S, Beyrer C, Ford N, et al. Definitions of implementation science in HIV/AIDS. The Lancet HIV. 2015;2(5):e178–e80. pmid:26423000
- 10. Higgins JPT GS. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0. The Cochrane Collaboration, 2011; 2011.
- 11. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine. 2009;6(7):e1000097. pmid:19621072
- 12.
MEDLINE Database [Internet]. 2018. Available from: https://www.nlm.nih.gov/bsd/pmresources.html.
- 13.
ABI/INFORM [Internet]. 2018. Available from: https://www.proquest.com/products-services/abi_inform_complete.html.
- 14.
Elsevier Limited. Lancet Titles Database. 2018.
- 15.
Embase Database [Internet]. 2018. Available from: https://www.elsevier.com/solutions/embase-biomedical-research#search.
- 16.
Gale Group Health Periodicals Database [Internet]. 2018. Available from: https://www.gale.com/uk.
- 17.
New England Journal of Medicine Database [Internet]. 2018. Available from: https://www.nejm.org/.
- 18.
Cochrane Central Register of Controlled Trials (CENTRAL) Database [Internet]. 2018. Available from: http://www.cochranelibrary.com/about/central-landing-page.html.
- 19.
DH-Data: Health Administration, Medical Toxicology and Environmental Health Database [Internet]. 2018.
- 20.
The Allied and Complementary Medicine Database [Internet]. 2018. Available from: https://www.bl.uk/collection-guides/science-electronic-resources.
- 21.
Adis PharmacoEconomics & Outcomes News Database [Internet]. 2018. Available from: https://www.springer.com/adis/journal/40274.
- 22. Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact. Medical care. 2012;50(3):217–26. pmid:22310560
- 23. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration and Policy in Mental Health. 2011;38(2):65–76. pmid:20957426
- 24. ProQuest LLC. ProQuest Dialog 2018 [Available from: https://dialog.proquest.com/professional/Login.
- 25. Cochrane. Cochrane Library 2018 [Available from: http://www.cochrane.org/.
- 26. Schackman BR. Implementation science for the prevention and treatment of HIV/AIDS. Journal of acquired immune deficiency syndromes (1999). 2010;55 Suppl 1:S27–31.
- 27. The Foundation for AIDS Research (amfAR). Towards defining an HIV implementation science agenda. 2015:1–17.
- 28. Addison D, Baim-Lance A, Suchman L, Katz B, Swain CA, Piersanti K, et al. Factors Influencing the Successful Implementation of HIV Linkage and Retention Interventions in Healthcare Agencies Across New York State. AIDS Behav. 2018.
- 29. Bentsianov S, Padilla D. Beyond Treatment as Prevention (TASP): Use of a Mobile Unit to Screen for Human Immunodeficiency Virus and Link High-Risk, Negative Adolescents and Young Adults to Prep Services. Journal of Adolescent Health. 2018;62(2):S38–S9.
- 30. Christopoulos KA, Riley ED, Carrico AW, Tulsky J, Moskowitz JT, Dilworth S, et al. A Randomized Controlled Trial of a Text Messaging Intervention to Promote Virologic Suppression and Retention in Care in an Urban Safety-Net HIV Clinic: The Connect4Care (C4C) Trial. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2018.
- 31. Demorat H, Lopes A, Chopin D, Delcey V, Clevenbergh P, Simoneau G, et al. Acceptability and feasibility of HIV testing in general medicine by ELISA or rapid test from finger-stick whole blood. Presse Med. 2018;47(2):e15–e23. pmid:29478794
- 32. Menza TW, Choi SK, LeGrand S, Muessig K, Hightow-Weidman L. Correlates of Self-Reported Viral Suppression Among HIV-Positive, Young, Black Men Who Have Sex With Men Participating in a Randomized Controlled Trial of An Internet-Based HIV Prevention Intervention. Sex Transm Dis. 2018;45(2):118–26. pmid:28876283
- 33. Saag LA, Tamhane AR, Batey DS, Mugavero MJ, Eaton EF. Mental health service utilization is associated with retention in care among persons living with HIV at a university-affiliated HIV clinic. AIDS Res Ther. 2018;15(1):1. pmid:29338735
- 34. Shacham E, Lopez JD, Brown TM, Tippit K, Ritz A. Enhancing Adherence to Care in the HIV Care Continuum: The Barrier Elimination and Care Navigation (BEACON) Project Evaluation. AIDS Behav. 2018;22(1):258–64. pmid:28597342
- 35. Smith LR, Amico KR, Fisher JD, Cunningham CO. 60 Minutes for health: examining the feasibility and acceptability of a low-resource behavioral intervention designed to promote retention in HIV care. AIDS Care. 2018;30(2):255–65. pmid:28657333
- 36. Yee LM, Miller ES, Statton A, Ayala LD, Carter SD, Borders A, et al. Sustainability of Statewide Rapid HIV Testing in Labor and Delivery. AIDS Behav. 2018;22(2):538–44. pmid:28986656
- 37. Bean MC, Scott L, Kilby JM, Richey LE. Use of an Outreach Coordinator to Reengage and Retain Patients with HIV in Care. AIDS Patient Care STDS. 2017;31(5):222–6. pmid:28488904
- 38. Bouris A, Jaffe K, Eavou R, Liao C, Kuhns L, Voisin D, et al. Project nGage: Results of a Randomized Controlled Trial of a Dyadic Network Support Intervention to Retain Young Black Men Who Have Sex With Men in HIV Care. AIDS Behav. 2017;21(12):3618–29. pmid:29079949
- 39. Castel Amanda D., Kuo Irene, Mikre Meriam, Young Toni, Meredith Haddix, Das Suparna, Maugham Geoffrey, and Reisen Carol,. Feasibility of Using HIV Care Continuum Outcomes to Identify Geographic Areas for Targeted HIV Testing. Journal of acquired immune deficiency syndromes (1999). 2017;74(Suppl 2):S96–S103.
- 40. Coleman TE, LeViere A, Carcano J, Bailey M, Heine A, Quinlivan EB, et al. Integrating a Statewide HIV Call Line: An Innovative and Tailored Approach for Rapid Linkage to HIV Care. J Assoc Nurses AIDS Care. 2017;28(6):953–63. pmid:28756114
- 41. Cunningham WE, Weiss RE, Nakazono T, Malek MA, Shoptaw SJ, Ettner SL, et al. Effectiveness of a Peer Navigation Intervention to Sustain Viral Suppression Among HIV-Positive Men and Transgender Women Released From Jail: The LINK LA Randomized Clinical Trial. JAMA Intern Med. 2018;178(4):542–53. pmid:29532059
- 42. Davies SC, Koh A, Lindsay HE, Fulton RB, Fernando SL. Providing HIV results via SMS one day after testing: more popular than rapid point-of-care tests. Int J STD AIDS. 2017;28(7):702–7. pmid:27538723
- 43. Dong BJ, Williams MR, Bingham JT, Tokumoto J, Allen JD. Outcome of challenging HIV case consultations provided via teleconference by the Clinician Consultation Center to the Federal Bureau of Prisons. J Am Pharm Assoc (2003). 2017;57(4):516–9.
- 44. Gomez CA, Tat SA, Allen D, Gordon D, Browe D. What Will It Take to End the HIV/AIDS Epidemic? Linking the Most Disenfranchised Into Care Through Outreach. AIDS Patient Care STDS. 2017;31(3):122–8. pmid:28282248
- 45. Hsieh YH, Beck KJ, Rothman RE, Gauvey-Kern M, Woodfield A, Peterson S, et al. Factors associated with patients who prefer HIV self-testing over health professional testing in an emergency department-based rapid HIV screening program. Int J STD AIDS. 2017;28(11):1124–9. pmid:28114880
- 46. Johnson DM, Johnson NL, Beckwith CG, Palmieri PA, Zlotnick C. Rapid Human Immunodeficiency Virus Testing and Risk Prevention in Residents of Battered Women's Shelters. Womens Health Issues. 2017;27(1):36–42. pmid:27914861
- 47. Kutnick AH, Gwadz MV, Cleland CM, Leonard NR, Freeman R, Ritchie AS, et al. It's a Process: Reactions to HIV Diagnosis and Engagement in HIV Care among High-Risk Heterosexuals. Front Public Health. 2017;5:100. pmid:28540287
- 48. Leblanc J, Jegou C, Fossoux N, Lancien C, Bastide T, Verbrugghe R, et al. Effectiveness of Nurse-Driven HIV Screening Targeting Key Populations in Emergency Departments in Metropolitan Paris: The Anrs DICI-VIH Cluster-Randomized Two-Period Crossover Trial. Clinical Therapeutics. 2017;39(8).
- 49. Leblanc J, Hejblum G, Costagliola D, Durand-Zaleski I, Lert F, de Truchis P, et al. Targeted HIV Screening in Eight Emergency Departments: The DICI-VIH Cluster-Randomized Two-Period Crossover Trial. Ann Emerg Med. 2017.
- 50. Mackie K DA, Poole S, Hopkins R, Hoy J, Dooley M,. PHARMACIST REVIEW OF MEDICATIONS FOR HIV-POSITIVE PEOPLE SEEN IN GENERAL PRACTICE (PROM-GP). Antiviral Therapy. 2017;22:A59.
- 51. Maulsby C, Jain KM, Weir BW, Enobun B, Riordan M, Charles VE, et al. The Cost and Threshold Analysis of Retention in Care (RiC): A Multi-Site National HIV Care Program. AIDS Behav. 2017;21(3):643–9. pmid:27873083
- 52. Rebecca Metcalfe CG, Kilmurray Kim, Carroll Louise, Pollock Kay, Murchie Martin, Smith Hannah,. P153: Using Information Technology to Improve Linkage Into Sexual Health Care in Patients Receiving HIV Post Exposure Prohylaxis for Sexual Exposure (PEPSE) in Emergency Departments Sex Transm Infect. 2017;92(Suppl 1):A1–103.
- 53. Moitra E, LaPlante A, Armstrong ML, Chan PA, Stein MD. Pilot Randomized Controlled Trial of Acceptance-Based Behavior Therapy to Promote HIV Acceptance, HIV Disclosure, and Retention in Medical Care. AIDS Behav. 2017;21(9):2641–9. pmid:28451891
- 54. Mutch AJ, Lui CW, Dean J, Mao L, Lemoire J, Debattista J, et al. Increasing HIV testing among hard-to-reach groups: examination of RAPID, a community-based testing service in Queensland, Australia. BMC Health Serv Res. 2017;17(1):310. pmid:28454592
- 55. Osorio G, Hoenigl M, Quartarolo J, Barger K, Morris SR, Reed SL, et al. Evaluation of opt-out inpatient HIV screening at an urban teaching hospital. AIDS Care. 2017;29(8):1014–8. pmid:28114789
- 56. Anuj V. Patel RER, Abrams Samuel, Jett-Goheen Mary, Angie S. Kim, Signer Danielle, Latkin Carl, Charlotte Gaydos aY-HH. 261: Increasing HIV Testing Through Provision of Home HIV Self-testing Kits to Emergency Department Patients: A Pilot Randomization Study of a Novel Emergency Department Intervention. Acad Emerg Med. 2017;24 Suppl 1:S98.
- 57. Ryan KE, Pedrana A, Leitinger D, Wilkinson AL, Locke P, Hellard ME, et al. Trial and error: evaluating and refining a community model of HIV testing in Australia. BMC Health Serv Res. 2017;17(1):692. pmid:29017561
- 58. Senn TE, Braksmajer A, Coury-Doniger P, Urban MA, Rossi A, Carey MP. Development and Preliminary Pilot Testing of a Peer Support Text Messaging Intervention for HIV-Infected Black Men Who Have Sex with Men. Journal of acquired immune deficiency syndromes (1999). 2017;74(Suppl 2):S121–S7.
- 59. Tesoriero JM, Johnson BL, Hart-Malloy R, Cukrovany JL, Moncur BL, Bogucki KM, et al. Improving Retention in HIV Care Through New York's Expanded Partner Services Data-to-Care Pilot. J Public Health Manag Pract. 2017;23(3):255–63. pmid:27902561
- 60. Uccella I, Petrelli A, Vescio MF, De Carolis S, Fazioli C, Pezzotti P, et al. HIV rapid testing in the framework of an STI prevention project on a cohort of vulnerable Italians and immigrants. AIDS Care. 2017;29(8):996–1002. pmid:28107787
- 61. Westergaard RP, Genz A, Panico K, Surkan PJ, Keruly J, Hutton HE, et al. Acceptability of a mobile health intervention to enhance HIV care coordination for patients with substance use disorders. Addict Sci Clin Pract. 2017;12(1):11. pmid:28441962
- 62. Brondani M, Chang S, Donnelly L. Assessing patients' attitudes to opt-out HIV rapid screening in community dental clinics: a cross-sectional Canadian experience. BMC Res Notes. 2016;9:264. pmid:27165490
- 63. Fernandez-Lopez L, Folch C, Majo X, Gasulla L, Casabona J. Implementation of rapid HIV and HCV testing within harm reduction programmes for people who inject drugs: a pilot study. AIDS Care. 2016;28(6):712–6. pmid:27007000
- 64. Drummond KL, Painter JT, Curran GM, Stanley R, Gifford AL, Rodriguez-Barradas M, et al. HIV patient and provider feedback on a telehealth collaborative care for depression intervention. AIDS Care. 2016:1–9.
- 65. Gardner LI, Marks G, Shahani L, Giordano TP, Wilson TE, Drainoni ML, et al. Assessing efficacy of a retention-in-care intervention among HIV patients with depression, anxiety, heavy alcohol consumption and illicit drug use. AIDS. 2016;30(7):1111–9. pmid:26760454
- 66. Giordano TP, Cully J, Amico KR, Davila JA, Kallen MA, Hartman C, et al. A Randomized Trial to Test a Peer Mentor Intervention to Improve Outcomes in Persons Hospitalized With HIV Infection. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2016;63(5):678–86.
- 67. Jain KM, Maulsby C, Brantley M, team SIFi, Kim JJ, Zulliger R, et al. Cost and cost threshold analyses for 12 innovative US HIV linkage and retention in care programs. AIDS Care. 2016;28(9):1199–204. pmid:27017972
- 68.
Carrie Jeffries PR, Sabrina Matoff-Stepp, Renata Thompson, Jennie L. Harris, Jennifer D. Uhrig, Laura Cheever, editor UCARE4LIFE: Mobile Texting to Improve HIV Care Continuum Outcomes for Minority Youth. Conference on Retroviruses and Opportunistic Infections; 2016; Boston.
- 69. Kenya S, Okoro IS, Wallace K, Ricciardi M, Carrasquillo O, Prado G. Can Home-Based HIV Rapid Testing Reduce HIV Disparities Among African Americans in Miami? Health promotion practice. 2016;17(5):722–30. pmid:27091604
- 70. Lucas KDE, Valorie; Behrends Czarina N.; Charlotte Wheeler; MacGowan Robin J.; Mohle-Boetani Janet C. Evaluation of Routine HIV Opt-Out Screening and Continuum of Care Services Following Entry into Eight Prison Reception Centers—California, 2012. MMWR Morbidity and mortality weekly report. 2016;65(7).
- 71. Metsch LR, Feaster DJ, Gooden L, Matheson T, Stitzer M, Das M, et al. Effect of Patient Navigation With or Without Financial Incentives on Viral Suppression Among Hospitalized Patients With HIV Infection and Substance Use: A Randomized Clinical Trial. JAMA. 2016;316(2):156–70. pmid:27404184
- 72. Preau M, Lorente N, Sagaon-Teyssier L, Champenois K, Gall JM, Mabire X, et al. Factors associated with satisfaction with community-based non-medicalized counseling and testing using HIV rapid tests among MSM in France. AIDS Care. 2016;28(10):1240–8. pmid:27088324
- 73. Safren SA, Bedoya CA, O'Cleirigh C, Biello KB, Pinkston MM, Stein MD, et al. Cognitive behavioural therapy for adherence and depression in patients with HIV: a three-arm randomised controlled trial. The Lancet HIV. 2016;3(11):e529–e38. pmid:27658881
- 74. Stenstrom R, Ling D, Grafstein E, Barrios R, Sherlock C, Gustafson R, et al. Prevalence of HIV infection and acceptability of point-of-care testing in a Canadian inner-city emergency department. Can J Public Health. 2016;107(3):e291–e5. pmid:27763845
- 75. Xiao H, Mains W. Relationship between Housing Status and Retention Rates among HIV-Positive African Americans Enrolled in a Comprehensive Care Program. J Psychoactive Drugs. 2016;48(2):109–14. pmid:26960017
- 76. Bove JM, Golden MR, Dhanireddy S, Harrington RD, Dombrowski JC. Outcomes of a Clinic-Based Surveillance-Informed Intervention to Relink Patients to HIV Care. Journal of acquired immune deficiency syndromes (1999). 2015;70(3):262–8.
- 77. Chan D, Stewart M, Smith M, Price T, Lusk J, Ooi C, et al. The rise of targeted HIV oral rapid testing in Australia. Med J Aust. 2015;202(5):251–5. pmid:25758695
- 78. Conway DP, Guy R, McNulty A, Couldwell DL, Davies SC, Smith DE, et al. Effect of testing experience and profession on provider acceptability of rapid HIV testing after implementation in public sexual health clinics in Sydney. HIV Med. 2015;16(5):280–7. pmid:25604470
- 79. Conway DP, Guy R, Davies SC, Couldwell DL, McNulty A, Smith DE, et al. Rapid HIV Testing Is Highly Acceptable and Preferred among High-Risk Gay And Bisexual Men after Implementation in Sydney Sexual Health Clinics. PLoS One. 2015;10(4):e0123814. pmid:25898140
- 80. Fernandez-Balbuena S, Marcos H, Perez-Rubio A, Hoyos J, Belza MJ, de la Fuente L. The rapid test in Spanish pharmacies: a novel programme to reach heterosexual men? HIV Med. 2015;16(6):362–9. pmid:25689242
- 81. Flash CA, Pasalar S, Hemmige V, Davila JA, Hallmark CJ, McNeese M, et al. Benefits of a routine opt-out HIV testing and linkage to care program for previously diagnosed patients in publicly funded emergency departments in Houston, TX. Journal of acquired immune deficiency syndromes (1999). 2015;69 Suppl 1:S8–15.
- 82. Hakobyan S. Cascade of care of HCV & HIV infected patients identified through community pop-up clinics (CPCs). Hepatology. 2015;62:1131A.
- 83. Hood KB, Robertson AA, Baird-Thomas C. Implementing solutions to barriers to on-site HIV testing in substance abuse treatment: a tale of three facilities. Eval Program Plann. 2015;49:1–9. pmid:25462936
- 84. Hale A, Coombes I, Stokes J, Aitken S, Clark F, Nissen L. Patient satisfaction from two studies of collaborative doctor-pharmacist prescribing in Australia. Health Expect. 2016;19(1):49–61. pmid:25614342
- 85. Katz DG, Matthew; James Hughes; Carey Farquhar; Joanne Stekler. HIV self-testing increases HIV testing frequency among highrisk men who have sex with men: A randomized controlled trial. Journal of the International AIDS Society. 2015;18:95.
- 86. Kielly J, Kelly DV, Asghari S, Burt K, Biggin J. Patient satisfaction with chronic HIV care provided through an innovative pharmacist/nurse-managed clinic and a multidisciplinary clinic. Can Pharm J (Ott). 2017;150(6):397–406.
- 87. Kim JJ, Maulsby C, Zulliger R, Jain K, Positive Charge Intervention T, Charles V, et al. Cost and threshold analysis of positive charge, a multi-site linkage to HIV care program in the United States. AIDS Behav. 2015;19(10):1735–41. pmid:26139421
- 88. Knapp H, Chan K. HIV Rapid Testing in a VA Emergency Department Setting: Cost Analysis at 5 Years. Value Health. 2015;18(5):735–7. pmid:26297102
- 89. Knight VW, Handan; Gray James; Keen Phillip; McNulty Anna; Guy Rebecca. Implementation and Operational Research: Convenient HIV Testing Service Models Are Attracting Previously Untested Gay and Bisexual Men: A Cross-sectional Study. Journal of Acquired Immune Deficiency Syndromes. 2015;69(5):e147–e55. pmid:25970653
- 90. Leber W, McMullen H, Anderson J, Marlin N, Santos AC, Bremner S, et al. Promotion of rapid testing for HIV in primary care (RHIVA2): a cluster-randomised controlled trial. The Lancet HIV. 2015;2(6):e229–e35. pmid:26423195
- 91. Lecher SL, Shrestha RK, Botts LW, Alvarez J, Moore JH Jr., Thomas V, et al. Cost analysis of a novel HIV testing strategy in community pharmacies and retail clinics. J Am Pharm Assoc (2003). 2015;55(5):488–92.
- 92. Lessard D, Lebouche B, Engler K, Thomas R, Machouf N. Explaining the appeal for immigrant men who have sex with men of a community-based rapid HIV-testing site in Montreal (Actuel sur Rue). AIDS Care. 2015;27(9):1098–103. pmid:25849524
- 93. MacGowan RJ, Lifshay J, Mizuno Y, Johnson WD, McCormick L, Zack B. Positive Transitions (POST): Evaluation of an HIV Prevention Intervention for HIV-Positive Persons Releasing from Correctional Facilities. AIDS Behav. 2015;19(6):1061–9. pmid:25190222
- 94. Maturo D, Powell A, Major-Wilson H, Sanchez K, De Santis JP, Friedman LB. Transitioning Adolescents and Young Adults With HIV Infection to Adult Care: Pilot Testing the "Movin' Out" Transitioning Protocol. J Pediatr Nurs. 2015;30(5):e29–35. pmid:26276460
- 95. Maulsby C, Positive Charge Intervention T, Charles V, Kinsky S, Riordan M, Jain K, et al. Positive Charge: Filling the Gaps in the U.S. HIV Continuum of Care. AIDS Behav. 2015;19(11):2097–107. pmid:25673009
- 96. Metsch LR, Pereyra M, Messinger S, Jeanty Y, Parish C, Valverde E, et al. Effects of a Brief Case Management Intervention Linking People With HIV to Oral Health Care: Project SMILE. Am J Public Health. 2015;105(1):77–84. pmid:24832421
- 97. Moitra E, Chan PA, Stein MD. Open Trial of an Acceptance-Based Behavior Therapy Intervention to Engage Newly Diagnosed HIV Patients in Care: Rationale and Evidence of Feasibility and Acceptability. Behavior modification. 2015;39(5):670–90. pmid:26111724
- 98. Pence BW, Gaynes BN, Adams JL, Thielman NM, Heine AD, Mugavero MJ, et al. The effect of antidepressant treatment on HIV and depression outcomes: results from a randomized trial. AIDS. 2015;29(15):1975–86. pmid:26134881
- 99. Poirier C, Aymeric S, Grammatico-Guillon L, Lebeau JP, Bernard L, Le Bret P, et al. Rapid HIV test in family practice. Med Mal Infect. 2015;45(6):207–14. pmid:25982343
- 100. Prekker ME, Gary BM, Patel R, Olives T, Driver B, Dunlop SJ, et al. A comparison of routine, opt-out HIV screening with the expected yield from physician-directed HIV testing in the ED. Am J Emerg Med. 2015;33(4):506–11. pmid:25727169
- 101. Schackman BR, Leff JA, Barter DM, DiLorenzo MA, Feaster DJ, Metsch LR, et al. Cost-effectiveness of rapid hepatitis C virus (HCV) testing and simultaneous rapid HCV and HIV testing in substance abuse treatment programs. Addiction. 2015;110(1):129–43. pmid:25291977
- 102. Schafer JJ, Naples JG, Pizzi LT, DeSimone JA. The effects of a pharmacist-delivered patient education programme on retention in human immunodeficiency virus care: a pilot study. Journal of Pharmaceutical Health Services Research. 2015;6(2):83–9.
- 103. Shrestha RK, Gardner L, Marks G, Craw J, Malitz F, Giordano TP, et al. Estimating the cost of increasing retention in care for HIV-infected patients: results of the CDC/HRSA retention in care trial. Journal of acquired immune deficiency syndromes (1999). 2015;68(3):345–50.
- 104. Swendeman D, Ramanathan N, Baetscher L, Medich M, Scheffler A, Comulada WS, et al. Smartphone self-monitoring to support self-management among people living with HIV: perceived benefits and theory of change from a mixed-methods randomized pilot study. Journal of acquired immune deficiency syndromes (1999). 2015;69 Suppl 1:S80–91.
- 105. Terzian AS, Irvine MK, Hollod LM, Lim S, Rojas J, Shepard CW. Effect of HIV Housing Services on Engagement in Care and Treatment, New York City, 2011. AIDS Behav. 2015;19(11):2087–96. pmid:25631320
- 106. Bamford L, Ellenberg JH, Hines J, Vinnard C, Jasani A, Gross R. Factors associated with a willingness to accept rapid HIV testing in an urban emergency department. AIDS Behav. 2014;18(2):250–3. pmid:23536139
- 107. Drainoni M-L, Farrell C, Sorensen-Alawad A, Palmisano JN, Chaisson C, Walley AY. Patient Perspectives of an Integrated Program of Medical Care and Substance Use Treatment. AIDS Patient Care and STDs. 2014;28(2):71–81. pmid:24428768
- 108. Eggman AA, Feaster DJ, Leff JA, Golden MR, Castellon PC, Gooden L, et al. The cost of implementing rapid HIV testing in sexually transmitted disease clinics in the United States. Sex Transm Dis. 2014;41(9):545–50. pmid:25118967
- 109. Esposito-Smythers C, Brown LK, Wolff J, Xu J, Thornton S, Tidey J, et al. Substance abuse treatment for HIV infected young people: an open pilot trial. J Subst Abuse Treat. 2014;46(2):244–50. pmid:23988190
- 110. Gardner LI, Giordano TP, Marks G, Wilson TE, Craw JA, Drainoni ML, et al. Enhanced personal contact with HIV patients improves retention in primary care: a randomized trial in 6 US HIV clinics. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2014;59(5):725–34.
- 111. Hooshyar D, Suris AM, Czarnogorski M, Lepage JP, Bedimo R, North CS. Rapid HIV testing experience at Veterans Affairs North Texas Health Care System's Homeless Stand Downs. AIDS Care. 2014;26(1):95–9. pmid:23750751
- 112. Kasting MLC, Anthony D; Cox Dena; Fife Kenneth H; Katz Barry P; Zimet Gregory D. The effects of HIV testing advocacy messages on test acceptance: a randomized clinical trial. BMC medicine. 2014;12:204. pmid:25374047
- 113. Knapp H, Hagedorn H, Anaya HD. A five-year self-sustainability analysis of nurse-administered HIV rapid testing in Veterans Affairs primary care. Int J STD AIDS. 2014;25(12):837–43. pmid:24480848
- 114. Knapp H, Hagedorn H, Anaya HD. HIV rapid testing in a Veterans Affairs hospital ED setting: a 5-year sustainability evaluation. Am J Emerg Med. 2014;32(8):878–83. pmid:24908442
- 115. Konkle-Parker DJ, Amico KR, McKinney VE. Effects of an intervention addressing information, motivation, and behavioral skills on HIV care adherence in a southern clinic cohort. AIDS Care. 2014;26(6):674–83. pmid:24116922
- 116. Magidson JF, Seitz-Brown CJ, Safren SA, Daughters SB. Implementing Behavioral Activation and Life-Steps for Depression and HIV Medication Adherence in a Community Health Center. Cogn Behav Pract. 2014;21(4):386–403. pmid:25419102
- 117. Marlin RY, Sean; Bristow Claire C.; Jose Ortiz; Rhea Mathew; Keith Daniel; Greg Wilson; Jeffrey Rodriguez; Jeffrey Klausner. Feasibility of HIV self-test vouchers to raise community-level serostatus awareness, Los Angeles. Sexually Transmitted Diseases. 2014;41:S96.
- 118. Merchant RC, Baird JR, Liu T, Taylor LE, Montague BT, Nirenberg TD. Brief intervention to increase emergency department uptake of combined rapid human immunodeficiency virus and hepatitis C screening among a drug misusing population. Acad Emerg Med. 2014;21(7):752–67. pmid:25125271
- 119. O’Connor G, Ni Flaitheartaigh A, Lacey A, O’Halloran J, Brazil E, Calderon Y, et al. A randomized, controlled study exploring factors associated with decision to undergo HIV screening. BMC Infectious Diseases. 2014;14(Suppl 2).
- 120. Rosales-Statkus ME, Belza-Egozcue MJ, Fernandez-Balbuena S, Hoyos J, Ruiz-Garcia M, de la Fuente L. Who and how many of the potential users would be willing to pay the current or a lower price of the HIV self-test? The opinion of participants in a feasibility study of HIV self-testing in Spain. Enferm Infecc Microbiol Clin. 2014;32(5):302–5. pmid:24468627
- 121. Schechter-Perkins EM, Koppelman E, Mitchell PM, Morgan JR, Kutzen R, Drainoni ML. Characteristics of patients who accept and decline ED rapid HIV testing. Am J Emerg Med. 2014;32(9):1109–12. pmid:24993683
- 122. Smillie K, Van Borek N, Abaki J, Pick N, Maan EJ, Friesen K, et al. A qualitative study investigating the use of a mobile phone short message service designed to improve HIV adherence and retention in care in Canada (WelTel BC1). J Assoc Nurses AIDS Care. 2014;25(6):614–25. pmid:24768442
- 123. Vachirasudlekha B, Cha A, Berkowitz L, Shah B. Interdisciplinary HIV care—patient perceptions. Int J Health Care Qual Assur. 2014;27(5):405–13. pmid:25087338
- 124. Yang M, Prestage G, Maycock B, Brown G, de Wit J, McKechnie M, et al. The acceptability of different HIV testing approaches: cross-sectional study among GMSM in Australia. Sex Transm Infect. 2014;90(8):592–5. pmid:25015651
- 125. Young SD, Daniels J, Chiu CJ, Bolan RK, Flynn RP, Kwok J, et al. Acceptability of using electronic vending machines to deliver oral rapid HIV self-testing kits: a qualitative study. PLoS One. 2014;9(7):e103790. pmid:25076208
- 126. Young SD, Klausner J, Fynn R, Bolan R. Electronic vending machines for dispensing rapid HIV self-testing kits: a case study. AIDS Care. 2014;26(2):267–9. pmid:23777528
- 127. d'Almeida KW, Pateron D, Kierzek G, Renaud B, Semaille C, de Truchis P, et al. Understanding providers' offering and patients' acceptance of HIV screening in emergency departments: a multilevel analysis. ANRS 95008, Paris, France. PLoS One. 2013;8(4):e62686. pmid:23638133
- 128. Baltzer Turje RC, Miranda; Julie Kille; Jennifer Shergold; Allison Macbeth; Martin Payne; Dianne Simpson; Grace Dalgarno; Patrick McDougall; Maxine Davis. Improving health outcomes for people living with HIV/AIDS who face multiple barriers to care including addiction, mental health issues, homelessness and other social or environmental factors: Results from a Seek and treat to optimally prevent (STOP) HIV/AIDS collaborative project with Vancouver coastal health, providence health care and the Dr. Peter centre's day health program. Canadian Journal of Infectious Diseases and Medical Microbiology. 2013;24:93A.
- 129. Bilardi JE, Walker S, Read T, Prestage G, Chen MY, Guy R, et al. Gay and bisexual men's views on rapid self-testing for HIV. AIDS Behav. 2013;17(6):2093–9. pmid:23297083
- 130. Burns FE, S G; Woods J; Haidari G; Calderon Y; Leider J; Morris S; Tobin R; Cartledge J; Brown M. Acceptability, feasibility and costs of universal offer of rapid point of care testing for HIV in an acute admissions unit: results of the RAPID project. HIV medicine. 2013;14(Suppl 3):10–4.
- 131. Ewing M, Read P, Knight V, Morgan S, Hanlon M, McDonald A, et al. Do callers to the NSW Sexual Health Infoline attend the services they are referred to? Sex Health. 2013;10(6):530–2. pmid:24119390
- 132. Gaydos CA, Solis M, Hsieh Y-H, Jett-Goheen M, Nour S, Rothman RE. Use of tablet-based kiosks in the emergency department to guide patient HIV self-testing with a point-of-care oral fluid test. International journal of STD & AIDS. 2013;24(9):716–21.
- 133. Hallum-Montes R, Morgan S, Rovito HM, Wrisby C, Anastario MP. Linking peers, patients, and providers: a qualitative study of a peer integration program for hard-to-reach patients living with HIV/AIDS. AIDS Care. 2013;25(8):968–72. pmid:23311339
- 134. Hennessey Z, Osorio G, Pati R, Sharp V, Giurgiulescu A, Wiener D, et al. A quality framework to improve routine rapid HIV screening, diagnosis, and linkage to care at a high-volume, urban emergency department in New York City. The Lancet. 2013;382.
- 135. King KC S.; Nalabanda A.; Evans R. Routine opt-out HIV screening in adult mental health services: Results from pilot intervention to improve screening. International Journal of STD and AIDS. 2013;24:13.
- 136. Knapp H, Anaya HD. Implementation science in the real world: a case study of HIV rapid testing. Int J STD AIDS. 2013;24(1):5–11. pmid:23427213
- 137. Kurth AE, Severynen A, Spielberg F. Addressing unmet need for HIV testing in emergency care settings: a role for computer-facilitated rapid HIV testing? AIDS Educ Prev. 2013;25(4):287–301. pmid:23837807
- 138. Leber W, McMullen H, Marlin N, Bremner S, Santos AC, Terris-Prestholt F, et al. Point-of-care HIV testing in primary care and early detection of HIV (RHIVA2): a cluster randomised controlled trial. The Lancet. 2013;382.
- 139. Lewis NM, Gahagan JC, Stein C. Preferences for rapid point-of-care HIV testing in Nova Scotia, Canada. Sex Health. 2013;10(2):124–32. pmid:23369262
- 140. Read TR, Hocking JS, Bradshaw CS, Morrow A, Grulich AE, Fairley CK, et al. Provision of rapid HIV tests within a health service and frequency of HIV testing among men who have sex with men: randomised controlled trial. BMJ. 2013;347:f5086. pmid:24004988
- 141. Seewald R, Bruce RD, Elam R, Tio R, Lorenz S, Friedmann P, et al. Effectiveness and feasibility study of routine HIV rapid testing in an urban methadone maintenance treatment program. Am J Drug Alcohol Abuse. 2013;39(4):247–51. pmid:23841865
- 142. Shahkolahi MM, Wilson A, Zucker L. Mobile-Application-Based HIV Intervention: An Opt-In Approach to Promote Rapid HIV Screening in an Inner-City Emergency Department. Annals of Emergency Medicine. 2013;62(4).
- 143. Stenstrom RG E.; Poureslami I.; Osati F.; Scheuermeyer F.X.; Harris D.; Hunte G.; O'Donnell S.; Rich K.; Nemethy K. Feasibility of and patient satisfaction withf HIV point-ofcare testing in the emergency department. Canadian Journal of Emergency Medicine. 2013;15:S30.
- 144. Williams MSK.; Cavanaugh T. Project impact: Improving patient adherence through communication at transition. Journal of the American Pharmacists Association. 2013;53(2):e15–e6.
- 145. Young SD, Cumberland WG, Lee SJ, Jaganath D, Szekeres G, Coates T. Social networking technologies as an emerging tool for HIV prevention: a cluster randomized trial. Ann Intern Med. 2013;159(5):318–24. pmid:24026317
- 146. Nilsen P. Making sense of implementation theories, models and frameworks. Implementation Science. 2015;10(1):53.
- 147. Kemp CG, Weiner BJ, Sherr KH, Kupfer LE, Cherutich PK, Wilson D, et al. Implementation science for integration of HIV and non-communicable disease services in sub-Saharan Africa: a systematic review. Aids. 2018;32 Suppl 1:S93–s105.