Skip to main content
Download PDF

Predicting resistance as indicator for need to switch from first-line antiretroviral therapy among patients with elevated viral loads: development of a risk score algorithm

BMC Infectious Diseases volume 16, Article number: 280 (2016) Cite this article

Abstract

Background

In resource-limited settings, where resistance testing is unavailable, confirmatory testing for patients with high viral loads (VL) delays antiretroviral therapy (ART) switches for persons with resistance. We developed a risk score algorithm to predict need for ART change by identifying resistance among persons with persistently elevated VL.

Methods

We analyzed data from a Phase IV open-label trial. Using logistic regression, we identified demographic and clinical characteristics predictive of need for ART change among participants with VLs ≥1000 copies/ml, and assigned model-derived scores to predictors. We designed three models, including only variables accessible in resource-limited settings.

Results

Among 290 participants with at least one VL ≥1000 copies/ml, 51 % (148/290) resuppressed and did not have resistance testing; among those who did not resuppress and had resistance testing, 47 % (67/142) did not have resistance and 53 % (75/142) had resistance (ART change needed for 25.9 % (75/290)). Need for ART change was directly associated with higher baseline VL and higher VL at time of elevated measure, and inversely associated with treatment duration. Other predictors included body mass index and adherence. Area under receiver operating characteristic curves ranged from 0.794 to 0.817. At a risk score ≥9, sensitivity was 14.7–28.0 % and specificity was 96.7–98.6 %.

Conclusions

Our model performed reasonably well and may be a tool to quickly transition persons in need of ART change to more effective regimens when resistance testing is unavailable. Use of this algorithm may result in public health benefits and health system savings through reduced transmissions of resistant virus and costs on laboratory investigations.

Peer Review reports

Background

The World Health Organization (WHO) recommends viral load (VL) as the preferred method for monitoring antiretroviral therapy (ART) and diagnosing viral failure in HIV-infected patients [1]. An elevated VL is an important gauge of treatment effectiveness, indicating poor adherence and/or drug resistance [2–5]. Failing to switch persons with drug resistance to second-line therapy in a timely manner increases morbidity and mortality, likelihood of second-line treatment failure, and transmission of resistant virus [2, 6–15]. Drug resistance testing is rarely available in resource-limited settings, where the majority of persons accessing ART reside [16]. Distinguishing persons with modifiable poor adherence without resistance mutations from persons with drug resistance (for whom improved adherence will not result in viral resuppression) is critical to reduce the spread of resistance and improve effectiveness of second-line therapies.

Current VL monitoring algorithms require confirmatory testing for elevated initial tests (Fig. 1) [1]. This two-step process presents an opportunity for counseling that may improve adherence leading to virological resuppression [17]. However, for persons with resistant viruses, requiring a second test unnecessarily postpones the treatment switch. The delays introduced with confirmatory testing are especially relevant in resource-limited settings: programmatic and patient-related obstacles may substantially increase the interval between first and confirmatory testing. Among persons with confirmed virological failure in South Africa, switch to second-line therapy took >5 months after confirmatory VL [18, 19].

Fig. 1
figure 1

World Health Organization viral load testing strategy for treatment failure [1]. Persons eligible for viral load testing may be tested using plasma-based assays or dried blood spots. For plasma assays, a viral load >1000 copies/ml prompts an evaluation of adherence to antiretroviral therapy and targeted adherence counseling if deficiencies in adherence are observed. The viral load test is repeated 3 to 6 months later (confirmatory test). Patient management is dictated by results of this second test – patients with confirmed elevated (>1000 copies/ml) viral loads are switched to second-line therapy. The dashed arrow represents implementation of the risk score algorithm. Persons with a risk score exceeding the predefined algorithm threshold would be switched immediately to second-line therapy

Full size image

In sub-Saharan Africa, >25 % of ART persons may not achieve viral suppression by 12 months [20], and rates of virological failure may be as high as 14 % at five years [19]. With nearly ten million persons receiving ART in low- and middle-income countries [21], eliminating confirmatory testing for even a fraction of ART-resistant persons will produce substantial cost savings. Furthermore, early identification of treatment failure may avoid significant morbidity and mortality for patients who otherwise remain on inappropriate therapy.

Distinguishing persons with elevated viremia with without drug resistance mutations is challenging in resource-limited settings where resistance testing is unavailable. A simple risk score algorithm may help providers identify patients with probable ART resistance who could be switched to second-line therapy immediately without confirmatory testing. Using patient demographics, clinical, and laboratory-based predictors that would be readily available in most clinical settings, we developed a risk score algorithm to predict the need for ART change by identifying resistance among patients with persistently elevated VL. Patients exceeding a pre-specified risk score threshold could be switched immediately; patients below this threshold would have confirmatory VL testing prior to treatment switch decisions.

Methods

Study setting and population

Eligible participants in the Prospective Evaluation of Antiretrovirals in Resource-Limited Settings (PEARLS) trial (Adult AIDS Clinical Trials Group (ACTG) A5175, NCT00084136) were studied. PEARLS was an open-label, Phase IV, randomized clinical trial that investigated efficacy and safety of once- vs. twice-daily regimen dosing. Details of the PEARLS study population and design have been described elsewhere [22]. In brief, A5175 enrolled 1571 HIV-infected participants ≥18 years old from nine countries, over-sampling participants from resource-limited settings. Participants were excluded from PEARLS if they: had a CD4 cell count >300 cells/mm3, previous exposure to ART (exception for women who received ART for prevention of mother-to-child transmission), were pregnant, or were acutely ill and/or clinically unstable. PEARLS was approved by institutional review boards and ethics committees at participating institutions.

This study is a post-hoc analysis of a subset of de-identified data restricted to participants initiated on non-nucleoside reverse transcriptase inhibitor (NNRTI)-based regimens (lamivudine/zidovudine/efavirenz or emtricitabine/tenofovir/efavirenz) who had at least one VL ≥1000 copies/ml at any point after week 16 of enrollment. This population was distinct from the A5175 definition of failure which required two consecutive measurements of plasma HIV-1 RNA ≥1000copies/ml after 16-weeks or disease progression. Primary analyses included participants from all study sites; a sensitivity analysis restricted the population to participants enrolled from resource-limited settings. This analysis was approved by the University of North Carolina, School of Medicine Institutional Review Board.

Data collection

Per A5175 study protocol, participants received a targeted physical exam, adherence interview, serum chemistries, CD4 lymphocyte count, and plasma HIV RNA (Roche Amplicor Monitor assay [v1.5]) at least every eight weeks. Any treatment modification (participant, provider, or protocol-mandated) was assessed at each visit. Diagnosis criteria were collected using a standardized case report form.

Resistance tests were done retrospectively at four regional laboratories participating in the National Institute of Allergy and Infectious Diseases Division of AIDS Virology Quality Assurance program, coordinated by the HIV Prevention Trials Network Laboratory Center, using ViroSeq HIV-1 Genotyping Assay (Celera Diagnostics, Alameda, California) on stored specimens [23]. Resistance testing was done for participants meeting study-specific virological failure criteria (defined below) or who had disease progression (new or recurrent AIDS-defining opportunistic infection or malignancy) ≥12 weeks after randomization.

Measures

The outcome (need for ART change after first VL ≥1000 copies/ml) was assessed using the following algorithm: resuppression (<1000 copies/ml) – no ART change needed; no resuppression and no resistance – no ART change needed; and no resuppression and resistance – ART change needed. Participants with NRTI or NNRTI resistance mutations, defined by 2014 International AIDS Society guidelines (excluding mutation 101P), were classified as resistant [24]. Resistance to protease inhibitors (PI) were not included; this class of drugs is reserved for second-line therapy and thus, if observed in the absence of NRTI/NNRTI resistance, would not be an indication for treatment change from first-line regimen. We did not distinguish between baseline and acquired resistance. Resistance testing was not done on participants who had a VL ≥1000 copies/ml and resuppressed at their subsequent study visit. Any participant who resuppressed was classified as not needing ART change. Participants who had two sequential study visits with VL ≥1000 copies/ml, but who did not have a resistance test, were excluded.

Potential predictors of needing ART change included demographics, clinical diagnoses prior to treatment initiation, immunological markers (CD4 cell count), self-reported and provider-assessed ART adherence, and therapy duration (based on the number of days between ART initiation and a participant’s first VL ≥1000 copies/ml). Per WHO and other country ART guidelines, the six-month visit is frequently identified as the first point that a participant is eligible for VL monitoring [1, 25, 26]. A six-month visit was defined as any time point ≥16-week visit and ≤ 212 days after ART initiation; this time frame includes an acceptable 30-day extension of the six-month window period. The 12-month visit was similarly classified as any time after the six-month window up to and including 30 days after 12 months on ART (395 days).

Statistical analyses

All analyses were conducted using Stata statistical software (Version 13.0; Stata Corporation, College Station, TX).

We constructed three multivariable models to predict need for ART change that reflected variations in availability of CD4 and VLs at time of ART initiation. Although many countries have scaled up access to CD4 testing to determine ART eligibility, the roll-out of Option B+, in which HIV-infected pregnant women are initiated on lifelong ART regardless of CD4, could mean that many persons will not have a CD4 cell count at treatment initiation [1]. In light of these policies and the capacity constraints in resource-limited settings, models were constructed to reflect three scenarios:

  • Model 1 assumed that VL and CD4 at ART initiation were available, so both were included as eligible predictors.

  • Model 2 assumed that treatment initiation CD4 was available but that treatment initiation VL was not and thus excluded as an eligible predictor.

  • Model 3 assumed that neither treatment initiation VL nor CD4 were available; thus neither was included as eligible predictors.

To evaluate the association between predictors and need for ART change, we calculated unadjusted prevalence odds ratios (OR) and 95 % confidence intervals (CI) for each potential predictor in each model [27].

The full models contained all variables with bivariate p-values <0.5; this high threshold was chosen to ensure that available important predictors were considered [28]. Variables with low frequency, extreme collinearity, or insufficient detail to permit clinical implementation were excluded, regardless of p-value. We tested four categorizations of time on treatment and selected the category with the lowest Akaike’s information criteria (AIC) value for our reference models [29].

We developed the predictive models using multiple logistic regression with backward elimination [27]. Beginning with the variable with the largest p-value, we removed variables one at a time until ≤5 remained (regardless of p-value). The five-variable limit was selected to facilitate eventual implementation of risk scores in resource-limited clinical settings [30, 31]. We assessed the equality of the area under the receiver operating characteristic curves (AUROC) between each model (chi-squared test) [32]. AUROC measures the risk score’s discriminatory power –where 1.0 indicates a perfect test (i.e., 100 % sensitivity and 100 % specificity) [33]. Likelihood ratio (LR) comparing successive models were evaluated to confirm that variable removal did not adversely affect the model’s predictive capacity. We also compared LR-test statistics from each reduced model to the full model.

We used the three predictive models to develop the associated risk scores by assigning each variable in the final models a predictor score equal to two times the beta coefficient rounded to the nearest integer. We doubled the coefficient to retain inherent discrimination between betas, while keeping the absolute numbers manageable. Patients with a high VL (≥1000 copies/ml) and a risk score equal to or greater than a pre-specified cutoff are classified as likely needing to switch to second-line ART without a confirmatory VL test. For each model, we assessed sensitivity, specificity, and associated risk scores at cutoffs selected based on clinically-acceptable model-performance criteria [34, 35]. Given the undesirable consequences of prematurely switching persons to second-line therapy, we maintained a high specificity threshold (>95.0 %) for all models to minimize false positives. We also calculated the number of patients in a hypothetical cohort of 10,000 ART patients who would be switched without confirmatory testing at each cutoff. We internally validated the model and risk score performance using 1000 bootstrap samples with replacement [27, 36]. Bootstrapping is a process in which each iteration resamples a random cohort of data points from within our full dataset and assess validity of our calculated estimates using this “new” sample. Bootstrapping is preferred over data splitting and cross validation for internal validation [37–41]. Model calibration was assessed using Hosmer-Lemeshow (HL) goodness-of-fit tests (null hypothesis of statistically significant difference between observed and predicted estimates), and residuals were evaluated to verify appropriate model fit [42].

Sensitivity analyses

We conducted a sensitivity analysis to evaluate model performance using only study participants from resource-limited settings. Given the implementation and policy implications and hypothesized biological association of ART duration and drug resistance, we tested multiple forms of the treatment time variable (Additional file 1: Table S1-S3). Models 4–6 evaluate therapy duration categorized as <7, 7–24, and >24 months; models 7–9 dichotomized duration (<7 vs ≥7 months). We compared these alternatives to the primary models using AIC.

Results

Study population

Among 1045 participants, 305 had at least one VL ≥1000 copies/ml after week 16; 15 participants were excluded despite having two sequential VL ≥1000 copies/ml because resistance results were unavailable at the time of confirmed elevated VL, for a final sample of 290. Age ranged from 19 to 65 years, and 53 % of persons were male (Table 1). Mean CD4 at enrollment was 156 cells/mm3 and median VL at enrollment was 115,383 copies/ml.

Table 1 Bivariable association of need for ART change and potential predictor characteristics
Full size table

Bivariable analyses

Among the 290 with at least one VL ≥1000 copies/ml, 53 % (148/290) resuppressed at the next visit. Among the remaining 142 who did not resuppress, 75 had resistance mutations (either transmitted or acquired) to NRTI or NNRTI drugs. Thus, overall, NRTI or NNRTI resistance was detected in 25.9 % (95 % CI 20.8 %, 30.9 %) of participants with at least one VL ≥1000 copies/ml. Participants with a higher VL at ART initiation (>100,000 copies/ml) (OR = 2.5, 95 % CI 1.4, 4.3) were more likely to need ART change than participants with a lower VL at ART initiation (Table 1). At time of VL elevation, VL >100,000 copies/ml (OR = 3.3, 95 % CI 1.6, 6.9) or 10,000–100,000 copies/ml (OR = 5.7, 95 % CI 3.0, 10.7) also were associated with increased likelihood of needing ART change, compared to participants with VL <10,000 copies/ml. Participants who were on therapy <7 months (OR = 5.1, 95 % CI 2.6, 9.8), or 7–12 months (OR = 3.2, 95 % CI 1.5, 6.8) were more likely to need ART change than participants on therapy >12 months. Participants whose BMI > 25.0 kg/m2 at ART initiation were more likely to need ART change at time of first VL ≥1000 copies/ml than participants with BMI ≤25.0 kg/m2 (OR = 2.5, 95 % CI 1.4, 4.5).

Multivariable analyses

Model 1 - Including treatment initiation VL and CD4

The full model included ten predictor variables (AUROC = 0.842) and showed acceptable HL test, failing to reject the null hypothesis (p = 0.70). Our final model contained five predictor variables: age <30, BMI > 25.0, treatment initiation VL ≤100,000 copies/ml, time on treatment, and VL at time of first VL ≥1000 (Table 2). The AUROC was 0.8165 for the reduced model, which showed acceptable calibration, (HL p = 0.12)

Table 2 Adjusted odds ratios and risk scores of need for ART change
Full size table

Model 2 - Excluding treatment initiation VL

The full model included nine predictor variables (AUROC = 0.819) and showed acceptable calibration (HL failed to reject null, p = 0.84). The reduced model contained six predictor variables (AUROC = 0.807). To meet the predefined criterion of a five-variable model, we eliminated the variable with the lowest OR (self-reported adherence). Our final model contained: age <30, screening CD4 < 100 cells/mm3, BMI >25.0, time on treatment, and VL at time of first VL ≥1000 (AUROC = 0.7981) (Table 2). The reduced model showed acceptable calibration (HL p = 0.84).

Model 3 - Excluding treatment initiation VL and CD4

The full model included eight predictor variables (AUROC = 0.801) and showed acceptable calibration (HL p = 0.37). The final model contained: age <30, self-reported missed medications, BMI > 25.0, time on treatment, and VL at time of first VL ≥1000 (AUROC = 0.7937) (Table 2). The reduced model showed acceptable calibration (HL, p = 0.10).

Reduced Model 1 performed slightly better than reduced Model 2, but the difference was not significant (p = 0.23). Reduced Model 3 performed slightly worse again, but compared to reduced Model 1, the difference was not statistically significant (p = 0.22) (Fig. 2). Bootstrapping demonstrated consistent performance for all models over 1000 replications. Further model diagnostics demonstrated normally distributed residuals; graphs of predicted probabilities against residuals suggested random distribution.

Fig. 2
figure 2

Receiver operating characteristic (ROC) curves for Models 1–3. The area under an ROC curve is a measure of model performance. Specifically, the area measures discrimination – in this case the ability of the predictive model to correctly classify persons with and without resistance. Model 1, in which we assumed that viral load and CD4 cell counts from time of treatment initiation were available, performed the best, and had an area under the ROC curve of 0.8165. In Model 2, when viral load from treatment initiation was excluded as an eligible predictor, performed slightly less well (area under ROC curve of 0.7981). Finally, in Model 3, we assumed that neither viral load nor CD4 cell counts from time of treatment initiation were available. This model performed the poorest of all three models evaluated, with an area under the ROC curve of 0.794 – although this difference was not statistically significant and may not be clinically meaningful

Full size image

Risk scores

The weighted risk scores ranged from 0–12 for Models 1 and 2, and 0–11 for Model 3 (Table 2). The maximum attained score by any individual in the tested population was 11 for each model. The predictive power of the model was retained when predicted probabilities were transformed to risk scores (AUROC for Model 1 = 0.813 (p = 0.69), Model 2 = 0.797 (p = 0.91), and Model 3 = 0.802 (p = 0.57)). A risk score cutoff of ≥9 met predefined specificity threshold (>95.0 %) (Table 3).

Table 3 Performance of models and derived risk scores
Full size table

We estimated the number of patients who would be immediately switched to second-line therapy in a hypothetical population of 10,000 ART patients receiving VL monitoring. Given the proportion of patients who did not resuppress and who harbored drug resistance (~25 % of entire study population), Model 1 risk score would accurately identify 700 persons who needed ART change (true positives) and incorrectly classify 248 persons as needing ART change when they did not (false positives) (Fig. 3). At this same drug resistance prevalence, Model 2 would correctly switch 400 persons in need of ART change and have 105 false positives. Model 3 would correctly switch 368 persons in need of ART change, with 143 false positives. However, as the underlying drug resistance prevalence increases, so too does the number of true positives as well as the ratio of true positive:false positive. For example, with a prevalence of 55 % in a population of 10,000 ART patients with a VL ≥1000 copies/ml, Model 1 would correctly identify 1540 patients as needing ART change with only 149 false positives.

Fig. 3
figure 3

Number of false positive and true positive results in hypothetical cohort of 10,000 ART patients with elevated viral load at varied resistance prevalence estimates. Using the sensitivities and specificities for each risk score at the defined threshold, we generated the number of false positives and true positives that would be expected among a 10,000-person cohort of patients with an initially elevated viral load. We evaluated these outcomes at varying levels of ART resistance. As the prevalence of resistance increases, the positive predictive value of the risk scores also improves

Full size image

Sensitivity analyses

Model performance was comparable when the study population was restricted to persons from resource-limited settings: AUROC =0.823 (Model 1), 0.812 (Model 2), and 0.804 (Model 3). Using the same risk score cutoff as in the unrestricted model (≥9), the sensitivity for the three models ranged from 10.0–26.0 %; specificity ranged from 97.4–99.5 % (Table 3).

Discussion

Current WHO guidelines recommend confirmatory testing for ART patients with high VL (≥1000 copies/ml). A subset of patients will be resistant at the time of initial elevated VL; for these persons, requiring confirmatory testing unnecessarily delays switch to second-line therapy. We developed a risk score using only parameters that are likely to be available to providers in resource-limited settings that successfully identifies person with persistently elevated VL or are resistant and need immediate ART change. The risk score performed well, >98 % specific in most model iterations. Although increased specificity comes at the cost of the lower sensitivity (~15–30 %), this is less concerning as these “misses” will undergo confirmatory testing as is the standard-of-care for virological failure [1]. Rapidly switching patients with resistance to more efficacious second-line therapy could reduce transmission of resistant viral strains and transmission overall, and could prevent further declines in CD4 cell counts, resulting in meaningful public health benefits.

Utilizing this risk score may also reduce costs by avoiding unnecessary confirmatory VL tests. Alternative cost-saving strategies for virological monitoring include pooling specimens and targeting VL tests based on clinical or immunological criteria [1, 43–47]. Despite potential cost-savings, pooling requires additional laboratory support for linkage and deconstruction of positive pools. Applying a conservative estimate of treatment failure (16.0 % at 12 months) would translate to >1,000,000 ART patients having an elevated VL in sub-Saharan Africa alone [20]. Even a modest reduction in confirmatory test volume resulting from implementation of our algorithm could substantially reduce expenditures and patients’ clinical deterioration.

Our risk score balances predictive ability and practicality. Notably, given our goal of point-of-care application, we considered only predictors that were likely available in ART clinics within resource-limited settings. We also sacrificed some precision for ease-of-use by collapsing continuous variables into discrete categories and limiting the number of included variables.

Maximizing specificity was essential to decrease false positives. We selected 95.0 % as the lower threshold for specificity, though selected risk score thresholds had higher specificities (96.7–98.6 %). Even at specificities >98 %, prematurely switching a patient to second-line therapy (false positive) still occurs and has significant person- and system-wide consequences. For patients, false positive misclassification results in lost potential life years from remaining on first-line therapy. These are patients who, with improved adherence, may resuppress. For the healthcare system, premature second-line switching results in increased drug costs– as much as 6–10 times the cost of first-line therapy [26, 48]. Conversely, missing patients who are resistant is also associated with substantial health consequences and healthcare system costs, including accumulation and potential transmission of resistant viral strains. The current study used the WHO-accepted threshold of <1000 copies/ml for defining resuppression, and assumed that persons who resuppressed were not harboring clinically significant resistance mutations. However, resistance may still be present at low-level viremia (<1000 copies/ml) [49–53], and may be associated with subsequent virological failure [54], suggesting that policies for treatment change thresholds, and thus the proposed algorithm, may misclassify the need for ART switch in the presence of resistance at lower viral loads. Modeling the consequences of delayed second-line initiation versus premature switching may help elucidate the trade-offs inherent to these thresholds. Importantly, trade-offs may vary by population: for example, providers may be more willing to “risk” false positive results in HIV-infected pregnant women given that viral suppression at time of delivery prevents vertical transmission. Acceptable true positive:false positive ratios may also differ depending on anticipated time-to-referral, as the patient and public health benefits of immediate switching may be greater in settings with extensive delays in second-line initiation [18, 19].

These data came from a controlled clinical trial, and enrolled participants may not be representative of larger ART populations. Viral suppression was similar to other cohorts with nearly 30 % of participants having a VL ≥1000 copies/ml after ≥16 weeks on ART [20]. Participants received frequent virological monitoring (every 8 weeks) in the study, which is unlikely in the intended settings for this risk score. The risk score used >12 months as the referent category, however, sensitivity analyses with alternative categorization of therapy duration did not change model performance (Additional file 1). Furthermore, all patients included in this analysis were initiated on efavirenz-based first-line regimens – resistance patterns and predictors may not apply to non-efavirenz-based regimens depending on different barriers to resistance mutations. Importantly, participants were recruited largely from resource-limited settings and the risk score performed well in this subgroup. Furthermore, PEARL’s broad inclusion criteria improves generalizability. Study-driven CD4 cell count eligibility were consistent with WHO guidelines (<300 cells/mm3), but these guidelines have since changed, expanding ART eligibility to HIV-infected persons earlier in the course of disease (<500 cells/mm3) [1]. If CD4 is included (Model 2), having more patients with high CD4 at treatment initiation could mean that fewer patients reach the switch score threshold, potentially dampening the efficiency gains of the algorithm.

Among patients with ≥1 elevated VL, resistance rates were lower than observed in sub-Saharan African cohorts (as high as 70 %) [2, 11, 55–57], however, this may result from more frequent VL monitoring in the study, which increases the likelihood of detecting transiently elevated VL. A higher prevalence of resistance would favor use of the risk score, increasing the score’s positive predictive value. Assuming 55 % resistance among patients with an elevated VL, we demonstrated that in a hypothetical cohort of 10,000 ART patients, >1500 would be appropriately classified as resistant and switched immediately, with only 150 false positives.

Conclusions

To our knowledge, this risk score is the first to identify the need for immediate ART change among persons with a single elevated VL. We successfully identified predictors that reliably distinguished between persons who do and do not need immediate ART change from first-line regimens. Our risk score is sensitive to realities in resource-limited settings: we used a limited number of readily-available categorical variables and minimized false positive results. This model is a promising opportunity to quickly transition patients with probable resistance to more effective regimens – improving ART morbidity and mortality outcomes. Using this risk score may reduce transmission of resistant viral strains and save healthcare systems scarce resources by reducing personnel and equipment costs incurred with unnecessary confirmatory VL testing. These potential benefits should be assessed and externally validated prospectively by evaluating the effect of the risk score on health outcomes and resource utilization, taking into account the trade-offs associated with misclassifying even a small subset of patients as needing ART change when they do not [31].

Abbreviations

ACTG, AIDS Clinical Trials Group; AIC, Akaike’s information criteria; ART, antiretroviral therapy; AUROC, area under the receiver operating characteristic; BMI, body mass index; CI, confidence intervals; HL, Hosmer-Lemeshow; LR, likelihood ratio; NNRTI, non-nucleoside reverse transcriptase inhibitor; NRTI, nucleoside reverse transcriptase inhibitor; OR, odds ratios; PEARLS, Prospective Evaluation of Antiretrovirals in Resource-Limited Settings; PI, protease inhibitors; VL, viral load; WHO, World Health Organization

References

  1. WHO. Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. Geneva: WHO; 2013.

    Google Scholar 

  2. Hosseinipour MC, van Oosterhout JJ, Weigel R, Phiri S, Kamwendo D, Parkin N, Fiscus SA, Nelson JA, Eron JJ, Kumwenda J. The public health approach to identify antiretroviral therapy failure: high-level nucleoside reverse transcriptase inhibitor resistance among Malawians failing first-line antiretroviral therapy. AIDS. 2009;23(9):1127–34.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Tang MW, Shafer RW. HIV-1 antiretroviral resistance: scientific principles and clinical applications. Drugs. 2012;72(9):e1–25.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Mee P, Fielding KL, Charalambous S, Churchyard GJ, Grant AD. Evaluation of the WHO criteria for antiretroviral treatment failure among adults in South Africa. AIDS (London, England). 2008;22(15):1971–7.

    Article  Google Scholar 

  5. Ndembi N, Goodall RL, Dunn DT, McCormick A, Burke A, Lyagoba F, Munderi P, Katundu P, Kityo C, Robertson V, et al. Viral rebound and emergence of drug resistance in the absence of viral load testing: a randomized comparison between zidovudine-lamivudine plus Nevirapine and zidovudine-lamivudine plus Abacavir. J Infect Dis. 2010;201(1):106–13.

    Article  CAS  PubMed  Google Scholar 

  6. Kumarasamy N, Madhavan V, Venkatesh KK, Saravanan S, Kantor R, Balakrishnan P, Devaleenal B, Poongulali S, Yepthomi T, Solomon S, et al. High frequency of clinically significant mutations after first-line generic highly active antiretroviral therapy failure: implications for second-line options in resource-limited settings. Clin Infect Dis. 2009;49(2):306–9.

    Article  CAS  PubMed  Google Scholar 

  7. de la Rosa R, Ruiz-Mateos E, Rubio A, Abad MA, Vallejo A, Rivero L, Genebat M, Sanchez-Quijano A, Lissen E, Leal M. Long-term virological outcome and resistance mutations at virological rebound in HIV-infected adults on protease inhibitor-sparing highly active antiretroviral therapy. J Antimicrob Chemother. 2004;53(1):95–101.

    Article  PubMed  Google Scholar 

  8. Gupta RK, Hill A, Sawyer AW, Cozzi-Lepri A, von Wyl V, Yerly S, Lima VD, Gunthard HF, Gilks C, Pillay D. Virological monitoring and resistance to first-line highly active antiretroviral therapy in adults infected with HIV-1 treated under WHO guidelines: a systematic review and meta-analysis. Lancet Infect Dis. 2009;9(7):409–17.

    Article  CAS  PubMed  Google Scholar 

  9. Hosseinipour MC, Kumwenda JJ, Weigel R, Brown LB, Mzinganjira D, Mhango B, Eron JJ, Phiri S, van Oosterhout JJ. Second-line treatment in the Malawi antiretroviral programme: high early mortality, but good outcomes in survivors, despite extensive drug resistance at baseline. HIV Med. 2010;11(8):510–8.

    CAS  PubMed  PubMed Central  Google Scholar 

  10. Karlsson AC, Younger SR, Martin JN, Grossman Z, Sinclair E, Hunt PW, Hagos E, Nixon DF, Deeks SG. Immunologic and virologic evolution during periods of intermittent and persistent low-level viremia. AIDS. 2004;18(7):981–9.

    Article  PubMed  Google Scholar 

  11. Marconi VC, Sunpath H, Lu Z, Gordon M, Koranteng-Apeagyei K, Hampton J, Carpenter S, Giddy J, Ross D, Holst H, et al. Prevalence of HIV-1 drug resistance after failure of a first highly active antiretroviral therapy regimen in KwaZulu Natal, South Africa. Clin Infect Dis. 2008;46(10):1589–97.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Pujades-Rodriguez M, O’Brien D, Humblet P, Calmy A. Second-line antiretroviral therapy in resource-limited settings: the experience of Medecins Sans Frontieres. AIDS. 2008;22(11):1305–12.

    Article  CAS  PubMed  Google Scholar 

  13. Reynolds SJ, Kityo C, Mbamanya F, Dewar R, Ssali F, Quinn TC, Mugyenyi P, Dybul M. Evolution of drug resistance after virological failure of a first-line highly active antiretroviral therapy regimen in Uganda. Antivir Ther. 2009;14(2):293–7.

    CAS  PubMed  PubMed Central  Google Scholar 

  14. Keiser O, Chi BH, Gsponer T, Boulle A, Orrell C, Phiri S, Maxwell N, Maskew M, Prozesky H, Fox MP, et al. Outcomes of antiretroviral treatment in programmes with and without routine viral load monitoring in Southern Africa. AIDS. 2011;25(14):1761–9.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Keiser O, Tweya H, Braitstein P, Dabis F, MacPhail P, Boulle A, Nash D, Wood R, Luthi R, Brinkhof MW, et al. Mortality after failure of antiretroviral therapy in sub-Saharan Africa. Trop Med Int Health. 2010;15(2):251–8.

    Article  PubMed  Google Scholar 

  16. UNAIDS. Access to Antireretroviral Therapy in Africa: Status report on progress towards the 2015 targets. Geneva: UNAIDS; 2013.

    Google Scholar 

  17. Wilson D, Keiluhu AK, Kogrum S, Reid T, Seriratana N, Ford N, Kyawkyaw M, Talangsri P, Taochalee N. HIV-1 viral load monitoring: an opportunity to reinforce treatment adherence in a resource-limited setting in Thailand. Trans R Soc Trop Med Hyg. 2009;103(6):601–6.

    Article  PubMed  Google Scholar 

  18. Fox MP, Cutsem GV, Giddy J, Maskew M, Keiser O, Prozesky H, Wood R, Hernan MA, Sterne JA, Egger M, et al. Rates and predictors of failure of first-line antiretroviral therapy and switch to second-line ART in South Africa. J Acquir Immune Defic Syndr. 2012;60(4):428–37.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Boulle A, Van Cutsem G, Hilderbrand K, Cragg C, Abrahams M, Mathee S, Ford N, Knight L, Osler M, Myers J, et al. Seven-year experience of a primary care antiretroviral treatment programme in Khayelitsha, South Africa. AIDS. 2010;24(4):563–72.

    Article  PubMed  Google Scholar 

  20. McMahon JH, Elliott JH, Bertagnolio S, Kubiak R, Jordan MR. Viral suppression after 12 months of antiretroviral therapy in low- and middle-income countries: a systematic review. Bull World Health Organ. 2013;91(5):377–385E.

    Article  PubMed  PubMed Central  Google Scholar 

  21. HIV/AIDS [http://www.who.int/mediacentre/factsheets/fs360/en/]. Accessed 21 Oct 2015.

  22. Campbell TB, Smeaton LM, Kumarasamy N, Flanigan T, Klingman KL, Firnhaber C, Grinsztejn B, Hosseinipour MC, Kumwenda J, Lalloo U, et al. Efficacy and safety of three antiretroviral regimens for initial treatment of HIV-1: a randomized clinical trial in diverse multinational settings. PLoS Med. 2012;9(8):e1001290.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Kantor R, Smeaton L, Vardhanabhuti S, Hudelson SE, Wallis CL, Tripathy S, Morgado MG, Saravanan S, Balakrishnan P, Reitsma M, et al. Pretreatment HIV Drug Resistance and HIV-1 Subtype C Are Independently Associated With Virologic Failure: Results From the Multinational PEARLS (ACTG A5175) Clinical Trial. Clin Infect Dis. 2015;60(10):1541–9.

    PubMed  PubMed Central  Google Scholar 

  24. Hammer SM, Eron Jr JJ, Reiss P, Schooley RT, Thompson MA, Walmsley S, Cahn P, Fischl MA, Gatell JM, Hirsch MS. Antiretroviral treatment of adult HIV infection: 2008 recommendations of the International AIDS Society-USA panel. JAMA. 2008;300(5):555–70.

    Article  CAS  PubMed  Google Scholar 

  25. Clinical Guidelines for the Management of HIV&AIDS in Adults and Adolescents. Pretoria: National Department of Health; 2010. http://www.who.int/hiv/pub/guidelines/south_africa_art.pdf. Accessed 7 June 2016.

  26. Clinical Management of HIV in Children and Adults. First edition. Lilongwe: Ministry of Health, Malawi; 2011. http://apps.who.int/medicinedocs/documents/s18802en/s18802en.pdf. Accessed 7 June 2016.

  27. Harrell F. Regression modeling strategies: with applications to linear models, logistic regression, and survival analysis. New York: Springer; 2001.

    Book  Google Scholar 

  28. Sun GW, Shook TL, Kay GL. Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. J Clin Epidemiol. 1996;49(8):907–16.

    Article  CAS  PubMed  Google Scholar 

  29. Pan W. Akaike’s information criterion in generalized estimating equations. Biometrics. 2001;57(1):120–5.

    Article  CAS  PubMed  Google Scholar 

  30. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA. 1997;277(6):488–94.

    Article  CAS  PubMed  Google Scholar 

  31. Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085–94.

    Article  CAS  PubMed  Google Scholar 

  32. DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.

    Article  CAS  PubMed  Google Scholar 

  33. Deeks JJ. Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. BMJ. 2001;323(7305):157–62.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. McClamroch KJ, Kaufman JS, Behets FM. A formal decision analysis identifies an optimal treatment strategy in a resource-poor setting. J Clin Epidemiol. 2008;61(8):776–87.

    Article  PubMed  Google Scholar 

  35. Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med. 2006;144(3):201–9.

    Article  PubMed  Google Scholar 

  36. Harrell Jr FE, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.

    Article  PubMed  Google Scholar 

  37. Altman DG, Royston P. What do we mean by validating a prognostic model? Stat Med. 2000;19(4):453–73.

    Article  CAS  PubMed  Google Scholar 

  38. Steyerberg EW, Harrell Jr FE, Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol. 2001;54(8):774–81.

    Article  CAS  PubMed  Google Scholar 

  39. Molinaro AM, Simon R, Pfeiffer RM. Prediction error estimation: a comparison of resampling methods. Bioinformatics. 2005;21(15):3301–7.

    Article  CAS  PubMed  Google Scholar 

  40. Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation. J Am Stat Assoc. 1983;78(382):316–30.

    Article  Google Scholar 

  41. Kohavi R. A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In: International Joint Conference on Artificial Intelligence. 1995.

    Google Scholar 

  42. Hosmer DW, Lemeshow S. Applied Logistic Regression. 2nd edition. Hoboken, New Jersey: John Wiley & Sons, Inc; 2000.

  43. Davies MA, Boulle A, Technau K, Eley B, Moultrie H, Rabie H, Garone D, Giddy J, Wood R, Egger M, et al. The role of targeted viral load testing in diagnosing virological failure in children on antiretroviral therapy with immunological failure. Trop Med Int Health. 2012;17(11):1386–90.

    Article  PubMed  Google Scholar 

  44. van Zyl GU, Preiser W, Potschka S, Lundershausen AT, Haubrich R, Smith D. Pooling strategies to reduce the cost of HIV-1 RNA load monitoring in a resource-limited setting. Clin Infect Dis. 2011;52(2):264–70.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Tilghman MW, Guerena DD, Licea A, Perez-Santiago J, Richman DD, May S, Smith DM. Pooled nucleic acid testing to detect antiretroviral treatment failure in Mexico. J Acquir Immune Defic Syndr. 2011;56(3):e70–4.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Smith DM, May SJ, Perez-Santiago J, Strain MC, Ignacio CC, Haubrich RH, Richman DD, Benson CA, Little SJ. The use of pooled viral load testing to identify antiretroviral treatment failure. AIDS. 2009;23(16):2151–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Pannus P, Fajardo E, Metcalf C, Coulborn RM, Duran LT, Bygrave H, Ellman T, Garone D, Murowa M, Mwenda R. Pooled, et al. HIV-1 viral load testing using dried blood spots to reduce the cost of monitoring antiretroviral treatment in a resource-limited setting. J Acquir Immune Defic Syndr. 2013;64(2):134–7.

  48. Harries AD, Zachariah R, van Oosterhout JJ, Reid SD, Hosseinipour MC, Arendt V, Chirwa Z, Jahn A, Schouten EJ, Kamoto K. Diagnosis and management of antiretroviral-therapy failure in resource-limited settings in sub-Saharan Africa: challenges and perspectives. Lancet Infect Dis. 2010;10(1):60–5.

    Article  PubMed  Google Scholar 

  49. Mackie N, Dustan S, McClure MO, Weber JN, Clarke JR. Detection of HIV-1 antiretroviral resistance from patients with persistently low but detectable viraemia. J Virol Methods. 2004;119(2):73–8.

    Article  CAS  PubMed  Google Scholar 

  50. Waters L, Mandalia S, Asboe D. Successful use of genotypic resistance testing in HIV-1-infected individuals with detectable viraemia between 50 and 1000 copies/ml. AIDS. 2006;20(5):778–9.

    Article  PubMed  Google Scholar 

  51. Cane PA, Kaye S, Smit E, Tilston P, Kirk S, Shepherd J, Hopkins M, Zhang H, Geretti AM. Genotypic antiretroviral drug resistance testing at low viral loads in the UK. HIV Med. 2008;9(8):673–6.

    Article  CAS  PubMed  Google Scholar 

  52. Gonzalez-Serna A, Min JE, Woods C, Chan D, Lima VD, Montaner JS, Harrigan PR, Swenson LC. Performance of HIV-1 drug resistance testing at low-level viremia and its ability to predict future virologic outcomes and viral evolution in treatment-naive individuals. Clin Infect Dis. 2014;58(8):1165–73.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Santoro MM, Fabeni L, Armenia D, Alteri C, Di Pinto D, Forbici F, Bertoli A, Di Carlo D, Gori C, Carta S. Reliability and clinical relevance of the HIV-1 drug resistance test in patients with low viremia levels. Clin Infect Dis. 2014;58(8):1156–64.

    Article  CAS  PubMed  Google Scholar 

  54. Swenson LC, Min JE, Woods CK, Cai E, Li JZ, Montaner JS, Harrigan PR, Gonzalez-Serna A. HIV drug resistance detected during low-level viraemia is associated with subsequent virologic failure. AIDS. 2014;28(8):1125–34.

    Article  PubMed  PubMed Central  Google Scholar 

  55. Hamers RL, Sigaloff KC, Wensing AM, Wallis CL, Kityo C, Siwale M, Mandaliya K, Ive P, Botes ME, Wellington M, et al. Patterns of HIV-1 drug resistance after first-line antiretroviral therapy (ART) failure in 6 sub-Saharan African countries: implications for second-line ART strategies. Clin Infect Dis. 2012;54(11):1660–9.

    Article  CAS  PubMed  Google Scholar 

  56. Hoffmann CJ, Charalambous S, Sim J, Ledwaba J, Schwikkard G, Chaisson RE, Fielding KL, Churchyard GJ, Morris L, Grant AD. Viremia, resuppression, and time to resistance in human immunodeficiency virus (HIV) subtype C during first-line antiretroviral therapy in South Africa. Clin Infect Dis. 2009;49(12):1928–35.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Aghokeng AF, Monleau M, Eymard-Duvernay S, Dagnra A, Kania D, Ngo-Giang-Huong N, Toni TD, Toure-Kane C, Truong LX, Delaporte E, et al. Extraordinary heterogeneity of virological outcomes in patients receiving highly antiretroviral therapy and monitored with the World Health Organization public health approach in sub-saharan Africa and southeast Asia. Clin Infect Dis. 2014;58(1):99–109.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank the PEARLS study participants who volunteered their time and efforts. We would also like to thank Thomas B. Campbell, M.D., Colorado School of Medicine, Aurora, Colorado, Susan Eshleman, M.D., Johns Hopkins University, Baltimore, Maryland, Rami R. Kantor, M.D., Brown University, Providence, Rhode Island, and Angela Bengtson, PhD, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, for their review and contributions to this manuscript.

Funding

Research reported in this publication was supported by the National Institute of Allergy and Infectious Diseases of the National Institutes of Health under Award Number UM1 AI068634, UM1 AI068636 and UM1 AI106701. The pharmaceutical sponsors (Boehringer Ingelheim Pharmaceuticals Inc., Bristol-Myers Squibb, Gilead Sciences, and Viiv Healthcare/GlaxoSmithKline) provided study drug and Gilead Sciences provided funding to purchase study drug that was not otherwise available. SER was supported by F30 MH098731 and T32 GM008719. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Availability of data and materials

The clinical data are available from the authors with written permission from the ACTG (AIDS Clinical Trials Group).

Authors’ contributions

All authors meet criteria for authorship as recommended by the International Committee of Medical Journal Editors (ICMJE). SER, MCH, MW, SBW, AKB, JJE, and WCM contributed to the conception, design, and analysis of this study. CW, PB, JWM, MM, SS, ST, SV contributed to collection and interpretation of data. SER, MCH, MW, and WCM drafted the initial manuscript. All authors provided substantive edits to the manuscript and approved the final draft.

Authors’ information

Not applicable.

Competing interests

The author(s) declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

The parent study was approved by institutional review boards and ethics committees at participating institutions. All participants provided written, informed consent. Human experimentation guidelines of the US Department of Health and Human Services were followed. This analysis was approved by the University of North Carolina, School of Medicine Institutional Review Board.

Author information

Authors and Affiliations

  1. Department of Health Policy and Management, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Sarah E. Rutstein, Morris Weinberger, Stephanie B. Wheeler & Andrea K. Biddle

  2. Division of Infectious Diseases, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Sarah E. Rutstein, Mina C. Hosseinipour, Joseph J. Eron & William C. Miller

  3. UNC Project, Lilongwe, Malawi

    Mina C. Hosseinipour

  4. BARC-SA and Lancet Laboratories, Johannesburg, South Africa

    Carole L. Wallis

  5. YRG Centre for AIDS Research and Education (YRG CARE), Voluntary Health Services, Taramani, Chennai, 600113, India

    Pachamuthu Balakrishnan

  6. Division of Infectious Diseases, University of Pittsburgh, Pittsburgh, PA, USA

    John W. Mellors

  7. Department of STD, AIDS, and Viral Hepatitis, Brazilian National STD and AIDS Program, Rio de Janeiro, Brazil

    Mariza Morgado

  8. YRG Centre for AIDS Research and Education, Chennai, India

    Shanmugam Saravanan

  9. Institute for Leprosy and Other Mycobacterial Diseases, Tajganj, Agra, India

    Srikanth Tripathy

  10. Harvard T.H. Chan School of Public Health, Boston, MA, USA

    Saran Vardhanabhuti

  11. Department of Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    William C. Miller

Authors
  1. Sarah E. Rutstein
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mina C. Hosseinipour
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morris Weinberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Stephanie B. Wheeler
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Andrea K. Biddle
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Carole L. Wallis
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Pachamuthu Balakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. John W. Mellors
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mariza Morgado
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Shanmugam Saravanan
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Srikanth Tripathy
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Saran Vardhanabhuti
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Joseph J. Eron
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. William C. Miller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sarah E. Rutstein.

Additional file

Additional file 1:

Supplementary Tables, S1–S3. (DOC 185 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rutstein, S.E., Hosseinipour, M.C., Weinberger, M. et al. Predicting resistance as indicator for need to switch from first-line antiretroviral therapy among patients with elevated viral loads: development of a risk score algorithm. BMC Infect Dis 16, 280 (2016). https://0-doi-org.brum.beds.ac.uk/10.1186/s12879-016-1611-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s12879-016-1611-2

Keywords

BMC Infectious Diseases

ISSN: 1471-2334

Contact us