Figures
Abstract
Background
CD4 T-cell counts are widely used to assess treatment eligibility and to follow-up HIV-infected patients. The World Health Organization prequalification of in vitro diagnostics program conducted a performance evaluation of the FACSPresto (BD Biosciences), a new point-of-care instrument to measure absolute CD4-T cell (CD4) counts and percentages in venous and capillary blood samples from HIV-infected patients.
Methods
Patients were recruited in Belgium (200 patients) and in Tanzania (247 patients). Venous blood samples were analyzed in two nearby reference laboratories. In addition, nurses/technicians collected a capillary blood sample by finger prick directly into a FACSPresto CD4 cartridge. Assay precision was assessed on fresh blood and on external quality control samples. Trueness (bias) was assessed by comparing results from FACSPresto with the reference (single-platform FACSCalibur). Clinical misclassification was measured at 200, 350 and 500 cells/μL thresholds.
Results
Intra-assay precision was < 6%, and inter-assay < 8%. CD4 results from FACSPresto and reference method resulted in regression slopes of 0.99–1.11 using either venous or capillary blood. Correlation was better for venous than for capillary blood (minimum 0.97 vs 0.93 respectively). Capillary blood resulted in a larger bias than venous blood, with 24 and 83 cells/μL for absolute CD4 counts on capillary blood in Antwerp and Dar es Salaam respectively, vs 12 and 41 cells/μL on venous blood. Bias on CD4% was < 1% on both venous and capillary blood, and was proportionally better than for absolute CD4 counts. Clinical misclassification was in line with the average overestimation, showing a very good specificity, but sensitivity around 70–90%. The rejection rate was 11% on first reading, leading to 6% of all samples without final result after a second reading.
Citation: Daneau G, Aboud S, Prat I, Urassa W, Kestens L (2017) Performance of FACSPresto Point-of-Care Instrument for CD4-T Cell Enumeration in Human Immunodeficiency Virus (HIV)-Infected Patients Attending Care and Treatment Clinics in Belgium and Tanzania. PLoS ONE 12(1): e0170248. https://doi.org/10.1371/journal.pone.0170248
Editor: Carlo Federico Perno, Universita degli Studi di Roma Tor Vergata, ITALY
Received: September 5, 2016; Accepted: December 30, 2016; Published: January 27, 2017
Copyright: © 2017 Daneau et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This study was partially funded by the World Health Organization Prequalification of Diagnostics Programme http://www.who.int/diagnostics_laboratory/evaluations/cd4/en/). The funder designed the generic protocol for the evaluation. There was no additional external funding received for this study.
Competing interests: The authors have declared that no competing interests exist.
Introduction
In 2014, there were about 36.9 million people living with HIV [1]. Of these, 32.5 million were eligible for antiretroviral treatment (ART) but, in low- and middle-income countries, only 36% received treatment [2]. ART initiation was for long based on absolute CD4 T-cell counts (CD4 count) apart from clinical examination. Currently, the World Health Organization (WHO) recommends starting treatment for all HIV-infected patients irrespective of CD4 count levels, with priority for patients with severe or advanced HIV clinical disease (WHO clinical stage 3 or 4) and adults with less than 350 cells/μL [3]. The previous guidelines mentioned a threshold of 500 cells/μL, which could be seen as an intermediary step towards treatment for all in settings where the threshold of 350 cells/μL to initiate ART is still applied [4]. Even when the importance of CD4 counts to monitor ART is decreasing, it is still recommended to monitor CD4 counts at regular time intervals until stable viral load suppression is achieved [5;6]. In addition, as long as viral load testing is difficult due to technical and financial constraints, it is expected that low-income regions will continue using CD4 counts as an alternative [7]. CD4 counts are also used to decide on prophylaxis against opportunistic infections, for instance to screen for cryptococcal infection in patients with ≤ 100 CD4 cells/μL and stop prophylaxis when patient presents with > 200 cells/μL, or start co-trimoxazole in case CD4 counts are ≤ 350 cells/μL [3]. The threshold of 200 cells/μL is also important to decide on vaccination benefits and risks [3].
Clinical management of HIV-infected patients is therefore still dependant on CD4 counting capacity. The CD4 reference method, developed on large flow cytometers, is usually implemented by skilled staff in well-equipped laboratories [8;9]. In the last decade, several instruments dedicated to CD4 counting were developed. They are easier to operate and cheaper than the traditional flow cytometers used in clinical laboratories [10]. However, this new generation of instruments still require a laboratory infrastructure and trained staff. Simple CD4 technologies at the point of care (POC) could fill a last gap by bringing CD4 counting to small health centres or as instruments operated in mobile teams [11]. Having a CD4 technology nearby the patient with a short turn-around time for results would help to reduce the number of patients lost to follow up [12]. Currently, there is no instrument-free rapid test for CD4 enumeration. All POC CD4 technologies still require an instrument; in this case, a reader.
Before being widely distributed, manufacturer-independent evaluations of new technologies are valuable to confirm the appropriate use of such technologies in different settings. In this perspective, the WHO prequalification of in vitro diagnostics program includes the evaluation of CD4 counting instruments, in particular those meant for use in resource-limited settings. The FACSCount CD4 and the Pima CD4 were successfully evaluated by WHO in 2012 [13]. In 2014, WHO evaluated a new CD4 POC instrument, the FACSPresto. This instrument presents similar features to the Pima CD4 but, in addition to CD4 counts, also measures CD4 percentages (CD4%) and haemoglobin concentration. The 25μl-blood drop from the patient is inserted in a ready-to-use disposable cartridge, and incubated on the bench during 18 minutes prior to analysis. The CD4 counting process itself, in the FACSPresto instrument, is fully automated with multiple internal control checks, and results are available within 3–4 minutes.
This study analyses the analytical performance of this recent POC instrument in two different settings, comparing CD4 counting results in Europe and in Africa. In accordance with the WHO protocol, analytical precision and trueness of FACSPresto were measured in two independent reference labs, in Belgium and in Tanzania. This is one of the first publications on the performance of FACSPresto, and the first manufacturer-independent reporting of paired comparison of venous and capillary blood samples in a reference setting [14–17].
Materials and Methods
Study populations
Adult patients presenting for routine CD4 enumeration at the HIV outpatient clinic at the Institute of Tropical Medicine (ITM) Antwerp, Belgium (in April-June 2014), and at the Infectious Diseases Clinic (IDC) in Dar es Salaam, Tanzania (in June 2014), were invited to donate a capillary blood sample (by a single finger prick) in addition to routine venipuncture in K3-EDTA vacutainer tubes. After a first phase of recruiting all consecutive patients, a second phase focused on patients with low CD4 counts, based on measurement at their previous visit, in order to reach target numbers per CD4 category.
The study was approved by the Institutional Review Board from ITM and the Ethical committee at University Hospital Antwerp, Belgium, and by the Senate Research and Publications Committee of Muhimbili University of Health and Allied Sciences in Dar es Salaam, Tanzania. All patients signed an informed consent form prior to the inclusion in the study.
Analytical precision.
Intra-laboratory variation testing included instrument precision, intra-assay variation, inter-instrument variation, and inter-assay variation, using venous blood specimens. Capillary blood specimens were only used to assess instrument precision. Both sites had multiple FACSPresto instruments to facilitate work management (four FACSPresto instruments in Antwerp and three in Dar es Salaam); the variation between those instruments was measured (inter-instrument precision).
The mean, standard deviation (SD) and coefficient of variation (% CV) of the obtained results were calculated and compared between FACSPresto instruments (Chi-square).
In Antwerp, blood samples were stratified in categories around clinically relevant CD4 thresholds: 200 (150–250), 350 (300–400) and 500 (450–550) CD4 cells/μL. The CD4 category was determined by the first CD4 result of venous blood sample on the FACSPresto, except for instrument precision studies which was based on capillary blood results. Precision analysis in Dar es Salaam mainly focused on low CD4 samples, i.e. values between 100 and 300 CD4 cells/μL.
Instrument precision (run-to-run) consisted of re-reading a single CD4 cartridge ten times on the same FACSPresto instrument. The corresponding venous blood was processed in the same way. Only when the venous blood sample was included in the same CD4 category as capillary blood, precision results were included in the study.
For intra-assay precision (tube-to-tube variability), each blood sample was prepared to fill ten cartridges, and each cartridge was read once, with all ten cartridges on the same FACSPresto instrument.
For inter-instrument precision (instrument-to-instrument variability), each sample was prepared to fill ten cartridges, and each cartridge was read four times, once on each of the four FACSPresto instruments in Antwerp, by the same operator. In Dar es Salaam, each sample was prepared to fill ten cartridges, by each of the three operators, and each cartridge was read on each of the three FACSPresto instruments. The average of the obtained results was calculated for each instrument, and for each operator. The inter-instrument variability (%CV inter-instruments) was calculated using the average results of the different instruments for each patient sample. The reproducibility (in Tanzania) was calculated between the averages of the different operators and instruments for each sample, and compared with a one-way ANOVA.
For inter-assay precision (day-to-day variability), a venous blood sample was analysed within 2h after blood collection. An aliquot of the venous blood sample was kept at room temperature and prepared for FACSPresto at two additional time points to include 6 and 24 hours after blood collection.
Global mean %CV for each category was calculated as a weighted average of the individual sample %CV’s.
Quality control.
The CD4 reference method implemented in Antwerp is ISO15189 certified, and in Dar es Salaam by the National AIDS Control Program in the Ministry of Health, Community Development, Gender, Elderly and Children in Tanzania. Both laboratories participate in external quality assessment programs (QASI and UK-NEQAS). Prior to the start of the study, the CD4 reference instruments (FACSCalibur) used in both settings were technically inspected by a service engineer from BD Biosciences.
During the study, the CD4 reference instruments were calibrated daily, and checked by running Multicheck Normal and Low controls (BD Biosciences). Three different lots of Normal controls and two lots Low controls were used during the three-month duration of the study in Antwerp. In Dar es Salaam, only one lot of Normal controls was used for the one-month study.
The FACSPresto contains internal control checks which are run each time the instrument is powered up. In addition, CD-Chex Plus BC Normal and Low (Streck, Omaha, NE) were run daily for two months on each FACSPresto instrument in Antwerp, and for the whole one-month study in Dar es Salaam (the same lot was used thorough the study).
For each blood control sample, the reported results were compared to the sample validation range provided by the manufacturer. The mean, SD and %CV of the obtained results were calculated and compared between FACSPresto instruments and/or operator.
Trueness.
Venous and capillary blood samples were collected by three nurses in Antwerp, and prepared (for venous blood samples) then read on FACSPresto by one technician on one of the four FACSPresto instruments. Blood samples in Dar es Salaam were collected and prepared by three technicians; each technician used a single FACSPresto instrument for reading.
CD4 counts and CD4% measured on FACSPresto using venous and capillary blood were compared to the corresponding CD4 venous blood results on FACSCalibur (BD Biosciences, San Jose, CA). CD4 counts were measured according to manufacturer's recommendations using Multiset reagents (CD3/CD4/CD8/CD45 monoclonal antibodies) and BD Trucount tubes (both from BD Biosciences, San Jose, CA). Analysis on FACSCalibur was done manually using CellQuest software using the accredited procedure used for routine CD4 counting in the laboratories.
Agreement between FACSPresto results and those obtained by the reference method, and between venous and capillary blood results obtained on FACSPresto, was assessed by the Passing and Bablok regression analysis [18], correlation with Spearman’s ρ coefficient, Bland-Altman [19] and Pollock analysis [20] using MedCalc version 10.0.2.0 (MedCalc Software, Mariakerke, Belgium). Outliers were identified and removed from the statistical analysis when the values for absolute and relative bias were both larger than four times the respective mean bias [21]. Correlation coefficient and bias were compared between sites and between parameters (blood types, CD4 vs CD4%, and influencing factors), applying paired or independent t-test, or one-way ANOVA as required.
Clinical misclassification was calculated for CD4 counts at clinically relevant CD4 thresholds of 200, 350, 500 cells/μL, using the reference method results as the “true” value for sensitivity and specificity. Inter-rater agreement was calculated with the Kappa coefficient [22].
Invalid results (Rejection rate)
Invalid results or sample rejection was defined as the inability of FACSPresto instrument to provide CD4 results from a patient blood sample. In case of an invalid test result on a capillary blood sample, it was not possible to redraw a second capillary blood specimen. As the study was based on four (Antwerp) or three (Dar es Salaam) different FACSPresto instruments in the participating labs, the cartridge was read on another FACSPresto instrument when rejected once or twice (repeated rejection). In case of a rejected first reading on venous blood, a second reading was performed on the same cartridge. If that the sample was rejected again, a new cartridge was prepared and read multiple times when required (after rejection). Venous and capillary blood samples from the same patient were always read on the same FACSPresto instrument.
Results
Study population
In Antwerp, 200 patients were recruited. Clinical data was available for 198 patients, of whom 197 were HIV infected. The median age (minimum—maximum) of the study participants was 46 years (19–83), and 71% were male. Most patients (96%) were on anti-retroviral therapy (ART). The median CD4 count was 420 cells/μL; 18 had CD4 < 200 cells/μL (median 158 cells/μL), 108 had CD4 counts between 200–500 cells/μL (351 cells/μL), and 73 had CD4 > 500 cells/μL (658 cells/μL). None of the patients presented malaria; three patients had suspected tuberculosis, but were finally sputum and/or culture negative.
In Dar es Salaam, 247 patients were recruited. The median age (minimum—maximum) was 41 years (15–75) and 32% were male. Most patients (93%) received ART. The median CD4 cell count was 266 cells/μL (n = 245); 82 had CD4 < 200 cells/μL (median 123 cells/μL), 114 had a CD4 count between 200–500 cells/μL (301 cells/μL), and 49 had CD4 > 500 cells/μL (636 cells/μL). Two patients had suspected malaria, one of these had suspected tuberculosis too.
Analytical precision
The mean instrument precision (%CV) for absolute CD4 counts and percentages in capillary and venous blood was <3%. The intra-assay variability showed an overall %CV of 5.9% for both absolute CD4 counts and CD4%. The %CV (intra-assay) for the paired samples was similar between the four FACSPresto instruments tested in Antwerp, and between the three instruments tested in Dar es Salaam (p > 0.05 for each site), and the inter-instrument variability was < 2%. The operator had no impact on the %CV (p > 0.05) (S1 Table). Finally, the inter-assay variability presented larger %CV, but still < 10% on average. Values for the different CD4 categories for each precision assay are presented in Table 1.
Quality control
All internal controls passed on the FACSPresto. Mean %CV in Antwerp was 0.14% for Normal and 0.16% for Low control. Mean %CV in Dar es Salaam was 0.07% for Normal and 0.11% for Low control. Values for the different FACSPresto instruments are presented in S2 Table.
Inter-assay variability based on commercially available stabilized blood samples CD-Chex Plus BC external controls in Antwerp showed globally %CVs < 10%; for CD4%: 2.1% for Normal and 6.2% for Low values, and for absolute CD4: 3.1% for Normal and 7.8% for Low values.
In Dar es Salaam, the global %CV was also <10%; for CD4%: 2.9% for Normal and 4.7% for Low values, and for absolute CD4 counts: 3.0% for Normal and 8.9% for Low.
All Multicheck controls analysed on FACSCalibur with Trucount tubes passed. In Antwerp, inter-assay variabilities (%CV) were 1.8% for Normal and 5.0% for Low for CD4 percentages. For absolute CD4 counts, the mean %CV was 6.7% for Normal and 4.9% for Low. Detailed results on different batches are presented in S2 Table. Values in Dar es Salaam for Normal were 16% for absolute CD4 counts and 4.0% for CD4%.
Trueness
Agreement between FACSPresto and the reference methods are shown in Table 2 (Venous blood on FACSPresto versus reference), Table 3 (Capillary blood on FACSPresto versus venous blood on reference), and Table 4 (Capillary blood versus venous blood on FACSPresto). Passing Bablok regression analyses showed slopes ranging from 0.99–1.11 for all comparisons, in both Antwerp and Dar es Salaam. The correlation coefficient was significantly higher when using venous blood compared to capillary blood (p < 0.01). The correlation coefficient was also different between sites, and was better (higher) on CD4% when comparing FACSPresto to FACSCalibur in Dar es Salaam (p < 0.05).
In Antwerp, the comparison of absolute CD4 counts in venous blood resulted in a mean absolute bias of +12 cells/μL between FACSpresto and FACSCalibur with limits of agreement up to +/- 87 cells/μL (1.96 x SD) around the mean bias. This comes down to a relative bias of 2.5%, with 1.96 x SD of 20% (Fig 1, upper left). Using capillary blood, the bias almost doubled (Fig 1, middle left). In Dar es Salaam the observed bias also doubled using capillary instead of venous blood (Fig 1, right). The absolute bias on CD4 percentages was also different between venous and capillary blood (each compared to the reference) (Table 3 and Fig 2). On the other hand, the absolute bias between venous blood and capillary blood (both on FACSPresto) was similar to the bias observed between capillary blood (on FACSPresto) and reference (venous blood on FACSCalibur) for absolute CD4 counts (Table 4 and Fig 1 lower). Similarly, performance on CD4 percentages on both venous and capillary blood comparison was different between sites (Tables 2, 3 and 4). In addition, the relative bias on CD4 counts in Dar es Salaam was significantly worse than on CD4% (p < 0.001; Tables 2, 3 and 4). In Antwerp, this difference was only observed for relative bias on FACSPresto (venous or capillary blood) vs reference (p < 0.001 and < 0.05 respectively; Tables 2 and 3).
Relative bias data (%) from Antwerp (left) and Dar es Salaam (right), with CD4 results (cells/μL) obtained by FACSPresto (Presto) using venous (upper) and capillary blood (midden), each compared to reference (Calibur), and to one another’ (lower). Mean bias is represented by the horizontal solid blue line, with the long-dashed brown line for limits of agreement (= 1.96SD), each mentioned with respective value.
Absolute bias data (%) from Antwerp (left) and Dar es Salaam (right), with value on FACSPresto (Presto) using venous (upper) and capillary blood (midden), each compared to reference (Calibur), and compared to each others’ (lower). Mean bias is represented by the horizontal solid blue line, with the long-dashed brown for limits of agreement (= 1.96SD), each mentioned with respective value.
We analyzed external factors that may influence the bias. Nurses collecting capillary blood in Antwerp had a significant impact on the observed bias (p = 0.028). On the contrary, operator in Dar es Salaam (technician collecting blood and measuring on FACSPresto device) had no influence (p > 0.05). Similarly, the different FACSPresto instruments produced a similar bias (p > 0.05).
Clinical misclassification
Misclassification analysis is shown in Table 5. Kappa coefficient was 0.642 to 0.893 for the different thresholds (200, 350 and 500 cells/μL). Specificity was in all cases higher than 90%. Sensitivity, however, was lower, in particular at the 200 and 350 thresholds. A misclassification means that the clinician will not start ART at the expected time. To have a broader idea of the time delay, bias between FACSPresto and FACSCalibur for misclassified samples is shown in Table 6. The larger the bias, the longer the expected time delay; samples with < 50 cells/μl may be expected to have no large impact on treatment timing despite the misclassification.
Invalid results (Rejection rate)
Invalid results on FACSPresto tend to occur more frequently with venous blood than with capillary blood specimens (11.0% versus 7.4% of agreement samples at the first reading; p = 0.08) (S3 Table). Re-reading a rejected capillary blood cartridge was not always sufficient to give a valid result, leading to 6% of patients not receiving a CD4 value. Using venous blood where new cartridges could be prepared, 5.8% of the blood samples did not give valid CD4 results. Most rejection codes were linked to an instrument failure (82% and 53% on capillary and venous blood respectively) (S4 Table). The frequency of invalid results was similar for all seven FACSPresto instruments (p > 0.05). In addition, three blood specimens failed to produce results due to very low values (< 20 cells/μl on FACSCalibur); they were considered as a valid reading on the FACSPresto.
Operators’ feed-back
Technicians found FACSPresto instrument easy to use after a 4-hour training.
Nurses in Antwerp found taking finger prick blood impractical and preferred taking blood by venipuncture. They found the success of taking capillary blood variable and too dependent on hand temperature. When hands are cold, despite manual warming, it was really hard to get a good blood drop, especially without squeezing the punctured finger. In contrast, warm hands tended to bleed too hard, requiring protective clothes, and spilling blood on the floor or on the cartridge itself, despite the frame around collector. Moreover, contact with HIV-infected blood was closer than with venous blood, requiring use of gloves for nurse safety.
Similarly, technicians in Dar es Salaam found the FASPresto easy to use. However, venous blood is routinely practiced and technicians preferred venous blood compared to capillary blood.
Discussion
The performance evaluation of the FACSPresto was conducted simultaneously in two complementary sites selected by WHO: one in Antwerp, Belgium, and the other in Dar es Salaam, Tanzania. Both study sites collected venous and capillary blood samples from each patient, which were analysed in the respective reference labs, with high quality assurance to ensure good working conditions.
The FACSPresto was found to perform well for the different parameters. Intra-assay precision was on average < 6%, which was as good as our reference instruments (data not shown). Inter-assay analysis, with repeat measurements at different time points, is used to assess repeatability of the test results. The instructions for use of FACSPresto recommend to use fresh blood samples or aged blood of 24h maximum, which does not allow day-to-day variability assays on stabilized blood covering more than 24h. We performed three repeated measurements on fresh blood samples, with two time points on the first day and an additional time point at 24h. In addition, we used stabilized blood samples (CD-Chex Plus BC quality controls) to measure day-to-day variability over a period of 1–3 months. The results confirm our observations on fresh blood samples. Furthermore, the precision on fresh blood and controls was found to be similar to that observed in the previous evaluations of the FACSPresto [14–17].
The trueness (bias) of venous blood results on FACSPresto was assessed by comparison with the FACSCalibur (reference). We found a slight overestimation of the absolute counts, similar to the one observed for other CD4-dedicated bench top instruments like the FACSCount [13]. Results from Dar es Salaam were also perfectly in line with FACSPresto results obtained in two other studies recently conducted in Africa [15;16]. Correlation and clinical misclassification of patients based on capillary blood results were also in agreement with the previous studies [14–16]. An additional element in our study was the analysis of factors influencing final result, with identification of capillary blood collection, and the nurse performing the finger prick. Lower performance on capillary blood samples as compared to venous blood illustrates that the type of blood specimen can have a substantial impact on results. In Dar es Salaam, CD4 percentages were quite similar between FACSPresto and FACSCalibur. However, there was less agreement between absolute CD4 counts, in capillary blood in particular. This raises the concern of the contribution of the observed bias as a consequence of possible lack of accuracy of the reference results. Absolute CD4 counting is based on a single platform measurement using Trucount tubes and precision of pipetting is very critical. The high CV on Multicheck Normal controls in Dar es Salaam is consistent with such variability from the reference; some outlier values within the batch could either be due to pipetting or heterogeneous sample. Unfortunately, no third instrument was included to confirm the patient results of the reference instrument. In addition, up to four outliers were excluded from the statistical analysis. As this approach is justified from a statistical point of view, exclusion of the FACSCalibur or FACSPresto results would probably not happen in the clinical practice. The patient would therefore be monitored according to the “outlier” result.
Concerning the statistical analyses included in our article, we applied a relative bias on both absolute CD4 counts and on CD4 percentages. Relative bias is usually not used on CD4%. However, this approach allowed us to compare bias estimates between different groups and between absolute CD4 counts and CD4 percentages, which showed a significant difference in our evaluation. Additionally, we applied the Pollock analysis after exclusion of samples with < 100 CD4 cells/μL, to avoid impact of a high relative bias at low CD4 counts (data not shown) on the overall average bias on all CD4 counts. This exclusion method did not reduce the LOA on relative bias by more than 1%.
The misclassification of patients for treatment initiation (prophylaxis or ART) was higher when capillary blood results were used compared to venous blood. The sensitivity was also affected by the threshold level, with better performance at higher thresholds. This may be partially explained by the sample distribution, as less samples had low CD4 counts, so the 24 missclassified samples represented a high proportion of the 82 samples with ≤ 200 cells/μL. We did not apply the threshold of 200 cells/μL on the CD4 results from Antwerp, as only 16 samples were under the target. For the same reason, the relevant threshold of 100 cells/μL could not be assessed. Patients presenting with low CD4 values are expected to become less frequent as a consequence of the Test and Treat strategy [3;7]. However, providing ART to all patients in resource-limited settings is not yet operational, and POC instruments are primarily designed for such settings. Moreover, future CD4 enumerations in the future may focus on prophylaxis, which applies to thresholds of 100 to 350 cells/μl. Similarly to the previous studies, the specificity was very high, due to an average positive bias from the FACSPresto, leading to treatment a short ART initiation delay, as the LOA range for the FACSPresto was less than 100 CD4 cells/μL. Such a low LOA has been shown to have a limited effect on the clinical decision [23]. Unfortunately, still 22 patients (37% of misclassified, 9.8% of total) were misclassified with > 100 cells/μL when capillary blood was used (threshold of 350 cells/μL in Dar es Salaam). Clinicians and decision-makers should be aware of this limitation when implementing this capillary blood approach in their setting, as patients may initiate their ART (where this threshold is still applied) or their prophylaxis with co-trimoxazole too late, increasing risks of developping opportunistic infections.
Sample rejection rate was lower than in South Africa (11% versus 16%), but most invalid results were linked to an instrument failure, and not to samples with low CD4 values. The proportion of samples without a final result (6%), higher than in the Kenyan study (1.9%), may be a concern although it was still considered acceptable for health care management (should be less than 10% according to the WHO evaluation protocol) [17]. However, in a setting with high numbers of patients with very low CD4 counts, the absence of a result rather than an indication of “low value” may be a limitation of the instrument; if precise quantitative data are indeed not possible under a certain CD4 threshold, a semi-quantitative result would have been more informative in this case.
The FACSPresto instrument has been designed for use in a non-laboratory environment close to the patient and patient care. Ease of use is similar to the Pima CD4, one other CD4-dedicated POC prequalified by WHO. When comparing the performance on both intruments, the FACSPresto showed better presicion, trueness and a lower rejection rate than Pima CD4 [13;24–27]. In both cases, capillary blood did not perform as good as venous blood. Interestingly, in contrast to FACSPresto, the Pima CD4 tended to underestimate the absolute CD4 count, leading to earlier treatment initiation rather than delay.
The evaluation conditions may also be considered as a limitation of the present study. This POC instrument was assessed in well-equipped reference labs, and an evaluation outside the laboratory by non-lab personnel has not been done. However, operators (technicians) found the FACSPresto very easy to use. In addition, it has been reported that patients accept finger prick better than venipuncture for CD4 counting at the point-of-care [28]. On the contrary, the staff responsible for capillary blood collection (nurses in Antwerp and technicians in Dar es Salaam) was not used to this approach, and preferred collecting venous over capillary blood. The lack of routine experience in finger pricking may also be a reason for the observed variability in performance. When evaluating or implementing capillary blood for CD4 enumeration, enough attention should be paid to this aspect, in addition to the specific training given by the manufacturer.
Another limitation of the FACSPresto may be the absence of integrated connectivity which would be desirable to facilitate electronic reporting of results and trouble shooting. The presence of an USB port may however allow the connection of an external GPRS, and the electronic export and transfer of results.
In conclusion, overall, FACSPresto showed a very good performance when using venous and capillary blood. The instrument was considered easy to use, with minimal training and infrastructure required, making it suitable for resource-limited settings. However, users should be aware that accuracy was lower when using capillary blood, emphasizing the need for dedicated training in settings where blood collection by finger prick would be applied.
Supporting Information
S1 File. Excel file with raw data from Belgium.
1. Clinical data; 2. Adverse events; 3. Intra-assay precision; 4. Inter-instrument precision; 5. Inter-assay precision; 6. Instrument precision; 7. Trueness FACSPresto vs FACSCalibur; 8. Internal controls FACSPresto; 9. External controls Multicheck on FACSCalibur; 10. External controls 10. CD-Chex Plus BC Streck on FACSPresto.
https://doi.org/10.1371/journal.pone.0170248.s001
(XLSX)
S2 File. Excel file with raw data from Tanzania.
1. Adverse events; 2. Intra-assay precision; 3. Inter-assay precision; 4. Instrument precision; 5. Intra-instrument precision (operator 1); 6. Intra-instrument precision (operator 2); 7. Intra-instrument precision (operator 3); 8. Trueness data on FACSPresto; 9. Trueness data on FACSPresto; 10. CD-Chex Plus BC Streck on FACSPresto.
https://doi.org/10.1371/journal.pone.0170248.s002
(XLSX)
S1 Table. Precision (%CV) determined by the different operators in Dar es Salaam.
For each sample (n = 4) and operator (n = 3), %CV was calculated from ten replicates on each of the three instruments (n = 30).
https://doi.org/10.1371/journal.pone.0170248.s003
(DOCX)
S2 Table. Precision (%CV) determined on internal controls and on stabilized blood.
Mean precision (%CV, coefficient of variation) in Antwerp and in Dar es Salaam, for different controls. Internal controls are in-built controls in each FACSPresto instrument (pilot 1–4 in Antwerp, and 5–7 in Dar es Salaam). CD-Chex controls (Streck) are external stabilized blood controls with a shelf life of one month (2 lots used on each FACSPresto instrument). Multicheck controls (Becton Dickinson) are external controls with a shelf life of one month (up to 3 lots used on the FACSCalibur). * n = 51 for CD4%.
https://doi.org/10.1371/journal.pone.0170248.s004
(DOCX)
S3 Table. Rejection (invalid result) occurrence when reading 447 samples in agreement study.
Rejection rate on 447 agreement samples on capillary and venous blood, in Antwerp and in Dar es Salaam.
https://doi.org/10.1371/journal.pone.0170248.s005
(DOCX)
S4 Table. Rejection (invalid result) occurrence on the different FACSPresto instruments.
Rejection rate on 447 agreement samples on capillary and venous blood, according to FACSPresto instrument (pilot 1–4 in Antwerp, and 5–7 in Dar es Salaam).
https://doi.org/10.1371/journal.pone.0170248.s006
(DOCX)
Acknowledgments
We are grateful to all patients who provided blood specimens and to all staff members involved in collecting and measuring samples.
Author Contributions
- Conceptualization: WU.
- Data curation: GD.
- Formal analysis: GD LK.
- Funding acquisition: IP WU.
- Investigation: GD SA.
- Methodology: GD WU LK.
- Project administration: IP WU.
- Resources: WU.
- Supervision: SA WU LK.
- Validation: GD.
- Visualization: GD.
- Writing – original draft: GD.
- Writing – review & editing: GD SA IP WU LK.
References
- 1.
World Health Organization. Global summary of the AIDS epidemic 2014. http://www.who.int/hiv/data/epi_core_july2015.png?ua=1 (accessed 21/03/2016). 2015.
- 2.
World Health Organization. Number of people receiving ART and percentage of all people living with HIV receiving ART in low- and middle-income countries overall and by WHO region, 2013. at http://www.who.int/hiv/data/artmap2014.png?ua=1 (accessed 21/03/2016). 2014.
- 3.
World Health Organization. Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach—2nd ed. http://apps.who.int/iris/bitstream/10665/208825/1/9789241549684_eng.pdf?ua=1 (accessed July 1st 2016). 2016.
- 4.
World Health Organization. 2013 Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection. 2013 Jun 30.
- 5.
Panel on Antiretroviral Guidelines for Adults and Adolescents. Guidelines for the use of antiretroviral agents in HIV-1-infected adults and adolescents. Department of Health and Human Services. http://www.aidsinfo.nih.gov/ContentFiles/AdultandAdolescentGL.pdf. Accessed August 18th 2016. 2016.
- 6. Ford N, Meintjes G, Pozniak A, Bygrave H, Hill A, Peter T, et al. The future role of CD4 cell count for monitoring antiretroviral therapy. Lancet Infect Dis 2015 Feb;15(2):241–7. pmid:25467647
- 7.
UNAIDS. 90-90-90 An ambitious treatment target to help end the AIDS epidemic. 2014.
- 8. Peter T, Badrichani A, Wu E, Freeman R, Ncube B, Ariki F, et al. Challenges in implementing CD4 testing in resource-limited settings. Cytometry B Clin Cytom 2008;74 Suppl 1:S123–S130.
- 9. Glencross DK, Scott LE. CD4 cell monitoring for HIV/AIDS: old options; new insights. SADJ 2001 Dec;56(12):602–5. pmid:11887447
- 10. Mandy F, Janossy G, Bergeron M, Pilon R, Faucher S. Affordable CD4 T-cell enumeration for resource-limited regions: a status report for 2008. Cytometry B Clin Cytom 2008;74 Suppl 1:S27–S39.
- 11. Glynn MT, Kinahan DJ, Ducree J. CD4 counting technologies for HIV therapy monitoring in resource-poor settings—state-of-the-art and emerging microtechnologies. Lab Chip 2013 Jul 21;13(14):2731–48. pmid:23670110
- 12.
Jani I, Sitoe N, Alfai E, Chongo P, Lehe J, Rocha B, et al. Point-of-care CD4 improves patient retention and time-to-initiation of ART in Mozambique. Vienna 2010.
- 13. Wade D, Daneau G, Aboud S, Vercauteren GH, Urassa WS, Kestens L. WHO multicenter evaluation of FACSCount CD4 and Pima CD4 T-cell count systems: instrument performance and misclassification of HIV-infected patients. J Acquir Immune Defic Syndr 2014 Aug 15;66(5):e98–107. pmid:24853304
- 14. Bwana P, Vojnov L, Adhiambo M, Akinyi C, Mwende J, Prescott M, et al. The BD FACSPresto Point of Care CD4 Test Accurately Enumerates CD4+ T Cell Counts. PLoS One 2015;10(12):e0145586. pmid:26720601
- 15. Coetzee LM, Moodley K, Glencross DK. Performance Evaluation of the Becton Dickinson FACSPresto Near-Patient CD4 Instrument in a Laboratory and Typical Field Clinic Setting in South Africa. PLoS One 2016;11(5):e0156266. pmid:27224025
- 16. Makadzange AT, Bogezi C, Boyd K, Gumbo A, Mukura D, Matubu A, et al. Evaluation of the FACSPresto, a New Point of Care Device for the Enumeration of CD4% and Absolute CD4+ T Cell Counts in HIV Infection. PLoS One 2016;11(7):e0157546. pmid:27388763
- 17. Angira F, Akoth B, Omolo P, Opollo V, Bornheimer S, Judge K, et al. Clinical Evaluation of the BD FACSPresto Near-Patient CD4 Counter in Kenya. PLoS One 2016;11(8):e0157939. pmid:27483008
- 18. Passing H, Bablok . A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, Part I. J Clin Chem Clin Biochem 1983 Nov;21(11):709–20. pmid:6655447
- 19. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 Feb 8;1(8476):307–10. pmid:2868172
- 20. Pollock MA, Jefferson SG, Kane JW, Lomax K, MacKinnon G, Winnard CB. Method comparison—a different approach. Ann Clin Biochem 1992 Sep;29 (Pt 5):556–60.
- 21.
Clinical and Laboratory Standards Institute. Method comparison and Bias Estimation Using Patient samples; Approved Guidelines—Second edition (Interim Revision) EP09-A2-IR. 2010; 17.
- 22. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960;20:37–46.
- 23. Daneau G, Buyze J, Wade D, Diaw PA, Dieye TN, Thai S, et al. CD4 results with a bias larger than hundred cells per microliter can have a significant impact on the clinical decision during treatment initiation of HIV patients. Cytometry B Clin Cytom 2016 Feb 25.
- 24. Sukapirom K, Onlamoon N, Thepthai C, Polsrila K, Tassaneetrithep B, Pattanapanyasat K. Performance evaluation of the Alere PIMA CD4 test for monitoring HIV-infected individuals in resource-constrained settings. J Acquir Immune Defic Syndr 2011 Oct 1;58(2):141–7. pmid:21709568
- 25. Scott LE, Campbell J, Westerman L, Kestens L, Vojnov L, Kohastsu L, et al. A meta-analysis of the performance of the Pima CD4 for point of care testing. BMC Med 2015;13:168. pmid:26208867
- 26. Galiwango RM, Lubyayi L, Musoke R, Kalibbala S, Buwembo M, Kasule J, et al. Field evaluation of PIMA point-of-care CD4 testing in Rakai, Uganda. PLoS One 2014;9(3):e88928. pmid:24614083
- 27. Diaw PA, Daneau G, Coly AA, Ndiaye BP, Wade D, Camara M, et al. Multisite evaluation of a point-of-care instrument for CD4(+) T-cell enumeration using venous and finger-prick blood: the PIMA CD4. J Acquir Immune Defic Syndr 2011 Dec 1;58(4):e103–e111. pmid:21909029
- 28. Daneau G, Gous N, Scott L, Potgieter J, Kestens L, Stevens W. Human Immunodeficiency Virus (HIV)-Infected Patients Accept Finger Stick Blood Collection for Point-Of-Care CD4 Testing. PLoS One 2016;11(8):e0161891. pmid:27556894
