Next Article in Journal
An Acinetobacter non-baumannii Population Study: Antimicrobial Resistance Genes (ARGs)
Next Article in Special Issue
It’s Not Easy Being Green: A Narrative Review on the Microbiology, Virulence and Therapeutic Prospects of Multidrug-Resistant Pseudomonas aeruginosa
Previous Article in Journal
Outpatient Antibiotic Prescriptions in Pregnant Women in Małopolska Province
Previous Article in Special Issue
Antimicrobial Resistance in Pathogens Isolated from Blood Cultures: A Two-Year Multicenter Hospital Surveillance Study in Italy
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Antibiotics
Volume 10
Issue 1
10.3390/antibiotics10010015
antibiotics-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Long-Term Carbapenems Antimicrobial Stewardship Program

by
José Francisco García-Rodríguez
1,*,
Belén Bardán-García
2,
Pedro Miguel Juiz-González
3,
Laura Vilariño-Maneiro
1,
Hortensia Álvarez-Díaz
1 and
Ana Mariño-Callejo
1
1
Infectious Diseases Unit, Department of Internal Medicine, University Hospital of Ferrol, Sergas, 15405 Ferrol, Spain
2
Department of Pharmacy, University Hospital of Ferrol, Sergas, 15405 Ferrol, Spain
3
Department of Microbiology, University Hospital of Ferrol, Sergas, 15405 Ferrol, Spain
*
Author to whom correspondence should be addressed.
Antibiotics 2021, 10(1), 15; https://0-doi-org.brum.beds.ac.uk/10.3390/antibiotics10010015
Submission received: 7 November 2020 / Revised: 22 December 2020 / Accepted: 23 December 2020 / Published: 26 December 2020
(This article belongs to the Special Issue Antibiotics Use and Antimicrobial Resistance in Hospital)
Browse Figures
Review Reports Versions Notes

Abstract

:
Objective. To evaluate clinical and antibiotic resistance impact of carbapenems stewardship programs. Methods: descriptive study, pre-post-intervention, between January 2012 and December 2019; 350-bed teaching hospital. Prospective audit and feedback to prescribers was carried out between January 2015 and December 2019. We evaluate adequacy of carbapenems prescription to local guidelines and compare results between cases with accepted or rejected intervention. Analysis of antibiotic-consumption and hospital-acquired multidrug-resistant (MDR) bloodstream infections (BSIs) was performed. Results: 1432 patients were followed. Adequacy of carbapenems prescription improved from 49.7% in 2015 to 80.9% in 2019 (p < 0.001). Interventions on prescription were performed in 448 (31.3%) patients without carbapenem-justified treatment, in 371 intervention was accepted, in 77 it was not. Intervention acceptance was associated with shorter duration of all antibiotic treatment and inpatient days (p < 0.05), without differences in outcome. During the period 2015–2019, compared with 2012–2014, decreased meropenem consumption (Rate Ratio 0.58; 95%CI: 0.55–0.63), candidemia and hospital-acquired MDR BSIs rate (RR 0.62; 95%CI: 0.41–0.92, p = 0.02), and increased cefepime (RR 2; 95%CI: 1.77–2.26) and piperacillin-tazobactam consumption (RR 1.17; 95%CI: 1.11–1.24), p < 0.001. Conclusions: the decrease and better use of carbapenems achieved could have clinical and ecological impact over five years, reduce inpatient days, hospital-acquired MDR BSIs, and candidemia, despite the increase in other antibiotic-consumption.

1. Introduction

Antimicrobial resistance is a threat to global public health and many countries reaffirm their commitment to develop national action plans to deal with the problem, establishing, among other measures, systems to guarantee a more appropriate use of antibiotics [1].
Antimicrobial overuse and inappropriate use remain significant problems and, although the highest consumption of antibiotics is at the outpatient level, most studies on the impact of antimicrobial stewardship programs (ASP) focused on the hospital setting, where patients are subjected to a higher pressure of antibiotic treatment and there is a greater risk of transmission of resistance.
Antimicrobial stewardship are quality improvement programs that include heterogeneous interventions [2]. The implementation of these ASP has shown a significant reduction of antibiotic use and hospital costs [3], but few studies refer to the impact on clinical outcome [4], antibiotic resistance [5,6], or incidence of Clostridioides difficile infection [7]. Interventions are generally more effective in prospective studies with clinical feedback on prescribing, but there are few studies of this type that cover the entire hospital with a duration long enough in time to evaluate the persistence of their effect [8,9].
The first steps toward ASP in an organization are to identify a leader or leaders for the program and to allocate sufficient administrative support for the ASP [10]. Limited resources for the antimicrobial stewardship implementation make necessary prioritizing those interventions that might have greater impact. Any antibiotic can have a potential effect on the development of bacterial resistance, but that effect and the persistence of resistance over time depends on the type of antibiotic and bacterial species, which may be more or less easy to develop resistance, and may have different rates of transmission of resistance [11,12,13]. In addition to the impact on bacterial resistance, different antimicrobial agents have a different impact on the human gastrointestinal colonization by the Candida species, depending on the pharmacological and antibacterial properties of each drug. It has been demonstrated in the murine model that carbapenems cause high increase of yeast counts in the gastrointestinal tract [14], and the use of carbapenems in the treatment of hospitalized patients is associated with the development of candidemia [15].
The increasing number of infections caused by extended spectrum ß-lactamase (ESBL) producing Enterobacteriaceae has led to an increase in carbapenems consumption, and the appearance and spread of carbapenem-resistant enterobacteriaceae, causing infections with high mortality due to the shortage of treatment alternatives [16].
Carbapenem-resistant enterobacteriaceae rank among the top three multidrug-resistant pathogens on the World Health Organization (WHO)’s priority list [17]. The aim of the study is to evaluate the clinical and antibiotic resistance impact of carbapenems stewardship implementation over 5 years. This manuscript is an extension of a previously published partial one [18], expanding the analysis to all carbapenems and adding years of study.

2. Results

2.1. Adequacy of Treatment

Between 2015 and2019, 1432 patients received treatment with carbapenems (1280 meropenem and 152 ertapenem). The indication of justified treatment with carbapenems progressively improved over time from 49.7% in 2015 to 80.9% in 2019, p <0.001.

2.2. Clinical Impact

Between 2015 and 2019, 1432 patients received treatment with carbapenems, of which 920 were male and 512 female; age 69.7 ± 15.2 years (range 1–97 years). The sites of infection were: urinary 518 (36.2%), abdominal 408 (28.5%), pulmonary 315 (22%), skin and soft tissue 60 (4.2%), febrile neutropenia 31 (2.2%), intravascular catheter 28 (1.9%), other 72 (5%). Place of infection acquisition: hospital-acquired 635 (44.3%), healthcare-associated 559 (39%), and community-associated 238 (16.6%).
Out of the 1432 patients who received treatment with carbapenems, in 984 (68.7%) of them the treatment was considered justified; in 448 (31.3%) the treatment was not considered justified and interventions were performed with a suggestion for appropriate treatment: in 371 (83%), the intervention was accepted, and in 77 it was not.
The clinical characteristics of patients were similar between patients with and without acceptance of ASP recommendations, although the degree of intervention acceptance varied according to prescriber (between 29% and 100%) and infection localization (Table 1).
There were no significant differences between cases with accepted intervention and cases with rejected intervention in clinical outcome or collateral damage. The acceptance of the intervention was associated with shorter duration inpatient days (p < 0.05), and less development of yeast colonization or infection, but not statistically significant (Table 2).
The Charlson index was similar throughout the intervention period (5.4 in 2015 vs. 5.8 in 2019, p = 0.07) and was higher in patients who died: 7.1 ± 2.3 vs. 5.3 ± 2.6, p < 0.001. The duration of all antibiotic treatment along the series decreased significantly from 2015 (12.8 ± 11.3 days) to 2019 (10.8 ± 10.1), p = 0.03.
Coinciding with the start-up of ASP, a significant change point was observed in 2014 in the trend analysis of meropenem consumption, in the period 2012–2014, meropenem consumption decreased −0.08% per year (95%CI: −29.4 to 41.5) and in 2014–2019 decreased −15.5% per year (95%CI: −26.2 to −3.3). There was a 42% decrease in the consumption of meropenem during the intervention period with respect to the years 2012–2014 (Rate Ratio 0.58; 95%CI: 0.5–0.6); and increased consumption of ertapenem (RR 1.2; 95%CI 1.1–1.3) (Figure 1).

2.3. Impact on Resistance

The evolution of total consumption and by class of antibiotics is shown in Figure 2. Total consumption did not decrease and the consumption of antibiotics used as an alternative to carbapenem treatment increased: cefepime (RR 2; 95%CI: 1.8–2.3) (11.7 defined daily doses (DDD)/1000 occupied bed days (OBDs) in 2012–2014 vs. 18.5 in 2015–2019), piperacillin-tazobactam (RR 1.17; 95%CI: 1.1–1.24) (54.4 DDD/1000 OBDs in 2012–2014 vs. 63.8 in 2015–2019), aminoglycosides (RR 1.2; 95%CI: 1.1–1.3 (30.2 DDD/1000 OBDs in 2012–2014 vs. 36.2 in 2015–2019), and third-generation cephalosporins (59.4 DDD/1000 OBDs in 2012–2014 vs. 66.6 in 2015–2019), p < 0.001. Ciprofloxacin and metronidazole consumption remained similar, ciprofloxacin (88.5 in 2015–2019 vs. 89.8 DDD/1000 OBDs in 2012–2014), metronidazole (30.5 in 2015–2019 vs. 32.5 DDD/1000 OBDs in 2012–2014).
Global incidence of bacteremia adjusted by 1000 OBDs increased by 5.6% during the period 2015–2019 vs. 2012–2014. The incidence density of candidemia and hospital-acquired multidrug-resistant (MDR) bloodstream infections (BSIs) decreased after ASP start-up in a matching with decrease in carbapenem consumption (Figure 1 and Figure 3). In 2015–2019 candidemia and hospital-acquired MDR BSIs rate was 0.08/1000 OBDs vs. 0.13 in 2012–2014 (RR 0.6; 95%CI: 0.4–0.9, p = 0.02) (candidemia RR 0.56, 95%CI: 0.3–1.04; MDR BSIs RR 0.7; 95%CI: 0.4–1.1). Conversely, the incidence density of hospital-acquired BSIs caused by non-MDR strains of the same microorganisms under study increased 25% during the intervention period (RR 1.2; 95%CI: 1.08–1.4, p = 0.03).
Patients with hospital-acquired candidemia and MDR BSIs had higher 30-day all-cause mortality than patients with non-MDR BSIs: 28.3% vs. 18.7%, p = 0.04. The incidence density of 30-day all-cause mortality for hospital-acquired candidemia and MDR BSIs decreased during the intervention period by 49%, without reaching statistical significance (0.02/1000 OBDs in 2015–2019 vs. 0.04 in 2012–2014; RR 0.61; 95%CI: 0.3–1.3). 30-day all-cause mortality rate for hospital-acquired bacteremia caused by non-MDR microorganisms increased over time, without statistical significance: 0.12/1000 patients-days in 2012–2014 to 0.14 in 2015–2019 (RR 1.15; 95%CI: 0.8–1.7).
Throughout the study period, neither carbapenemase-producing microorganisms nor Vancomycin-resistant Enterococcus spp. were observed in hospital-acquired bacteremia, and the incidence of Clostridioides difficile-associated diarrhea remained stable (0.20/1000 OBDs in 2015 to 0.15 in 2019; RR 0.71, 95%CI: 0.4–1.1), p = 0.19. The resistance to piperacillin-tazobactam, cefepime, and aminoglycosides in Klebsiella pneumoniae and Pseudomonas aeruginosa isolated in blood cultures did not change significantly during the intervention period. Resistance to piperacillin–tazobactam in Klebsiella pneumoniae 8.4 ± 9.2 in 2012–2104 vs. 11 ± 6.8 in 2015–2019 and in Pseudomonas aeruginosa 16.6 ± 9.3 in 2012–2104 vs. 12.7 ± 12.1 in 2015–2019. Resistance to cefepime in Klebsiella pneumoniae 11.5 ± 6.8 in 2012–2104 vs. 6.7 ± 4.7 in 2015–2019 and in Pseudomonas aeruginosa 16.6 ± 9.3 in 2012–2104 vs. 12.7 ± 12.1 in 2015–2019. Resistance to gentamicin in Klebsiella pneumoniae 4.4 ± 6 in 2012–2104 vs. 6.9 ± 4.9 in 2015–2019 and resistance in Pseudomonas aeruginosa to tobramycin 17.9 ± 14.2 in 2012–2104 vs. 3.6 ± 4.9 in 2015–2019 and to amikacin 0% throughout the 2012–2019 period.
We monitored some indicators to assess if there were changes in hospital activity that could have contributed to the decrease in the incidence density of hospital-acquired candidemia or MDR BSIs. In the period 2015–2019 compared to 2012–2014 there were increases in: the number of blood cultures performed per 1000 OBDs (RR 2.4; 95%CI: 2.39–2.5), catheter-associated BSIs rate (RR 1.2; 95%CI: 1.01–1.5) and the number of surgical procedures (RR 2.7; 95%CI: 2.69–2.8). The consumption of parenteral nutrition was similar (0 35.8 units/1000 OBDs in 2012–2014 vs. 35.4 in 2015–2019) and antifungal consumption decreased by 18% (0 27.6 in 2015–2019 vs. 32.8 DDD/1000 OBDs in 2012–2014; RR 0.82; 95%CI: 0.76–0.88, p < 0.001), Table 3.
Alcohol-based hand-rub consumption increased progressively from 2015 to 2019 about 3.6% per year (average consumption 15.1 L/1000 OBDs). Overall antibiotic use in the hospital during the study years increased from 95 DDD/1000 OBDs in 2012–2014 to 106 in 2015–2019.

3. Discussion

The implementation of our ASP improved the prescription of carbapenems and decreased their consumption, without negative impact on patient safety. The acceptance of the intervention made by infectious diseases physician decreased days of treatment, inpatients days, and the incidence of candidemia and hospital-acquired BSIs caused by MDR bacteria, despite having increased somewhat the consumption of other antibiotics that have a lower ecological impact.
The care of patients with suspected infections is complex and metrics to assess ASP impact are poorly defined [20,21]. Though shorter in their extension, other studies reported briefer hospital stays with no difference in mortality [8,22], or reported decrease in antimicrobial resistance patterns [9,23]. There are no prospective studies outside the ICUs that analyze the development of infection caused by yeast during antibiotic treatment, and only two prospective studies refer to readmissions after discharging, without differences between patients with or without acceptance of ASP recommendation [24,25]. A study carried out in a carbapenem-resistant Klebsiella pneumoniae endemic hospital showed a decrease in antibiotics consumption without changes in candidemia or consumption of antifungal [26].
The incidence of Clostridioides difficile-associated diarrhea in our hospital is low and remained stable, and our results show a decrease in the incidence of hospital-acquired candidemia and MDR BSIs; this decrease was parallel to the decrease in meropenem consumption and to the decrease in the days of all antibiotic treatment. Candidemia is the fourth cause (4.9%) of hospital-acquired BSIs in our hospital. The decrease in the duration of all antibiotic treatments, the increase in de-escalation from carbapenems to narrower-spectrum antibiotics and the lesser development of colonization caused by yeast in patients in whom the intervention was accepted, have all undoubtedly contributed to the decrease in candidemia and the consumption of antifungals, without having carried out a direct intervention on the adequate use of antifungals as in other hospitals [27]. The decrease in the incidence of yeast infection has occurred despite the increase in the number of surgical interventions and the increase of intravascular catheter-associated BSIs, with similar use of parenteral nutrition throughout the study period [28]. Our results are reinforced by a recent study that shows, on a multivariate analysis, that the treatment of patients with carbapenems was associated with candidemia, adding weight to antimicrobial stewardship efforts and restriction of the use of these broad-spectrum antibiotics [15].
The antimicrobial stewardship programs are underfunded and it is necessary to prioritize those interventions that may have a greater impact [29,30]. We decided to follow the use of carbapenems because they are the antibiotics with the broadest antibacterial spectrum and with a rapid induction of beta-lactamases. We have decreased the incidence of candidemia and hospital-acquired MDR BSIs, and associated mortality, despite the increases in the total incidence of bacteremia and global consumption of antibiotics.
Understanding the relationship between antibiotic use and antibiotic resistance is therefore critical for the design of a rational antibiotic stewardship strategy. Because antibiotic use is uneven, total use does not distinguish between broad use-many people receiving a few prescriptions- and intensive use, a few people receiving many prescriptions [31]. It is necessary to focus on those antibiotics with the most ecological impact to try to reach an appropriate level of antibiotic use [32,33]. The decrease in the consumption of meropenem was associated with a slight increase in the consumption of piperacillin-tazobactam and cefepime, which may have less ecological impact as they are less AmpC inducers [34,35], and a slight increase in third-generation cephalosporins, aminoglycosides and ertapenem. The increase in ertapenem consumption may be due to its use to treat ESBL-producing Enterobacteriaceae infections, at the patient’s hospital discharge.
The results obtained in our study are undoubtedly due to the good acceptance of the interventions by the prescribers, higher than 42.3% for persuasive interventions described in the literature [36]. The intervention rejection level it seemed to be more in relation with clinicians’ attitudes in different hospitalization units and is not associated with the severity or comorbidity of the patient, according to our previously published results [18]. The decrease in meropenem consumption does not appear to be related to demographic changes and it is not justified by a decrease in the indications for use, because the incidence of sepsis and ESBL infections has not decreased, nor does it appear to be related to changes in the information of pharmaceutical companies, since the use of new antibiotics did not increase.
The strength of our study is the large number of variables analyzed and prospective data collection over 5 years to evaluate the impact of ASP, in a community hospital (medium size) without endemicity of carbapenemase-producing Enterobacteriaceae. We assessed compliance with local guidelines as the standard for appropriate therapy to reduce the more subjective method of expert opinion-based definitions. We implement prospective audit with intervention and feedback, which is the mainstay of antimicrobial administration in the patient setting and is currently recommended based on evidence largely generated in studies conducted in large tertiary care hospitals. Community hospitals provide most of the medical care in some countries, but due to their characteristics and limited resources, there are few long-term studies in these centers on auditing and feedback of carbapenem administration [37,38]. Our study can be replicated in the hospitals where targeting a specific antibiotic is needed and an infectious diseases physician is available for intervention.
Our study has several limitations. The study was extended to the entire hospital except ICU, but the candidemia and MDR BSIs acquired in ICU between 2012and 2019 accounted for 6.6% of hospital-acquired bacteremia, without significant differences between pre and post intervention period, and we believe that the activity of this service has not influenced our results.
The sample size does not allow use an interrupted time series regression to provide good stability to the results obtained for all the variables analyzed, but they reflect the changes in consumption of meropenem and in the incidence of hospital-acquired candidemia and MDR BSIs, after starting the ASP. Meropenem consumption level drop from 50.5 DDD/1000 OBDs in 2014 to 35.2 in 2015 coinciding with the start-up of ASP, and consumption has remained lower throughout the intervention period. This change in trend seem to be due to our intervention and not to changes in healthcare during the study period. As an ecologic study, it depicts association and not causal relations, because antibiotic resistance is temporally dynamic. Further work is needed to distinguish between different causal pathways [39].
The single-center design limits the possibility of generalizing our results to other hospitals, and including preferred methods, such as control groups or randomization was impractical. Although the complexity of hospitals and infection prevention and control policies may be different, it should be noted that in last five years carbapenem-resistant Klebsiella pneumoniae and Pseudomonas aeruginosa, and MRSA is lower in our hospital than in the rest of the hospitals in our geographical area [40].

4. Materials and Methods

This study is part of a larger assessment and the detailed description of the procedure has already been previously reported elsewhere and is briefly described below.
At the end of 2014, a team of professionals was constituted for ASP implementation, local guidelines for empiric antibiotic treatment were developed, and between January 2015 and December 2019, a prospective follow-up of carbapenems use was performed. We analyzed the evolution of adequacy of carbapenems prescription to local guidelines and clinical impact, antibiotic consumption and the incidence of bloodstream infections acquired in the hospital.
A descriptive study pre-post-intervention was conducted between January 2012 and December 2019.
The study was conducted in a 350-bed teaching hospital without endemicity of carbapenemase-producing Enterobacteriaceae, located in Galicia (autonomous community of northwest of Spain). The hospital has one ICU with 10 beds and does not have transplant programs.
The infection prevention and control program was the same throughout the study. From 2012, an infectious diseases physician performed prospective active surveillance of all episodes of bloodstream infections (BSIs) [41].
Interventions:
Patients who started treatment with carbapenems available in our hospital (meropenem and ertapenem) were identified every day throughout the 5 years using a drug dispensation program (all hospital units, except ICU). Prescriber counselling measures were performed the first day of prescription, and annual training on optimization of antibiotic use was carried out for the first 3 years, targeting trainee pharmacists and physicians. An annual antimicrobial stewardship program update was presented at a hospital general clinical session.
An infectious diseases physician was released 6 h a week to perform active surveillance. For each case, the electronic medical record was reviewed by infectious diseases physician and antibiotic treatment recommendations to prescribers were given, on a face-to-face or telephone conversation basis, or through an electronic medical record. Additional differential diagnoses, investigations, and adjunctive therapy (for example, removal of urinary or central venous catheters, and drainage of infected collections) were also recommended. Adherence to, or rejection of the recommendations were reviewed by an infectious diseases physician 24 and 48 h post-recommendation, as part of the ASP workflow. Data were obtained by monitoring the information recorded in the electronic medical record. Prospective and protocolized information was collected for each case [18]. This study was approved by the Institutional Review Boards and Ethics Committee.

4.1. Adequacy of Treatment

Appropriate treatment with carbapenems was considered when it was prescribed in patients with: (1) severe sepsis [42]; (2) history of ESBLs colonization; or (3) hospital-acquired infection in which a broad-spectrum antibiotic treatment was considered necessary.

4.2. Clinical Impact

In cases in which carbapenem treatment was not justified during 2015–2019, a comparison was made between the cases with accepted intervention and cases with rejected intervention in their clinical outcome, days of antibiotic treatment, collateral damage (development of phlebitis, resistance to treatment, diarrhea caused by C. difficile, or for colonization-infection with Candida spp.), inpatient days, and hospital readmission. All-cause mortality was defined over one month of follow-up; 30-day infection-related and all-cause readmission were defined as readmission occurring within 30 days after discharge from current admission. During the study period, antibiotic consumption was assessed as defined daily doses (DDD) per 1000 occupied bed days (OBDs) [43]. The impact on the use of antibiotics was made comparing the DDD/1000 OBDs between the years 2012 and 2019.

4.3. Impact on Resistance

We analyzed, between January 2012 and December 2019, the evolution of incidence density per 1000 OBDs of hospital-acquired BSIs produced by the most frequently isolated bacteria and by Candida spp.
Hospital-acquired BSIs were defined as those diagnosed from blood cultures obtained ≥48 h after hospital admission or in those cases when, even occurring in the first 48 h, the patient had been hospitalized during the previous two weeks.
The identification of blood isolates and the determination of resistance to antibiotics were performed according to Clinical Laboratory Standard International (CLSI). The MDR categorization was applied for ESBLs or carbapenemase-producing Enterobacteriaceae, all isolates of methicillin-resistant Staphylococcus aureus (MRSA), and all Pseudomonas aeruginosa and Acinetobacter baumannii strains fulfilling the German Society for Hygiene and Microbiology criteria for MDR organisms [44]. Colonization was defined as the isolation of the organism from a non-sterile site in the absence of symptoms, and infection when the patient’s doctor prescribed treatment.

4.4. Statistical Analysis

A descriptive and comparative study of the variables was performed. Quantitative variables are reported as means ± standard deviations, and categorical as frequencies (%). Variables were compared between groups using Chi-square test or Fisher exact test for categorical variables, Student t-test, or Mann–Whitney U for continuous variables, as appropriate. Associations between the variables were expressed as odds ratios (ORs) and 95% confidence intervals (CIs). Antibiotic consumption (DDD per 1000 occupied bed days), resistance rates per 1000 OBDs of hospital acquired BSIs with a 95%CI and rates of mortality were calculated as Poisson event rates, and compared by testing for homogeneity of rates. Statistical analysis was performed using SPSS software version 19. All tests were 2-tailed; a p value < 0.05 was regarded as statistically significant.
An interrupted time-series analysis was performed to verify any significant change in antibiotic consumption line; we used the Joinpoint Regression Program 4.5.0.1 to run the calculations.

5. Conclusions

The results of this study show that the decrease and better use of carbapenems achieved by our stewardship program could have a sustained clinical and ecological impact over five years, reducing inpatient days and incidence of hospital-acquired MDR BSIs and candidemia, despite the increase in consumption of other antibiotics with less impact on the microbiome.

Author Contributions

Conceptualization, methodology, formal analysis and writing—original draft preparation, J.F.G.-R.; visualization, supervision, writing—review and editing, B.B.-G., P.M.J.-G., L.V.-M., H.Á.-D., A.M.-C. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board.

Informed Consent Statement

Patient consent was waived because it is a quality improvement study.

Data Availability Statement

Raw data is available to the public, upon request to the authors.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. United Nations. General Assembly of the United Nations: President of the 71st Session. 2016. Available online: http://www.un.org/pga/71/2016/09/21/press-release-hl-meeting-on-antimicrobial-resistance/ (accessed on 30 October 2020).
  2. Barlam, T.F.; Cosgrove, S.E.; Abbo, L.M.; MacDougall, C.; Schuetz, A.N.; Septimus, E.J.; Hamilton, C.W. Implementing an Antibiotic Stewardship Program: Guidelines by the Infectious Diseases Society of America and the Society for Healthcare Epidemiology of America. Clin. Infect. Dis. 2016, 62, e51–e77. [Google Scholar] [CrossRef] [PubMed]
  3. Karanika, S.; Paudel, S.; Grigoras, C.; Kalbasi, A.; Mylonakis, E. Systematic Review and Meta-analysis of Clinical and Economic Outcomes from the Implementation of Hospital-Based Antimicrobial Stewardship Programs. Antimicrob. Agents Chemother. 2016, 60, 4840–4852. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Schuts, E.C.; Hulscher, M.E.; Mouton, J.W.; Verduin, C.M.; Stuart, J.W.C.; Overdiek, H.W.; Schouten, J.A. Current evidence on hospital antimicrobial stewardship objetives: A systematic review and meta-analysis. Lancet Infect. Dis. 2016, 16, 847–856. [Google Scholar] [CrossRef]
  5. Lawes, T.; Lopez-Lozano, J.-M.; A Nebot, C.; Macartney, G.; Subbarao-Sharma, R.; Dare, C.R.; Wares, K.D.; Gould, I.M. Effects of national antibiotic stewardship and infection control strategies on hospital-associated and community-associated meticillin-resistant Staphylococcus aureus infections across a region of Scotland: A non-linear time-series study. Lancet Infect. Dis. 2015, 15, 1438–1449. [Google Scholar] [CrossRef]
  6. Molina, J.; Peñalva, G.; Gil-Navarro, M.V.; Praena, J.; A Lepe, J.; A Pérez-Moreno, M.; Ferrándiz, C.; Aldabó, T.; Aguilar, M.; Olbrich, P.; et al. Long-Term Impact of an Educational Antimicrobial Stewardship Program on Hospital-Acquired Candidemia and Multidrug-Resistant Bloodstream Infections: A Quasi-Experimental Study of Interrupted Time-Series Analysis. Clin. Infect. Dis. 2017, 65, 1992–1999. [Google Scholar] [CrossRef] [Green Version]
  7. Lawes, T.; Lopez-Lozano, J.-M.; A Nebot, C.; Macartney, G.; Subbarao-Sharma, R.; Wares, K.D.; Sinclair, C.; Gould, I.M. Effect of a national 4C antibiotic stewardship intervention on the clinical and molecular epidemiology of Clostridium difficile infections in a region of Scotland: A non-linear time-series analysis. Lancet Infect. Dis. 2017, 17, 194–206. [Google Scholar] [CrossRef]
  8. Liew, Y.X.; Lee, W.; Loh, J.C.Z.; Cai, Y.; Tang, S.S.L.; Lim, C.L.L.; Chlebicki, M.P. Impact of an antimicrobial stewardship program on patient safety in Singapore General Hospital. Int. J. Antimicrob. Agents 2012, 40, 55–60. [Google Scholar] [CrossRef]
  9. Tedeschi, S.; Trapani, F.; Giannella, M.; Cristini, F.; Tumietto, F.; Bartoletti, M.; Liverani, A.; Pignanelli, S.; Toni, L.; Pederzini, R.; et al. An Antimicrobial Stewardship Program Based on Systematic Infectious Disease Consultation in a Rehabilitation Facility. Infect. Control. Hosp. Epidemiol. 2016, 38, 76–82. [Google Scholar] [CrossRef]
  10. Marcelin, J.R.; Chung, P.; Van Schooneveld, T. Antimicrobial stewardship in the outpatient setting: A review and proposed framework. Infect. Control. Hosp. Epidemiol. 2020, 41, 833–840. [Google Scholar] [CrossRef]
  11. Van Duin, D.; A Arias, C.; Komarow, L.; Chen, L.; Hanson, B.M.; Weston, G.; Cober, E.; Garner, O.B.; Jacob, J.T.; Satlin, M.J.; et al. Molecular and clinical epidemiology of carbapenem-resistant Enterobacterales in the USA (CRACKLE-2): A prospective cohort study. Lancet Infect. Dis. 2020, 20, 731–741. [Google Scholar] [CrossRef]
  12. Mathers, A.J.; Vegesana, K.; German-Mesner, I.; Ainsworth, J.; Pannone, A.; Crook, D.; Sifri, C.; Sheppard, A.; Stoesser, N.; Peto, T.; et al. Risk factors for Klebsiella pneumoniae carbapenemase (KPC) gene acquisition and clinical outcomes across multiple bacterial species. J. Hosp. Infect. 2020, 104, 456–468. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Bush, K.; Bradford, P.A. Epidemiology of ß-Lactamase-Producing Pathogens. Clin. Microbiol. Rev. 2020, 33, eooo47-19. [Google Scholar] [CrossRef] [PubMed]
  14. Samonis, G.; Galanakis, E.; Ntaoukakis, M.; Sarchianaki, E.; Spathopoulou, T.; Dimopoulou, D.; Kofteridis, D.P.; Maraki, S. Effects of carbapenems and their combination with amikacin on murine gut colonisation byCandida albicans. Mycoses 2012, 56, 105–109. [Google Scholar] [CrossRef] [PubMed]
  15. Keighley, C.; Pope, A.; Marriott, D.; Chapman, B.; Bak, N.; Daveson, K.; Sorrell, T.C. Risk factors for candidaemia: A prospective multi-centre case-control study. Mycoses 2020. [Google Scholar] [CrossRef]
  16. Keighley, C.; Pope, A.; Marriott, D.; Chapman, B.; Bak, N.; Daveson, K.; Sorrell, T.C. European Antimicrobial Resistance Genes Surveillance Network EURGen-Net Capacity Survey Group. Worsening epidemiological situation of carbapenemase-producing Enterobacteriaceae in Europe, assessment by national experts from 37 countries, July 2018. Euro Surveill 2019, 24. [Google Scholar] [CrossRef]
  17. Tacconelli, E.; Carrara, E.; Savoldi, A.; Harbarth, S.; Mendelson, M.; Monnet, D.L.; Pulcini, C.; Kahlmeter, G.; Kluytmans, J.; Carmeli, Y.; et al. Discovery, research, and development of new antibiotics: The WHO priority list of antibiotic-resistant bacteria and tuberculosis. Lancet Infect. Dis. 2018, 18, 318–327. [Google Scholar] [CrossRef]
  18. García-Rodríguez, J.F.; Bardán-García, B.; Peña-Rodríguez, M.F.; Álvarez-Díaz, H.; Mariño-Callejo, A. Meropenem antimicrobial stewardship program: Clinical, economic, and antibiotic resistance impact. Eur. J. Clin. Microbiol. Infect. Dis. 2019, 38, 161–170. [Google Scholar] [CrossRef]
  19. Estimation of Weights and Costs of Hospital Processes in the National Health System: Methodology and Main Results. Health Information and Statistics 2019. Ministry of Health, Consumption and Social Welfare. Available online: https://www.mscbs.gob.es/estadEstudios/estadisticas/docs/CMBD/Inf_proc_estim_cost_hosp_SNS2017.pdf (accessed on 30 October 2020).
  20. Moehring, R.W.; Anderson, D.J.; Cochran, R.L.; Hicks, L.A.; Srinivasan, A.; Ashley, E.S.D.; Structured Taskforce of Experts Working at Reliable Standards for Stewardship (STEWARDS) Panel. Expert Consensus on Metrics to Assess the Impact of Patient-Level Antimicrobial Stewardship Interventions in Acute-Care Settings. Clin. Infect. Dis. 2017, 64, 377–383. [Google Scholar] [CrossRef] [Green Version]
  21. Spivak, E.S.; Cosgrove, S.E.; Srinivasan, A. Measuring Appropriate Antimicrobial Use: Attempts at Opening the Black Box. Clin. Infect. Dis. 2016, 63, 1–6. [Google Scholar] [CrossRef] [Green Version]
  22. Lin, Y.-S.; Lin, I.-F.; Yen, Y.-F.; Lin, P.-C.; Shiu, Y.-C.; Hu, H.-Y.; Yang, Y.-P. Impact of an antimicrobial stewardship program with multidisciplinary cooperation in a community public teaching hospital in Taiwan. Am. J. Infect. Control. 2013, 41, 1069–1072. [Google Scholar] [CrossRef]
  23. Lew, K.Y.; Ng, T.M.; Tan, M.; Tan, S.H.; Lew, E.L.; Ling, L.M.; Teng, C.B. Safety and Clinical outcomes of carbapenem de-escalation as part of an antimicrobial stewardship program in an ESBL-endemic setting. J. Antimicrob. Chemother. 2015, 70, 1219–1225. [Google Scholar] [PubMed] [Green Version]
  24. Teng, C.B.; Ng, T.M.; Tan, M.W.; Tan, S.H.; Tay, M.; Lim, S.F.; Ling, L.M.; Brenda, A.S.P.; Lye, D.C.B. Safety and effectiveness of improving carbapenem use via prospective review and feedback in a multidisciplinary antimicrobial stewardship programme. Ann. Acad. Med. Singap. 2015, 44, 19–25. [Google Scholar] [PubMed]
  25. Sadyrbaeva-Dolgova, S.; Aznarte-Padial, P.; Jimenez-Morales, A.; Expósito-Ruiz, M.; Calleja-Hernández, M.Á.; Hidalgo-Tenorio, C. Pharmacist recommendations for carbapenem de-escalation in urinary tract infection within an antimicrobial stewardship program. J. Infect. Public Health 2020, 13, 558–563. [Google Scholar] [CrossRef] [PubMed]
  26. Giacobbe, D.; On behalf of the San Martino Antimicrobial Stewardship Group; Del Bono, V.; Mikulska, M.; Gustinetti, G.; Marchese, A.; Mina, F.; Signori, A.; Orsi, A.; Rudello, F.; et al. Impact of a mixed educational and semi-restrictive antimicrobial stewardship project in a large teaching hospital in Northern Italy. Infection 2017, 45, 849–856. [Google Scholar] [CrossRef] [PubMed]
  27. Martín-Gutiérrez, G.; Peñalva, G.; De Pipaón, M.R.-P.; Aguilar, M.; Gil-Navarro, M.V.; Pérez-Blanco, J.L.; Pérez-Moreno, M.A.; Amaya-Villar, R.; Ferrándiz-Millón, C.; Gascón, M.L.; et al. Efficacy and safety of a comprehensive educational antimicrobial stewardship program focused on antifungal use. J. Infect. 2020, 80, 342–349. [Google Scholar] [CrossRef] [PubMed]
  28. Poissy, J.; Funginos, T.; Damonti, L.; Bignon, A.; Khanna, N.; Von Kietzell, M.; Boggian, K.; Neofytos, D.; Vuotto, F.; Coiteux, V.; et al. Risk factors for candidemia: A prospective matched case-control study. Crit. Care 2020, 24, 1–11. [Google Scholar] [CrossRef] [PubMed]
  29. Stenehjem, E.; Hyun, D.Y.; Septimus, E.; Yu, K.C.; Meyer, M.; Raj, D.; Srinivasan, A. Antibiotic Stewardship in Small Hospitals: Barriers and Potencial Solutions. Clin. Infect. Dis. 2017, 65, 691–696. [Google Scholar] [CrossRef]
  30. Apisarnthanarak, A.; Bhooanusas, N.; Yaprasert, A.; Mundy, L.M. Carbapenem De-escalation Therapy in a Resource-Limited Setting. Infect. Control. Hosp. Epidemiol. 2013, 34, 1310–1313. [Google Scholar] [CrossRef]
  31. Olesen, S.W.; Barnett, M.L.; MacFadden, D.R.; Brownstein, J.S.; Hernández-Díaz, S.; Lipsitch, M.; Grad, Y.H. The distribution of antibiotic use and its association with antibiotic resistance. eLife 2018, 7, e39435. [Google Scholar] [CrossRef]
  32. López-Lozano, J.-M.; THRESHOLDS Study Group; Lawes, T.; Nebot, C.; Beyaert, A.; Bertrand, X.; Hocquet, D.; Aldeyab, M.; Scott, M.; Conlon-Bingham, G.; et al. A nonlinear time-series analysis approach to identify thresholds in associations between population antibiotic use and rates of resistance. Nat. Microbiol. 2019, 4, 1160–1172. [Google Scholar] [CrossRef]
  33. Kazakova, S.V.; Baggs, J.; McDonald, L.C.; Yi, S.H.; Hatfield, K.M.; Guh, A.; Reddy, S.C.; A Jernigan, J. Association Between Antibiotic Use and Hospital-onset Clostridioides difficile Infection in US Acute Care Hospitals, 2006–2012: An Ecologic Analysis. Clin. Infect. Dis. 2020, 70, 11–18. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Meini, S.; Tascini, C.; Cei, M.; Sozio, E.; Rossolini, G.M. AmpC β-lactamase-producing Enterobacterales: What a clinician should know. Infection 2019, 47, 363–375. [Google Scholar] [CrossRef] [PubMed]
  35. Tabak, Y.P.; Srinivasan, A.; Yu, K.C.; Kurtz, S.G.; Gupta, V.; Gelone, S.; Scoble, P.J.; McDonald, L.C. Hospital-level high-risk antibiotic use in relation to hospital-associated Clostridioides difficile infections: Retrospective analysis of 2016–2017 data from US hospitals. Infect. Control. Hosp. Epidemiol. 2019, 40, 1229–1235. [Google Scholar] [CrossRef] [PubMed]
  36. Davey, P.; A Marwick, C.; Scott, C.L.; Charani, E.; McNeil, K.; Brown, E.; Gould, I.M.; Ramsay, C.R.; Michie, S. Interventions to improve antibiotic prescribing practices for hospital inpatients. Cochrane Database Syst. Rev. 2017, 2, CD003543. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Anderson, D.J.; Watson, S.; Moehring, R.W.; Komarow, L.; Finnemeyer, M.; Arias, R.M.; Huvane, J.; Hill, C.B.; Deckard, N.; Sexton, D.J.; et al. Feasibility of Core Antimicrobial Stewardship Interventions in Community Hospitals. JAMA Netw. Open 2019, 2, e199369. [Google Scholar] [CrossRef] [PubMed]
  38. Stenehjem, E.; Hersh, A.L.; Buckel, W.R.; Jones, P.; Sheng, X.; Evans, R.S.; Burke, J.P.; Lopansri, B.K.; Srivastava, R.; Greene, T.; et al. Impact of Implementing Antibiotic Stewardship Programs in 15 Small Hospitals: A Cluster-Randomized Intervention. Clin. Infect. Dis. 2018, 67, 525–532. [Google Scholar] [CrossRef]
  39. Zhang, D.; Hu, S.; Sun, J.; Zhang, L.; Dong, H.; Feng, W.; Lei, J.; Lu, T. Antibiotic consumption versus the prevalence of carbapenem-resistant Gram-negative bacteria at a tertiary hospital in China from 2011 to 2017. J. Infect. Public Health 2019, 12, 195–199. [Google Scholar] [CrossRef]
  40. Informe SOGAMIC Sobre Resistencias a Antimicrobianos en Galicia. 2019 December. Dirección Xeral de Saúde Pública da Consellería de Sanidade. Available online: http://sogamic.es/wp-content/uploads/2019/12/Informe-SOGAMIC-sobre-resistencias-2018-1.pdf (accessed on 30 October 2020).
  41. García-Rodríguez, J.F.; Álvarez, H.; Vilariño-Maneiro, L.; Lorenzo-García, M.V.; Canton, A.; Ordoñez-Barrosa, P.; Mariño-Callejo, A.I.; Sesma-Sánchez, P. Epidemiology and impact of a multifaceted approach in controlling central venous catheter associated blood stream infections outside the intensive care unit. BMC Infect. Dis. 2013, 13, 445. [Google Scholar] [CrossRef] [Green Version]
  42. Dellinger, R.P.; Levy, M.M.; Rhodes, A.; Annane, D.; Gerlach, H.; Opal, S.M.; Osborn, T.M. Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock: 2012. Crit. Care Med. 2013, 41, 580–637. [Google Scholar] [CrossRef]
  43. WHO Collaborating Center for Drug Statistics Methodology. DDD Definition and General Considerations. 2016. Available online: https://www.whocc.no/ddd/definition_and_general_considera/ (accessed on 8 May 2020).
  44. Mattner, F.; Bange, F.C.; Meyer, E.; Seifert, H.; Wichelhaus, T.A.; Chaberny, I.F. Preventing the spread of multidrug-resistant gram-negative pathogens: Recommendations of an expert panel of the German Society For Hygiene and Microbiology. Dtsch. Arztebl. Int. 2012, 109, 39–45. [Google Scholar]
Antibiotics 10 00015 g001 550
Figure 1. Evolution of carbapenems consumption and incidence of candidemia and MDR bacteria isolated in-hospital bloodstream infections (BSIs).
Figure 1. Evolution of carbapenems consumption and incidence of candidemia and MDR bacteria isolated in-hospital bloodstream infections (BSIs).
Antibiotics 10 00015 g001
Antibiotics 10 00015 g002 550
Figure 2. Evolution of antibiotic consumption between the years 2012 to 2019.
Figure 2. Evolution of antibiotic consumption between the years 2012 to 2019.
Antibiotics 10 00015 g002
Antibiotics 10 00015 g003 550
Figure 3. Evolution of the incidence density of candidemia and MDR bacteria isolated most frequently in-hospital bloodstream infections.
Figure 3. Evolution of the incidence density of candidemia and MDR bacteria isolated most frequently in-hospital bloodstream infections.
Antibiotics 10 00015 g003
Table
Table 1. Baseline demographic and clinical characteristics of patients with and without acceptance of ASP Recommendations. Years 2015–2019.
Table 1. Baseline demographic and clinical characteristics of patients with and without acceptance of ASP Recommendations. Years 2015–2019.
VariablesIntervention
Accepted n = 371		Intervention
Rejected n = 77		p
Male gender, n (%)243 (65.5)48 (62.3)0.6
Median age ± SD, years (range)67.2 ± 15.5 (10–96)66 ± 18.8 (6–95)0.6
Charlson comorbidity score, Median ± SD, (range)5.2 ± 2.9 (0–13.6)4.8 ± 2.9 (0–12)0.4
Neutropenia, <500/mL7 (1.9)3 (3.9)0.4
Sepsis15 (4)5 (6.5)0.4
Site of infections, n (%)
Pulmonary64 (17.3)24 (31.2)0.007
Abdominal99 (26.7)36 (46.8)0.001
Skin/soft tissue25 (6.7)1 (1.3)0.06
Urinary131 (35.3)12 (15.6)0.001
Other52 (14)4 (5.2)0.04
Acquisition place of infection
Hospital onset127 (34.2)33 (42.8)0.15
Healthcare-associated134 (36.1)28 (36.4)1
Community-associated110 (29.6)16 (20.8)0.13
Table
Table 2. Clinical results of patients with and without Acceptance of ASP recommendations. Years 2015–2019.
Table 2. Clinical results of patients with and without Acceptance of ASP recommendations. Years 2015–2019.
VariablesIntervention
Accepted n = 371		Intervention
Rejected n = 77		p
Evolution to healing329 (88.7%)64 (83.1%)0.4
Death caused by infection15 (4%)7 (9.1%)0.08
All-cause crude death42 (11.3%)12 (15.6%)0.3
Readmission in a month15 # (4%)4 * (5.2%)0.6
Adverse effects42 (11.3%)7 (9.1%)0.8
Phlebitis64 (17.3%)11 (14.3%)0.6
Development of resistance to treatment8 (2.2%)0 (0%)0.4
Diarrhea caused by C. difficile6 (1.6%)2 (2.6%)0.6
Colonization-Infection with Candida spp.40 (10.8%)11 (14.3%)0.4
Days of antibiotic treatment (intervention series)11 ± 10.212.7 ± 8.50.2
Total inpatient days, X ± SD17.7 ± 16.725.3 ± 22.30.006
Inpatient days post-intervention, X ± SD12.5 ± 14.216.7 ± 18.90.03
# 9 relapses of the infection, 6 due to other causes. * No relapse of infection due to other cause.
Table
Table 3. Potential Changes in Healthcare during the Study Period by Year.
Table 3. Potential Changes in Healthcare during the Study Period by Year.
Healthcare Variable.20122013201420152016201720182019
Nº of patients admitted14,72114,61514,97914,86714,85215,24815,56115,641
Nº of inpatient days119,885121,181119,615116,588114,072114,864120,133119,350
Blood cultures performed, No.32423340298530033419307435543331
Nº blood cultures/1000 OBDs *27.027.624.925.829.926.829.627.9
Hospital-acquired BSIs per 1000 OBDs *0.80.90.80.80.81.01.41.0
Hospital-acquired no-MDR
BSIs/1000 OBDs *		0.70.80.70.70.80.91.30.9
Intravascular catheter-associated
BSIs/1000 OBDs * 		0.30.40.50.40.40.50.70.4
Surgical procedures, Nº/1000 OBDs *.7477747882807679
Case mix index +1.51.51.61.60.80.90.90.9
Parenteral nutrition units,
No./1000 OBDs 		29.138.739.546.141.334.028.926.5
Consumption of antifungals,
DDD/1000 OBDs *		29.138.430.834.933.125.523.621.1
MDR: multidrug-resistant. BSIs: bloodstream infections. OBDs: occupied bed days. * In the period 2015–2019 compared to 2012–2014 increased the number of blood cultures performed, hospital-acquired BSIs and hospital-acquired no-MDR BSIs, catheter-associated BSIs and surgical procedures; antifungal consumption decreased (p < 0.001). + The calculation system was modified after 2015 (Ref. [19]), case mix index increased progressively between 2012 and 2014 and between 2015 and 2019.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

García-Rodríguez, J.F.; Bardán-García, B.; Juiz-González, P.M.; Vilariño-Maneiro, L.; Álvarez-Díaz, H.; Mariño-Callejo, A. Long-Term Carbapenems Antimicrobial Stewardship Program. Antibiotics 2021, 10, 15. https://0-doi-org.brum.beds.ac.uk/10.3390/antibiotics10010015

AMA Style

García-Rodríguez JF, Bardán-García B, Juiz-González PM, Vilariño-Maneiro L, Álvarez-Díaz H, Mariño-Callejo A. Long-Term Carbapenems Antimicrobial Stewardship Program. Antibiotics. 2021; 10(1):15. https://0-doi-org.brum.beds.ac.uk/10.3390/antibiotics10010015

Chicago/Turabian Style

García-Rodríguez, José Francisco, Belén Bardán-García, Pedro Miguel Juiz-González, Laura Vilariño-Maneiro, Hortensia Álvarez-Díaz, and Ana Mariño-Callejo. 2021. "Long-Term Carbapenems Antimicrobial Stewardship Program" Antibiotics 10, no. 1: 15. https://0-doi-org.brum.beds.ac.uk/10.3390/antibiotics10010015

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop