Abstract

REVIEW PAPERS

Applicability of the national nosocomial infections surveillance system risk index for the prediction of surgical site infections: a review

Flávia Falci Ercole^I; Carlos Ernesto Ferreira Starling^II; Tânia Couto Machado Chianca^I; Mariângela Carneiro^III

^INurse Department, Federal University of Minas Gerais

^IILife-Center, Vera Cruz Hospital; Department of Parasitology, ICB, Federal University of Minas Gerais

^IIIBelo Horizonte, MG, Brazil

^{Address for correspondence} Address for correspondence: Dr. Flávia Falci Ercole Av. Prof. Afredo Balena, 190, Santa Efigênia Zip code: 30.130-100, Belo Horizonte, MG, Brazil Phone: 55 31 32489887/55 31 32912273/88770221 E-mail: fercole@enf.ufmg.br

ABSTRACT

This review outlines the evolution of risk rates of surgical site infection adopted by the Study on the Efficacy of Nosocomial Infection Control (SENIC) and the National Nosocomial Infections Surveillance (NNIS) system, and evaluates their advantages and limitations. The types of surgical procedures for which the NNIS index has proven to be an efficient system for the classification of patients are presented. The necessity of adapting the NNIS index to take into account specific characteristics associated with hospitals and practices in various regions and countries is stressed. The requirement for further testing and development of the rate concerning specific procedures, particularly those involving orthopaedic surgery, is discussed.

Key-Words: SSI/NNIS Index, quality indicators, hospital infections, surgical procedures, orthopaedic operations.

Hospital infection remains a major health care problem not only in Brazil but also in developed countries [1-2]. It is estimated that in Brazilian hospitals between 0.55 and 1.1 million cases of hospital infections occur each year [3-5]. However, the actual number of cases is probably greater than these estimates since infection rates are generally under-recorded by most hospitals. This imprecision in the data is typically caused by difficulties in establishing an efficient epidemiological surveillance system during the period of patient hospitalisation and, most importantly, after the patient has been discharged [3-8].

According to some reports, 2 million patients acquired hospital infections in the USA in 1996 at a cost of US$ 4.5 million [7,9], whilst surgical site infections (SSIs) affected 325,000 hospitalised patients giving rise to an average extra expenditure of US$ 1 to 2 billion [9]. In England, the cost of hospital infections reached £111 million (ca. US$ 200 million) in 1986, and SSIs resulted in an increase of 8.2 days in the average period of hospitalisation, at a typical extra cost of £1041 (ca. US$ 1850), per patient [10,11].

A National Nosocomial Infections Surveillance (NNIS) survey covering the period from January 1993 to April 1995 indicated that, out of hospital infections reported, those involving the urinary tract were the most frequent (27.2%), followed by SSIs (18.7%) and finally pneumonia (17.3%) [6,12]. However, the occurrence of SSIs is increasing in general hospitals worldwide. A prospective study of 42,274 operations conducted during a 23-year period in the general surgical section of a Brazilian university hospital revealed that SSIs were the most frequent type of infection (11%), followed by respiratory (4 %) and urinary (2.8%) tract infections [13]. Another study conducted in Brazil reported that SSIs were the second or third most frequent type of infection found, being responsible for 16% of all hospital infections [14].

SSIs are considered to be surgical complications and may be defined as those infections that affect tissues, organs and cavities that have been manipulated or have suffered incision during a surgical procedure. These infections appear in the postoperative period (up to the 30^th day), or within the first year in the case of a prosthetic insertion [6,13,15,16]. The incidences of SSIs differ from one country to another according to the different systems employed for the epidemiological control of hospital infections [6,10,14,15,17-20]. Some authors report that the infection rate varies between 2.8 and 20% depending on the hospital, the clinical conditions of the patients and the type of procedures performed, and emphasise that the prevalence of SSIs results in increasing rates of morbidity, prolonged hospitalisation times and considerable socio-economic costs [20].

The system of epidemiological control of hospital infections within an institution can provide important information concerning the groups, the factors and the procedures that are most associated with the risk of SSI. Such information can be used to identify the most vulnerable cases and can reduce infections in patients undergoing surgery by 50% [13,14,21]. Furthermore, a key component of infection control systems is the stratification of infection rates according to SSI-associated risks, and this permits comparison of rates during different periods inside an institution or among institutions [10,22].

Whilst the main factors that determine the development of SSI are the characteristics of the patient and exposure to sources of infection, these aspects cannot isolatedly explain most of infections. Postoperative SSIs are linked essentially to a combination of factors and the risks differ significantly amongst patients [10,15]. Consequently, the surveillance methodology developed by the NNIS system, and adopted by the Centres for Disease Control and Prevention (CDC), is being continuously refined in order to allow the classification of surgical patients according to both intrinsic and extrinsic risk factors [10].

Typically, SSIs are complications which are inherent to advanced techniques that use highly invasive procedures and occur mainly in young patients (especially premature babies), elderly patients and patients that are immunodepressed [6]. Incidences of SSI that originate primarily from the care procedures provided during hospitalisation, which is about 30% of cases, can be avoided if appropriate precautions are taken [13]. For this reason, the quality of assistance given by hospitals has received considerable attention in the last decade and various indicators for assessment and improvement have evolved [6,8,13]. On the other hand, if indicators of the risk of SSI are to be used for comparison purposes, they should only assess the rates of infection relating to the various centres, services or surgeons that originate directly from the surgical procedure. For this reason, the risk element relating to the care given to patients in several health institutions must be excluded from the SSI indicator [23-25]. Normally, the adjustment method used is the one recommended by the NNIS when calculating the SSI risk index, which allows comparisons between institutions or surgical specialties that are independent on the profile of the patients assisted [6,19,22].

The present review evaluates the advantages and limitations of the SSI index adopted by NNIS in comparison to other relevant rates. The predicative efficiency of the SSI/NNIS index is considered with respect to various surgical procedures, and its applicability to orthopaedic operations is considered. The literature published between 1915 and 2005, available in the Index Medicus of the National Library of Medicine (MEDLINE), Banco de Dados em Enfermagem (BDENF), Literatura Latino-Americana do Caribe em Ciências da Saúde (LILACS) and the Cochrane Library, has been analysed.

Determination of Risk Indices

The increasing concern by private and public health institutions regarding the type of care offered to patients reflects a highly competitive business managed on commercial lines. Factors that increase the volume of services provided, reduce the length of hospitalisation, and decrease the cost of assistance rendered, whilst maintaining patient satisfaction, all determine the stability of an institution in the health care industry. Therefore, the use of indicators for measuring the quality of these services is an objective strategy of the surveillance process [26].

The infection rate within an institution or a service is a clinical indicator that helps in the evaluation of the quality of service delivered. Such indicators do not measure the quality of assistance directly, but make it easier for the institution to identify problems and to focus attention on lapses in performance and, thus, on the solutions of specific difficulties. Using such indices, the quality of assistance can be monitored within an institution or compared between institutions over a period of time [19,27].

Indices or rates of infection that are calculated by dividing the number of diagnosed SSIs by the number of admitted or discharged patients from a service over a particular period of time are poor indicators since they do not allow accurate comparisons and do not precisely convey the magnitude of the problem [19,22,28,29]. In order to establish comparisons, the denominator must take into account the size of the population exposed to specific risks, reflecting in particular the risk involved in each of the surgical procedures performed, the type of illness, the condition of the patient at the time of surgery, and specified intrinsic risks [19,22,29-31].

Comparable indices require adjustment according to the intrinsic risk of each sample population and each procedure. Thus, indicators must include parameters that take into account the structure, the process and the result of the assistance delivered, and the variability of these parameters should be used to adjust the final index value [19,22,29]. The SSI indices that have been developed in the last few decades are considered to represent an important advance in health care since they employ statistical analysis in the evaluation of hospital assistance thus conveying certain legitimacy to the results [31]. However, the idea of controlling the intrinsic risks of SSIs is not new, and clinical indicators have been used since 1895 in order to compare infection rates amongst surgeons. In 1915, Brewer surprised his colleagues at the Roosevelt Hospital in New York when he demonstrated statistically that the infection rate following "clean" abdominal surgery was 39%. Later, following innovations in surgical techniques, the infection rate for this type of operation was reduced by 95% [32].

The first mention of the surgical wounds classification was by Goff (1925) in a study conducted in the Woman's Hospital of New York in which the suture technique, the type of suture employed and the category of the abdominal surgery (clean or contaminated/infected) were considered in order to compare infection rates [33]. This approach, involving the standardisation of parameters for comparative studies, was taken up in 1964 by the National Research Council [34] in the construction of the first SSI risk indicator in which surgical procedures were classified according to their contamination potential (Table 1). One of the major limitations of this system was, however, that it ignored the intrinsic risk of patients in developing postoperative infection [35,36].

In 1961, Lidwell used multiple regression analysis to evaluate postoperative infections involving 3,000 patients in 20 different hospitals [37]. The results of this study were important because they showed that hospitals presented dissimilar infection rates that could be attributed to the characteristics of the patients, the quality of assistance delivered by the hospitals, and the types of surgery involved [30,37]. In the early 1970s, Cruse and Foord [38] reported the results of a prospective study of 23,649 surgical patients of which 1,124 presented infected wounds. Whilst the results provided information concerning infection rates for individual surgeons and services, they were not used for the development of a risk index [30]. Contemporaneous studies by Davidson et al. [39] involved multiple linear regression analysis of the risk factors associated with SSIs in 1,000 patients. Whilst no risk index was developed from this paper, the authors did establish that five independent factors were of significant importance (> 90% correlation) in the risk of SSI, namely, age of the patient, length of the surgery, type of procedure, presence of bacteria in the surgical wound, and the surgical environment.

Various studies conducted in the 1980s also analysed risk factors associated with SSI and, although they did not lead to the development of a risk index, they had the merit of pioneering this area of research [30]. In such study, Ehrenkranz [40] analysed SSI events following the performance of eight different types of surgical procedures carried out on 9,108 patients in a community hospital. The risk factors used for the classification of the population studied were remote infection, diabetes and long operations (above 4 h). This range of risk factors was extended, however, by studies involving multivariate analyses carried out by Simchen et al. [41], and using logistic regression analysis as employed by Nichols et al. [42]. In the latter, age, blood transfusion, colostomy and multiple organ lesions were identified as the main factors associated with the development of postoperative SSI in surgeries related to abdominal trauma (intestinal perforation) in 145 patients. Hooton et al. [43] used the chi-squared Automatic Interaction Detector (CHAID) technique (designed to partition a sample into mutually exclusive and homogeneous risk groups based on categorical data) to analyse data collected from the medical records of American hospitals participating on the project Study on the Efficacy of Nosocomial Infection Control (SENIC). Three site-specific multivariate indices for the control of various risk factors were developed and, whilst these indices could not be applied routinely owing to their complexity, they were used in the analyses performed by SENIC [30].

It was at this time that the American Society of Anaesthesiology (ASA) created its classification of the clinical conditions of patients (Table 2). The scoring system employed has, however, been criticised for its subjectivity, and this is supported by the low level of agreement between anaesthesiologists in applying the categorisation. Since the ASA classification is employed in the SSI indices subsequently developed by NNIS (as described below), this aspect of subjectivity remains problematic [44-45].

The first generally applicable SSI risk index was elaborated by Haley et al. [21] who employed logistic regression analysis to study 58,498 operations based on SENIC data. Amongst the 10 putative risk factors, four were shown to be independently associated with infection, namely, abdominal surgery, length of surgery (longer than 2 h), infected surgical areas, and patients presenting more than three diagnoses at the time of their discharge from hospital. Taking into account these variables, the authors created the SENIC index for assessing the level of risk involved in terms of the four factors considered (Table 3). Later on, Christou and co-workers developed an SSI index model that included the nutritional state and the defence mechanisms of the patient amongst the cited risk factors [30,46]. This new index proved to be a better predictor of SSI risk than the traditional system based on the classification of surgical wounds [30,35,36].

There is no doubt that an index composed of several risk factors was preferable to the approaches previously used for comparing SSI rates between hospitals and surgeons [35,47-49] and, furthermore, the SENIC index was also better at predicting occurrences of SSI compared with earlier methods [19,35,36,50]. Whilst the SENIC index was quite accurate in its predictions in many cases, the model was not suitable for routine application due to the lack of precision of two out of four variables, i.e. patient health state and the length of the operation. The problem with the former variable was that it was determined only at the point when the patient was discharged from hospital, whilst the second variable created a cut-off point of 2 h for the length of an operation irrespective of the surgical procedure considered. In order to address these problems, the NNIS proposed a corrected risk index in 1991 that contained three out of four variables already included in the SENIC index [3,28,36,49].

Evolution of the NNIS System and Development of the SSI/NNIS Index

The NNIS system was developed in the early 1970s when hospitals in the USA were selected to report their epidemiological surveillance data for inclusion in a national data bank [51,52]. Data were collected using four standard protocols and the clinical criteria and definitions used were those adopted by CDC [51]. The methodology proposed by NNIS was used in five hospitals in Brazil in 1991, and the method was soon accepted as being adequate for Brazilian hospitals. Later, the NNIS method was extended to more than 70 hospitals in Brazil, Argentina and Uruguay for the purpose of a prospective multi-centred study [3].

In the early 1990s, European countries had either not adopted a hospital infection surveillance system or, at best, used systems that were somewhat limited. In 1994, however, at the behest of the Commission of the European Community, the regional and national public health institutes in charge of the coordination of hospital-acquired infection surveillance set up the collaborative network Hospitals in Europe Link for Infection Control through Surveillance (HELICS). The main objective of this programme was to create a database intended for the comparative analysis of the rates of hospital-acquired infections in the 15 countries of the European Union. HELICS generated three new programmes of infection surveillance: the Nosocomial Infection. National Surveillance Service (NINSS) for English hospitals; the Dutch Preventie van Ziekenhuisinfecties door Surveillance (PREZIES), and the German Krankenhaus Infections Surveillance System (KISS). All three methods were based on the American NNIS model, although they were adapted to meet the specific characteristics of the respective countries. All programmes standardised their infection rates and classified the surgical procedures according to the NNIS risk index [53].

The methodology employed by the NNIS system represented a significant advance in the surveillance of hospital infections because it introduced fundamental concepts in epidemiological analysis and used parameters that could be adjusted to take into account the characteristics of the populations assisted. The standardisation of criteria and the consideration of factors that caused problems in the interpretation of infection rates allowed a better comparison between institutions [3,6,54]. The NNIS method has been continuously developing and refining measures for detecting the incidence of infections in order to improve the quality indicators applied in this area. The application of the index by hospitals participating in the programme is reflected in the quality of assistance delivered to the patients [15,28,29].

As outlined above, prior to 1991, many attempts had been made to develop methods for the stratification of surgical patients according to the risk of infection, but none of these methods were specific for SSI [10,40,55]. After evaluating 84,691 surgical procedures in 44 participating hospitals, NNIS proposed a risk index, by which surgical infection rates could be stratified, that took into account differential morbidity rates and the severity of the medical state of the patient. [3,5,6,35,56]. In calculating the SSI/NNIS risk index, the three independent risk factors considered, namely, poor clinical condition of the patient (corresponding to levels 3, 4 or 5 of the ASA classification), a contaminated/infected surgical wound, and surgery longer than 2 h or longer than the cut-off point (the 75th percentile) prescribed for the type of procedure performed, are all allocated to a score of 1 (Table 4). By summation of the scores, patients are classified into four groups with scores from 0 to 3 [22,35,57,58]. This method thus allows comparison of infection rates among different hospitals, or among surgeons and surgical events in a hospital. It may also be used to detect changes that occur during a period of time, free from the influence of variation in the characteristics of the patients [35,44,45,57,58].

Applications and Limitations of the SSI/NNIS Risk Index in Clinical Practice

Gaynes et al. [35] analysed 738,398 surgical procedures carried out between 1992 and 1998 using the SSI/NNIS index and demonstrated that the method was efficient in predicting the risk of infection associated with 34 of the 44 categories investigated. Whilst the NNIS method was not adequate for combined surgeries in general, it was considered suitable for six out of the thirteen combinations of surgical procedures in which the number of risk factors increased, such as surgeries of the cardiovascular, muscle-skeletal, digestive and endocrine systems, and operations involving eyes and nose.

The general effectiveness of the NNIS method was established by Culver and co-workers, who demonstrated that the SSI risk increased from 1.5% to 13% when the NNIS risk index increased from a score of 0 to 3 [36]. A number of studies have concluded that the NNIS risk index is a good SSI predictor compared with other indices since it includes a greater number of variables rather than just the contamination potential of the surgical wound, thus providing values that are more reliable [14,59,60]. Culver et al. [61], after applying the NNIS risk index to surgeries involving revascularisation of the myocardium, could even dichotomise the population of patients based on the cut-off point for the length of the surgery.

Starling et al. [58] evaluated 64,158 patients submitted to various surgical procedures and concluded that the NNIS risk index appropriately stratifies the groups of risk when applied to the global analysis of surgical procedures. In some specific procedures (26%), the method was quite efficient allowing the stratification of the population in three or four categories. In other procedures (52%), the efficiency was reasonable with stratification of the population in two categories. However, in some procedures (22%), the efficiency of the method was very poor since the entire population was classified in just one category. The authors concluded that other specific risk factors should be included in the analysis of some surgeries (myocardium revascularisation, nefrectomy, implant of ventricular shunt, transplant and hip prosthesis), as had been suggested by Culver et al. [61]. Similarly, whilst the predictive power of the SSI/NNIS risk index in paediatric patients was considered to be suitable in general, it was suggested that the age variable should be included in the computation [62].

It is clear that the NNIS risk index has some limitations that prevent generalisation to specific populations and surgical procedures. These limitations are a consequence of the diversity and specificity of most surgeries, since the method was developed for the evaluation of a great variety of surgeries [28]. Each type of operation has its own features depending on the individuals, the co-occurring diseases and the surgical techniques employed. General risk factors are only appropriate for general surgeries, but for the purpose of comparison of patients in the same institution, the NNIS risk factor may not be suitable as an epidemiological surveillance strategy [28]. Specific variables for specific procedures must be considered in the development of alternative models of surveillance that take into account not only the possible risk factors, but also the peculiarities of different types of surgery. In this context, Gaynes et al. [35] suggested the need to introduce a further factor into the computation of the NNIS risk index associated with, for example, appendectomy, cholecystectomy, and stomach and colon surgeries, all of which potentially involve laparoscopy. Laparoscopy and endoscopy have very frequently been used since 1992 because they diminish the risk of infection in the postoperative period. In orthopaedic surgery, for example, the introduction of knee or shoulder prosthesis is facilitated through the use of laparoscopy. This risk factor has now been incorporated into the SSI/NNIS index as applied to various types of surgeries [35,53]. Thus, when other risk factors included in the NNIS method are controllable, the use of laparoscopy in a particular procedure diminishes the risk index score by 1 [35,53].

Some studies have suggested that the NNIS risk index is not suitable for craniotomy, myocardium revascularisation and cardiothoracic surgeries, since it did not precisely stratify patients submitted to clean procedures [3,56,63,64]. Similarly, Culver et al. [36] found that the NNIS risk index could not be applied in seven surgical procedures, namely, caesarean, abdominal hysterectomy, obstetric procedures, nefrectomy, prostatectomy and other procedures of the endocrine system and eyes. Furthermore, in specific surgical procedures, including nefrectomy, splenectomy and limb amputation, the infection index did not increase with the increased number of risk factors [35]. The NNIS risk index was also found to be an unsuitable predictor for vascular and digestive surgeries [28]. Similarly, following an analysis of 609 surgeries of the digestive tract conducted in Brazil, it was concluded that the NNIS index was a poor predictor of SSI risk [18]. Such apparent unreliability of the NNIS index was demonstrated in a study conducted in Bolivia by Soleto et al. [65] involving 376 general surgeries. According to these authors, there was no significant correlation between the NNIS index and the SSI rates, and it was not possible to discriminate adequately the high-risk patients from those appearing low risk levels.

According to Vernet et al. [66], however, the risk indices developed for American hospitals must be adapted before being applied to other countries. The authors emphasise that the NNIS index is a helpful tool in infection surveillance, but it must be applied considering individual situations and surgical procedures. It is thus probable that lack of adaptation is primarily responsible for the negative findings of some studies carried out in developing countries. In complete contrast, in Belgium an analysis of 16,799 surgeries in 51 hospitals revealed that the NNIS index was highly correlated with SSI rates and exhibited good predictive power [67]. Similar conclusions also derived from a study in Hungry where 5,126 patients submitted to 6,006 surgeries (colostomy, herniorraphy, appendectomy and setting of open fracture) performed in 20 public hospitals were analysed using the NNIS index. Vernet and co-workers applied the NNIS index to the evaluation of neurological surgeries performed in Barcelona, Spain and found that the introduction of a new risk factor (preoperative prophylaxis with antibiotic) for low risk patients submitted to craniotomy, increased the predictive power of NNIS index [66]. Compared with classification based on the infection potential of the surgical wound, patients could be grouped according to SSI risk more efficiently [68]. The NNIS index was considered adequate for predicting SSI risk in the University Hospital of the Federal University of Santa Catarina, Florianópolis, Brazil [14]. The simple and uncomplicated structure of the NNIS index was considered to be the main advantage and, according to these authors, this index should be adopted as a tool for epidemiological surveillance in Brazilian hospitals [14]. Both the SENIC and NNIS indices proved to be good predictors for postoperative SSI risk according to a case-control study conducted in a Spanish hospital, although the later was claimed to be more efficient [69].

Perspectives for the Use of the SSI/NNIS Index

As outlined above, there is considerable controversy concerning the use of the NNIS index for predicting SSI risk, especially with respect to some specific groups of surgical patients. Since the original development of the index in the early 1990s, new individualised indices have proposed based on multivariate analyses of potential risk factors, however, the original risk index is still used with respect to surgical procedures.

In Brazil, the NNIS index has been applied to specific procedures, such as cardiothoracic, digestive, neurological and paediatric surgeries, with controversial results. In all of these studies it was necessary to include specific variables depending on the procedure involved. This practice encourages the development of alternative models for predicting SSI, but researchers must use established methods for the development and validation of such new indices, and these must be compared to recognised indices using appropriate statistical procedures. Furthermore, the efficiency of the NNIS index should be fully revaluated through multivariate analysis before other indices are proposed by adapting the official NNIS index to local needs [46,47].

Clearly, all risk indices present limitations, and some are more applicable to certain groups of patients than others [28,66]. Although the NNIS index was considered to be of little value in some studies, it should be evaluated in further untested surgical procedures, since it is easy to apply. Furthermore, there are few studies at the national and international level that have tested the NNIS methodology, and even fewer studies have been conducted in hospitals outside the programme that have compared and validated the NNIS index. In Brazil, most hospitals do not participate in such a programme, therefore it is important to test the NNIS index in different hospitals with similar characteristics and specific type of surgeries [16].

Despite its failings, the NNIS index is still the best evaluation method for establishing SSI benchmarks [33]. With respect to orthopaedic surgeries in particular, there are no reports concerning the application of the NNIS index. Since infections resulting from this type of surgery are serious and extremely detrimental for patients and hospitals alike, the use of this risk index as a predictor is highly recommended. Only careful evaluation of the NNIS index in this speciality will determine the need for introducing new variables that may assist with the stratification of patients submitted to orthopaedic surgeries.

Received on 17 August 2006; revised 2 December 2006.

Publication Dates

History