Assessing Sustainability in Rural Water Supply Systems in Developing Countries Using a Novel Tool Based on Multi-Criteria Analysis

Abstract

Rural water supply systems (RWSS) in developing countries typically have deficiencies that threaten their sustainability. This research used Multi-Criteria Analysis and the Analytical Hierarchy Process to identify indicators that can be used to assess the sustainability of RWSS. The assessment tool developed is composed of 17 attributes with 95 quantifiable indicators. The tool enables the assessment of the sustainability of RWSS, using data collected through semi-structured interviews, social cartography, technical inspection, household surveys, and water monitoring. The tool was applied in a case study of a RWSS in the Andean region of Colombia, illustrating a participatory, holistic, and structured assessment that provided a single sustainability measure for the system (3.0/5.0). The tool’s completeness is represented by its extensive attributes and indicators that deliver a robust baseline on the state of a system, help identify improvement strategies, and monitor system performance over time that can assists rural community organizations with RWSS management.

1. Introduction

In 2015, the 2030 Agenda for Sustainable Development adopted the Sustainable Development Goals (SDGs) to end poverty, protect the planet, and ensure prosperity for all. Goal 6 seeks to ensure the availability and sustainable management of water and sanitation for all [1]. Although by 2015 over 90% of the world’s population used improved drinking water sources, those living in rural areas typically experience much lower levels of access to improved water [2].

Rural water supply systems (RWSS), especially those in developing countries, are characterized by their infrastructure and technology deficit and a lack of knowledge or experience among those operating or managing the systems [3]. Research in this field indicates that a large percentage of water projects end in premature failure [4]. A leading cause of these failures is that organizations tend to misunderstand or ignore the concept of sustainability, and community opinions are not always considered with regards to system design and management both before and after the system’s construction [5].

In the context of the water supply and sanitation sector, different authors have provided definitions of sustainable rural water supply systems [4,6,7,8,9]. In addition, different conceptual frameworks, methodologies, and tools exist to assess sustainability in RWSS, which have been categorized, classified, and compared [10]. There are a number of studies that have developed multidimensional assessments of rural water supply systems in developing countries using multi-criteria analysis, specifically the Analytical Hierarchy Process (AHP). In some of these studies, AHP is used to select suitable alternatives for water provision [11,12], whereas others focus on the assessment of water supply services at a country level [13], district level [14], the individual family level [5,15], or the collective water system level [16,17]. Molinos-Senante et al. [17] assessed and compared 40 rural water supply systems in Chile. However, the assessment was based on expert judgement and secondary information, using only a few indicators to generate recommendations at the country level. Dwivedi and Bhadauria [16] assessed 11 systems in India, but acknowledged the potential bias in the collection of information resulting from data collection instruments focused on easy to collect information that comes from a small sample size. In addition, attributes measuring system performance in rural areas—such as those related to policies, rules and norms, collective action, population characteristics, users acceptability, accountability and transparency, conflicts, risks to service provision, and environmental impact of technology—are typically not included.

This paper aims to address the above problems by developing a tool to assess the sustainability of collective RWSS in rural areas of developing countries, which are characterized by limited technical, financial, and managerial capabilities. In particular, this research aims to: (1) identify state-of-the-art indicators of sustainability in RWSS; (2) develop a tool that integrates these indicators; and (3) apply the tool to study a RWSS in Colombia. The tool is developed using Multi-criteria Analysis and allows for the integration of sustainability indicators from different dimensions in a quantitative way. Furthermore, data collection procedures are detailed that could aid community organizations, depending on their level of development and external support, to engage in data collection and analysis, formulation of action plans, and system monitoring over time.

2. Materials and Methods

This study was developed in three phases as described in the following sections.

2.1. Indicators to Assess Sustainability in Rural Water Supply Systems

A top-down approach to the identification of suitable sustainability indicators was used [18]. The Scopus database was searched using different combinations of the keywords “rural”, “sustainability”, and various permutations for “water supply” (“water system”, “water provision”, “water service”) and “assessment” (“evaluation”). The search was followed by a screening process consisting of (1) reading a paper’s title, (2) reading the abstract, and (3) reading the content to ensure the paper was related to the sustainability assessment of community-managed RWSS in a developing country. Excel^® software was then used to tabulate data from the papers identified. The indicators contained in these papers were then grouped by category and attribute. Indicators with different wording and denominations, but with the same meaning, were combined. A single indicator was formulated for each group and was linked to a qualitative value using expert judgement. In addition, data collection methods for each indicator were identified.

2.2. Development of a Tool to Assess RWSS Sustainability

From the literature review, 95 sustainability indicators were identified, which were classified into 17 attributes using Multi-Criteria Analysis (MCA). MCA provides a structured framework for analyzing decision problems characterized by multiple objectives and criteria (quantitative and qualitative), and is considered comprehensive and appropriate in assessing sustainability of water projects [19]. There are various methods of MCA, among them, Multi-Attribute Theory (MAUT) methods, in which attributes can be weighted and scored, providing a one-dimensional measure that describes important properties of highly complex cases [20]. In this research, MAUT was used to integrate the different attributes and indicators of sustainability for the system under study. Five RWSS experts met and consensually assigned weights to the indicators based on their magnitude of influence on the sustainability of water systems using direct weighting [20]. To determine weights for the attributes, the Analytic Hierarchy Process (AHP) was used. AHP is a non-linear framework which takes several factors into consideration simultaneously and allows for dependence and feedback, making numerical trade-offs to arrive at a synthesis or conclusion [21].

The weights of the different attributes (Weight of attribute—Wa) were established following the stages and procedures recommended by Saaty [21]. Thus, the overall goal of the process was defined as: “the evaluation of the sustainability of a rural water supply system”, and the attributes to be compared were obtained from the literature review. An evaluation matrix containing 17 criteria was prepared to assess the attributes according to their importance in relation to achieving the overall goal. Using this evaluation matrix, five researchers from the Water Resources and Environmental Sanitation group of the Civil Engineering School at Universidad Industrial de Santander consensually expressed their preference for one attribute over another undertaking pairwise comparisons between attributes [22]. For this, the nine point scale proposed by Saaty [21] was used. Subsequently, a normalized matrix was prepared. The values in this matrix were obtained by dividing each component of the evaluation matrix by the sum of each column in the matrix (the last row of the evaluation matrix). Then the priority vector (Eigen vector) was obtained. This vector was determined using the simple average of each row of the normalized matrix and it defines the weight that corresponds to each attribute with respect to the overall objective [21]. The final step was the assessment of the researchers’ judgement consistency. This consistency was checked using the Consistency Ratio (CR). Consistency is considered acceptable when the CR is higher than 10% [21]. The weights of the indicators (Weight of indicator—Wi) within each attribute were assigned consensually by the five researchers, according to their importance. The weights within each attribute sum to 1.0. The detailed process, matrices, and calculations underlying the tool development process are provided in the Supplementary Material.

2.3. Application of the Tool in the Sustainability Assessment of a Rural Water Supply System in a Case of Study

2.3.1. Selection of the Study Area

A RWSS in the Berlin páramo (Santander—Colombia) that provides water to 85 households was selected as a case study. The RWSS is located in an ecosystem of strategic importance, since páramos are high-mountain ecosystems that provide abundant and sustained clean fresh water for multiple uses in Andean communities and downstream urban areas [23]. However, páramos are increasingly threatened due to anthropogenic activities [24]. In the Berlin páramo, around 1500 Ha are planted with onions, with a production of 380 tons per day that provide livelihoods for around 5000 families [25]. This economic activity has led to a conflict between ecosystem protection and the livelihoods of small farmers [26]. Figure 1 shows the location of the study area.

2.3.2. Data Collection

A data collection team consisting of people from the area and researchers was established at the beginning of the study. Community members that participated were those who showed interest in the project and were involved in different community initiatives related to the RWSS. The institutional and community team members actively took part in all the data collection activities and analysis. A range of data collection strategies were implemented.

Semi-structured interviews were conducted to collect information regarding managerial aspects of the water supply system, such as organizational structure, community participation, legal and financial aspects, human and physical resources, collective action, and post-construction support. Respondents were selected based on their leadership in relation to water management and their direct experience in the construction, administration, and operation and maintenance of the system.

A social cartography was carried out with members of the fieldwork team to locate each of the system users, water sources, system components, and properties of the distribution network [27].

Technical inspections of the rural water system components were undertaken. The inspections were complemented by interviews with personnel in charge of the water system to identify technical, operational, and environmental aspects that may generate threats to the water supply [28]. Inspection forms proposed by the World Health Organization (WHO) [29] were adapted to the system infrastructure components, and the assessment was carried out with researchers and members of the fieldwork team. The information was collected through observations of the water system components, their performance status, and potential risks to water quality.

A survey was developed for the 85 households in the community that were identified through the social cartography exercise. The survey was administered by the researchers, accompanied by members of the fieldwork team, over three consecutive days. Initially, a census was planned, but only 65 households were interviewed (76% response rate), since the remaining households were not available at the time of the survey. The surveys were administered through face-to-face interviews. Household members more than 18 years old were targeted as respondents. Verbal consent was requested before conducting the survey. Information was collected on: user characteristics; uses of water and land; perception of the water service; hygiene practices and health of household members; participation in the system’s operation and management; and understanding of aspects on system administration, operation, and maintenance.

A timeline [27] was built with the participation of community leaders and people from different generations, including men, women, and seniors. This strategy helped identify the significant changes in the history of the community, the population dynamics, and the construction, operation, and management of the system, as well as difficulties the community has faced and how those were solved.

Data on water quantity were collected at the source from flow metering, while data on water quality were collected through sampling and laboratory analysis for a range of parameters suggested by WHO [30]. Sampling sites included the intake, storage tank, and most remote house in the distribution network [31,32]. Water discharge was estimated using the volumetric method. Sampling was carried out at the same time of the day, in the early morning. Samples were collected, preserved if required on ice ensuring a temperature less than 4 °C, and transported to a laboratory located a two-hour drive from the last sampling station. All samples were analyzed within 24 h of collection following APHA methods [33]. The water monitoring took place during five days in the dry season and five days in the rainy season.

2.3.3. Data Analysis

Excel^® was used to analyze quantitative data from the household survey and the water monitoring. Qualitative data from the technical inspection, semi-structured interviews, and focus groups were analyzed according to the attributes studied. Both quantitative and qualitative data provided information to assess the level of compliance with indicators that consisted of a five-point scale: fully met (5); met to a high degree (4); met acceptably (3); met to a low degree (2); and not met (1). The results on the ratings of the indicators were then multiplied by the weight assigned to each indicator and then to the weights of the corresponding attributes to determine the sustainability score for the RWSS. In addition to the single sustainability measure, the attributes that affect sustainability were scored, revealing those that performed well and poorly. These data were synthesized using a radar diagram. This radar diagram visualized the multiple dimensions of sustainability, highlighting the relative performance of each attribute and the gaps that exist between the current and ideal state.

3. Results and Discussion

The results are presented and discussed in the following three sections focusing on (1) the tool development to assess the sustainability of a RWSS, (2) the sustainability assessment of a RWSS in a páramo community, and (3) a discussion of the tool in the context of rural water system sustainability.

3.1. Tool for Assessing the Sustainability of Rural Water Supply Systems

Most of the definitions of sustainability from the water and sanitation sector in rural areas have in common the ability of the system to provide benefits over time, such as improved lifestyle and human health [4], and enhanced quantity, quality, convenience, and continuity of water supply [6]. These benefits should be provided without adverse effects on the environment [6,8,9], or other people or services [4]. Brikke and Rojas [6] expand the definition of sustainability in rural water and sanitation services to include aspects such as administration, operation and maintenance, costs, community management, and external support, which influenced the selection of indicators in this research.

3.1.1. Selection of Indicators

The literature review identified 534 characteristics of sustainability in RWSS from 34 sources. By removing the characteristics with the same meaning, but with different wording, the list was reduced to 393 characteristics. This list was further refined by choosing characteristics that explicitly identify factors that influence the long-term functionality of rural water services in developing countries as recommended by Walters and Javernick-Will [34]. Each characteristic was reformulated as a quantifiable indicator and a qualitative value judgement was proposed for each. This process led to the selection of 95 indicators. These indicators were then grouped in 17 attributes identified as relevant to characterize the sustainability of RWSS. Each of these attributes is described below.

Population Characteristics

Indicators related to population characteristics have been proposed in the literature such as population growth [35], knowledge of users related to water, health, and hygiene [3], the understanding of the system operation and management by those tasked with this responsibility [4], and user capacity to pay for water [35]. For this study, the population attribute included seven indicators that consider aspects that influence water demand and the local capacities to run the system: Percentage (%) of people that have completed high school (1.1); Population growth rate (1.2); Per capita water demand/WHO standard [36] (1.3); Percentage (%) of users who have been trained in water, sanitation, and hygiene issues (1.4); Percentage (%) of users who understand how the system is managed (1.5); Percentage (%) of users who understand how the system is operated and maintained (1.6); and Percentage (%) of users who believe they are able to pay for the water service (1.7).

Users Acceptability

The users acceptability attribute has been proposed by authors such as Barnes and Ashbolt [4]. The water system should be able to supply the community throughout the day in all seasons, since water is a vital resource [37]. Although drinking, food preparation, and personal hygiene are considered as priorities for water uses, communities tend to prefer alternatives that can also provide water for livestock and irrigation [38]. Additionally, the acceptability of the appearance, taste, and odour of drinking water should be taken into account. If the source of water fails to satisfy these conditions, users could drink from unsafe and risky sources [39]. This attribute consisted of five indicators: Percentage (%) of users who are satisfied with water quality (2.1); Percentage (%) of users who are satisfied with water quantity (2.2); Percentage (%) of users who are satisfied with service reliability (2.3); Percentage (%) of users who have not experience illness perceived to be related to water (2.4); and Percentage (%) of users who are willing to pay for the water service (2.5).

Accountability and Transparency

Accountability and transparency indicators are considered key attributes of sustainability [40,41]. The committee members of a RWSS should define and follow a set of rules related to O&M, investment, and management. However, the absence of these rules combined with the lack of mechanisms to elect committee members or call for meetings can explain why a community is unable to adequately solve their problems [42]. Four indicators were used to characterize this attribute: Existence of democratic mechanisms to choose water committee members (3.1); Number of times in the last year the water committee met users (3.2); Percentage (%) of users who know rules for access and use of the service (3.3); and Existence of mechanisms to inform users about committee finances and use of funds (3.4).

Collective Action

Participation is an important attribute of sustainability in RWSS [8,22], together with the ability of community members to work together for common goals within groups, and trust and respect among members of a group [43,44]. In this case, collective action was selected as the attribute capturing indicators dealing with community participation in different aspects of (1) system design, operation, maintenance, and administration; and (2) leadership, trust, and respect. Ten indicators were used: Percentage (%) of users who participate in administration activities (4.1); Percentage (%) of users who participate in the planning and construction of the system (4.2); Number of initiatives developed by institutions with active community participation in the last year (4.3); Percentage (%) of users who participate by contributing in cash to the system (4.4); Percentage (%) of users who participate by supporting O&M activities (4.5); Percentage (%) of users who believe woman participate in all aspects of system management (4.6); Percentage (%) of users who participate in meetings (4.7); Percentage (%) of users who believe they are listened to and their opinions are respected (4.8); Number of initiatives undertaken for the committee regarding system improvement in the last year (4.9); and Percentage (%) of users who are willing to participate in activities related to the system (4.10).

Conflicts

The conflicts attribute included three indicators related to the existence of conflicts between users, or users and institutions and the mechanisms to deal with them [5,22]: Existence of institution-human conflicts over water sources (5.1); Existence of effective conflict resolution mechanisms (5.2); and Percentage (%) of users who have not experienced conflicts with other users regarding water (5.3).

Policies, Rules, and Norms

The policies, rules, and norms attribute is related to the fulfilment of formal legal requirements [34,45]. Legislation and policies are required to define roles, responsibilities, and procedures that regulate and coordinate the activities of a water scheme, not only for access, but also regarding the rules for tariff collection and infrastructure maintenance [3]. The existence of these rules is not sufficient; mechanisms for enforcing them are also required [46]. Two indicators were considered: (Number of regulations fulfilled/Number of applicable regulations) × 100 (6.1); and Existence of water rights (6.2).

Administration, Operation, and Maintenance (AO&M)

In the initial search of indicators, 51 of 374 indicators related to aspects of AO&M, and almost all sources included at least one indicator dealing with this attribute. Essential elements required for sustained O&M include: a formal community organization that operates the system (with literate and trained members); administrative structures; tariffs; regular meetings; and strategies for community engagement [3,9]. This attribute consisted in 17 indicators: Existence of a legally registered water committee (7.1); Existence of a functional water committee (7.2); Existence of water committee bylaws (7.3); (Active water committee members/Total number of committee members) × 100 (7.4); Existence of an effective mechanism for the rotation of the water committee (7.5); (Number of water committee members trained in water management/Total number of committee members) × 100 (7.6); (Number of female water committee members/Total number of committee members) × 100 (7.7); Existence of rules for access and use of the water service (7.8); Existence of mechanisms for enforcement of rules for access and use of the water service (7.9); Existence of an office for the water committee (7.10); Existence of strategies for asset reposition (7.11); Existence of good written records (7.12); Number of effectively solved user complaints/Total number of complaints (7.13); Annual frequency of operation and maintenance activities (7.14); Existence of a functioning caretaker or maintenance committee (7.15); Number of trainings received by the caretaker or the maintenance committee in the last five years (7.16); and Existence of a salary or compensation for the caretaker or maintenance committee (7.17).

Post-Construction Support

Long-term involvement by outside groups to support community management is considered critical for system sustainability [47,48]. However, different barriers to external support programs have been reported such as insufficient time, tools, and funds to support communities [49]. Since increasing the number of experts in RWSS in developing countries is an expensive and long-term endeavor, providing training to community members in order to operate and maintain their system is recommended [50]. Despite the importance of training, rural communities may lack the technical or financial capacity to carry out complex O&M activities [51]. For this reason, external support is necessary in order to keep the system functioning over time [52]. In this study, the post-construction support attribute consisted of three indicators: Amount of money or in-kind resources received by users or a water committee from government or NGOs in the last year (8.1); Technical/administrative/financial training received by users or a water committee from government or NGOs in the last year (8.2); and Technical/administrative/financial support received by users or a water committee from government or NGOs in the last year (8.3).

Access to Water

Improved water supply has among its main benefits the reduction of time and effort to obtain water, which is associated with other factors such as increased health, education, productivity, and leisure time [35]. Access to water is highly context-dependent and is related to factors such as water service coverage [3,35], distance from the household to the water source or point [3,53], connections to a particular type of source (e.g., improved, unimproved, piped) [3]; and fetching time [46]. In this study, the access attribute consisted of two indicators: (Households provided/Total households in the village) × 100 (9.1); and (Household connections/Households provided) × 100 (9.2).

Appropriate Technology

Appropriate technology has been connected with indicators such as: the impact of demand on the operation and maintenance of the technology [3]; the local availability of parts and technology [22,38,42]; system complexity [4]; adequacy [46]; and the compatibility of a technology with geographic characteristics [5]. Five indicators were proposed for this attribute: Distance and type of road to the nearest urban center (10.1); Number of mechanical and electrical parts in the system (10.2); Number of system parts that are locally available/Number of total parts (10.3); Number of system parts that are affordable to the community/Number of total parts (10.4); and Amount of non-renewable energy required for system functioning (Kw) (10.5).

Infrastructure

Hoko and Hertle [8] stress the importance of construction quality to ensure sustainability, including meeting construction standards. In general, infrastructure can be assessed by its condition [46]; lifetime or age [42]; adequacy of design [5]; and adequacy of construction or the implementation of a construction project [34]. The infrastructure attribute consisted of nine indicators: Demand used for design/Actual demand (11.1); Number of components properly designed/Total components (11.2); Number of components properly built/Total components (11.3); Length of fencing/Required length of fencing (11.4); Number of properly designed treatment components/Total components (11.5); Number of properly built treatment components/Total components (11.6); Number of visible leaks (11.7); Number of components with an age below its estimated lifetime/Total components (11.8); and Percentage (%) of users who have working customer meters (11.9).

Reliability

Reliability is a direct measure of the functionality of a water supply system [22], and can be captured by system disruption or downtime indicators [3,47]. System breakdowns due to malfunctioning parts in a rural water supply system and deficient management can force users to use a potentially unsafe unimproved source of water [54]. Reliability can be associated with a minimum system functioning target of hours/day (e.g., 8 h/day) or downtime in hours/week [47]. This attribute comprises five indicators: Percentage (%) of users that receive water 24 h a day in the dry season (12.1); Percentage (%) of users that receive water 24 h a day in the rainy season (12.2); Percentage (%) of users that have not experienced failures in water provision in the last year (12.3); Percentage (%) of users who believe that when the service gets disrupted, it is quickly repaired (12.4); Percentage (%) of users who believe water pressure is adequate (12.5).

Water Quality

Water quality can be measured by indicators such as Escherichia coli (E. coli) counts [53], a suite of chemical and microbiological parameters [3], cleanliness of the water source [4], or water quality at the source [22]. In this study, two indicators were used to measure water quality: Percentage (%) of water samples without E. coli (13.1); and Percentage (%) of water samples with turbidity below 2 Nephelometric Turbidity Units (NTU) (13.2).

Financial Knowledge

Water committee members should have the knowledge and skills to budget for system needs [35], understand expenses [9], and manage a bank account [8,22]. This attribute was characterized by four indicators associated with the knowledge and practices of those in charge of system management: Existence of an annual budget for the system (14.1); Number of financial training sessions received in the last five years (14.2); Existence of a fund with enough resources to carry out operation and maintenance activities (14.3); and Existence of a fund with enough resources for investments in system improvements (14.4).

Funding

Indicators related to the funding of systems include the collection of tariffs or fees [55], the raising of funds to support the system as needed [8], cost-recovery [34], and debtor rate [41]. To increase the sense of system ownership, Hoko and Hertle [8] recommend at least 20% of system costs should be covered by the users. In addition, user fees should not exceed 3.5% of the monthly household income [47]. Water committees should have income for recurrent costs and savings for eventual crisis maintenance activities [47] and asset rehabilitation/replacement [41]. In this study, the funding attribute was comprised of six indicators: Existence of a fee for water service linked to the volume consumed and with charges according to socioeconomic levels (15.1); Existence of payment records (15.2); Percentage (%) of users who are up to date with payments (15.3); Percentage (%) of users for whom fees for basic supply is up to 3.5% of monthly household income (15.4); Existence of sanctions for late or non-payment (15.5); and Revenue/Budgeted needs during the last year (15.6).

Risks to Service Provision

Risks to service provision can be captured by measures such as pollution near the source and water abstraction that exceeds 40% of the available resource, compromising environmental integrity [22]. This attribute uses four indicators: Water demand during the dry season projected for 20 years/Yield at source (dry season) (16.1); Absence of human activities (e.g., agriculture, industry, human settlements, mining, etc.) with potential to pollute the water source (16.2); Absence of landslides that could threaten infrastructure components (16.3); and Percentage (%) of users that utilize alternative water sources (16.4).

Environmental Impact of Technology

Indicators dealing with the environmental impact of water systems have been considered in previous studies [3,45]. Environmental effects of a project are usually identified after implementation, however, their impacts are often not properly assessed beforehand [56]. Since part of SDG 13 is to ensure environmental sustainability [1], a water system should minimize all negative impacts on its surroundings. This attribute included seven indicators: Volume of wastewater in a year/Volume of water supplied in a year (17.1); Area occupied by the system/Volume of water supplied in a year (17.2); (Water abstracted during the dry season/Yield at source (dry season)) × 100 (17.3); Tons of solid waste produced in a year/Volume of water supplied in a year (17.4); Tons of atmospheric emissions produced in a year/Volume of water supplied in a year (17.5); Decibels generated in a year/Volume of water supplied in a year (17.6); and kW of non-renewable energy used in a year/Volume of water supplied in a year (17.7).

3.1.2. Weighting of Attributes and Indicators

As explained before, the Multi-Attribute Theory (MAUT) method was selected to interpret the identified attributes and the Analytical Hierarchy Process (AHP) was used to assign weights to the selected attributes. This approach has been used in similar studies in the field [15,16,17,22,57], and is considered useful and adaptable [46].

Table 1. Attributes and indicators to assess RWSS sustainability.

Attribute	Wa	Indicator Code	Wi	Data Collection Method	Indicator Code	Wi	Data Collection Method	Indicator Code	Wi	Data Collection Method
1. Population characteristics	0.06	1.1	0.05	HS	1.4	0.05	HS	1.7	0.10	HS
		1.2	0.20	TL	1.5	0.05	HS
		1.3	0.50	WM	1.6	0.05	HS
2. Users acceptability	0.06	2.1	0.20	HS	2.3	0.20	HS	2.5	0.20	HS
		2.2	0.35	HS	2.4	0.05	HS
3. Accountability and transparency	0.02	3.1	0.25	FG; SSI	3.3	0.25	FG; SSI; HS
		3.2	0.25	FG; SSI	3.4	0.25	FG; SSI
4. Collective action	0.06	4.1	0.05	HS; FG; SSI	4.5	0.10	HS; FG; SSI	4.9	0.05	FG; SSI; TL
		4.2	0.05	HS; FG; SSI	4.6	0.05	HS; FG; SSI	4.10	0.05	HS; FG; SSI
		4.3	0.05	FG; SSI; TL	4.7	0.20	HS; FG; SSI
		4.4	0.35	HS	4.8	0.05	HS
5. Conflicts	0.04	5.1	0.30	FG; SSI; TL	5.2	0.40	FG; SSI; TL	5.3	0.30	HS; FG; SSI
6. Policies, rules, and norms	0.02	6.1	0.30	FG; SSI	6.2	0.70	FG; SSI
7. Administration, Operation, and Maintenance	0.04	7.1	0.02	FG; SSI	7.7	0.01	FG; SSI	7.13	0.02	FG; SSI; O
		7.2	0.15	FG; SSI	7.8	0.10	FG; SSI	7.14	0.10	FG; SSI; O; SI
		7.3	0.01	FG; SSI	7.9	0.10	FG; SSI; O	7.15	0.10	FG; SSI; O; SI
		7.4	0.01	FG; SSI	7.10	0.01	FG; SSI; O	7.16	0.10	FG; SSI
		7.5	0.01	FG; SSI	7.11	0.01	FG; SSI; O; SI	7.17	0.10	FG; SSI
		7.6	0.12	FG; SSI	7.12	0.03	FG; SSI; O
8. Post-construction support	0.01	8.1	0.10	FG; SSI; O	8.2	0.70	FG; SSI; O	8.3	0.20	FG; SSI; O
9. Access	0.08	9.1	0.70	FG; SSI; SI; O; SM	9.2	0.30	HS; FG; SSI
10. Appropriateness	0.13	10.1	0.20	O	10.3	0.20	FG; SSI; SI; O	10.5	0.20	FG; SSI; SI; O
		10.2	0.20	FG; SSI; SI; O	10.4	0.20	FG; SSI; SI; O
11. Infrastructure	0.09	11.1	0.40	WM; FG; SSI	11.4	0.10	FG; SSI; SI; O	11.7	0.05	FG; SSI; SI; O
		11.2	0.10	FG; SSI; SI; O	11.5	0.05	FG; SSI; SI; O	11.8	0.10	FG; SSI; SI; O
		11.3	0.10	FG; SSI; SI; O	11.6	0.05	FG; SSI; SI; O	11.9	0.05	HS; FG; SSI
12. Reliability	0.08	12.1	0.40	HS; FG; SSI	12.3	0.05	HS; FG; SSI	12.5	0.20	HS; FG; SSI
		12.2	0.30	HS; FG; SSI	12.4	0.05	HS; FG; SSI
13. Water quality	0.14	13.1	0.80	WM	13.2	0.20	WM
14. Financial knowledge	0.02	14.1	0.20	FG; SSI; O	14.3	0.30	FG; SSI; O
		14.2	0.30	FG; SSI; O	14.4	0.20	FG; SSI; O
15. Funding	0.05	15.1	0.30	FG; SSI; O	15.3	0.05	FG; SSI; HS	15.5	0.05	FG; SSI; O
		15.2	0.05	FG; SSI; O	15.4	0.20	FG; SSI; HS	15.6	0.35	FG; SSI
16. Risks to service provision	0.09	16.1	0.50	WM	16.3	0.10	SI
		16.2	0.30	SI	16.4	0.10	HS; O
17. Environmental impact of technology	0.01	17.1	0.15	WM	17.4	0.10	SI; WM	17.7	0.10	SI; WM
		17.2	0.10	SI; WM	17.5	0.10	SI; WM
		17.3	0.40	WM	17.6	0.05	SI; WM

Notes: Wa: Weight of attribute; Wi: Weight of indicator; Wa and Wi were obtained as explained in 2.2 and in the Supplementary material. HS: Household Survey; FG: Focus Groups; SSI: Semi-structured interviews; TL: Timeline; WM: Water Monitoring; O: Observation; SI: Sanitary Inspection.

summarizes the attributes and indicators (with codes) identified to assess the sustainability of RWSS with their corresponding weights and data collection methods.

While this tool is similar to others developed using multi-criteria analysis, specifically AHP, to assess the sustainability [16] or quality of service [17] of collective rural water supply systems, the attributes in this tool were not classified into the typical social, economic, and environmental categories [19,57] using a hierarchical tree approach [20]. In contrast, this tool consists of a large number of attributes (17) to ensure that attributes of high impact are not underestimated [58]. For example, if water quality were to be located in a technical category that is assigned a lower weight than, for instance, indicators in the social category, this critical measure would be underrepresented in the analysis. Although some attributes such as quantity, quality, continuity, finances, and community participation can be found in other tools, the assigned weights are different, which is not surprising since these weights are the result of subjective evaluations [20] and are also influenced by contextual factors. However, water quality was found to be the most important attribute in this study and in studies by Dwivedi and Bhadauria [16] and Molinos-Senante et al. [17].

Despite the similarities that exist between the proposed tool in this paper and the tools developed by Dwivedi and Bhadauria [16] and Molinos-Senante et al. [17], there are a number of substantial differences. In this work, the 17 attributes are clearly defined, explained, and further subdivided in 95 quantifiable indicators. The strategies for collecting data for each indicator are suggested, allowing people interested in this topic to include these measures in their own data collection activities. Other studies in the field do not provide clear definitions of attributes/indicators (i.e., quality of service, financial governance, capacity [16]), which can be interpreted differently. Other assessment tools use fewer indicators, a number of which are context-dependent (i.e., [17]). In this case, while the use of a large number of indicators will increase the data collection efforts, they allow for a holistic analysis and provide a more universal tool that can be adapted and used across a variety of developing country contexts. The universal nature of the proposed tool is achieved by selecting indicators from an extensive literature review focused on developing countries, combined with a careful assessment of the relevance of the indicators and their availability.

To improve the tool, the weightings of indicators could take into account the opinion and expertise of a wider number of local experts including water regulators, practitioners from the water sector, RWSS managers, and researchers as performed by Dwivedi and Bhadauria [16], and by considering uncertainty during the weight calibration process [17].

3.2. Sustainability Assessment of a Rural Water Supply System in a Páramo Community

A RWSS in the Berlin páramo (Santander—Colombia) was selected to apply the tool. The system is located 3200 m above sea level in the north of the eastern range in Colombia. The climate in this region has a bimodal behavior with two rainy and two dry seasons. The WSS had 85 household water connections (49% were owners and 51% were rented), with an average number of 4 inhabitants per household, that use the system for both domestic and productive uses. The main economic activity was cropping onions, which were irrigated twice a week for eight hours during the dry season. Users were responsible for AO&M and decision-making for the system. Water committee members were a small group of men and women who organized periodical meetings due to a lack of interest by others in the community to assume these responsibilities. Additionally, the community had little trust in external institutions due to bad previous experiences. The water intake structure for the community system was built across a stream located one hour walk from the households, in an area only accessible by foot. The intake was complemented by a sedimentation tank and a concrete reservoir.

In the following sections, the results of the evaluation of each sustainability attribute for the RWSS are presented and discussed, together with the single sustainability measure obtained. The complete data with scores and their rationale can be found in the Supplementary Material.

3.2.1. Population Characteristics

In the study area, this attribute scored low (1.7/5.0, see Table S6). The most critical aspects were the low educational level of the community, where only 12% of the survey respondents had completed high school or above; inexistent training in water, sanitation, and hygiene (0%); and the relatively high population growth (4%), which together with the high per capita water demand 740 l/d, could represent a risk for water provision in the future. One positive aspect was the perception of users on their ability to pay for the water service (97% believed they were able to pay).

Table 2. Assessment of the population characteristics indicator in the study area.

Sustainability Indicator	Weight of Indicator (Wi)	Qualitative Value Judgement	Data Collection Method	Behavior in the Case Study	Score (S)	Wi*S
Percentage (%) of people that have completed high school (1.1)	0.05	The educational level of the community facilitates system functioning	HS	Only 12% of SR had a high school or above level of education.	1	0.05
Population growth rate (%) (1.2)	0.20	Population growth is not a hazard for the system capacity in the short-term	TL	Population growth was 4%, which could be an issue and a pressure factor due to the fragile páramo ecosystem.	1	0.20
Percapita water demand/World Health Organization standard (100 lpcd) [36] (1.3)	0.50	User water practices are not a hazard for the system capacity in the short-term	WM	Water demand was 740 lpcd. With increasing population growth (4%) and productive uses of water, water provision could be at risk if water management strategies are not considered.	1	0.50
Percentage (%) of users who have been trained in water, sanitation, and hygiene issues (1.4)	0.05	Users have been trained in water, sanitation, and hygiene issues	HS	0% of SR had received training in water, sanitation, and hygiene issues.	1	0.05
Percentage (%) of users who understand how the system is managed (1.5)	0.05	Users understand how the system is managed	HS	52% of SR indicated they understood how the system was managed.	3	0.15
Percentage (%) of Users who understand how the system is operated and maintained (1.6)	0.05	Users understand how the system is operated and maintained	HS	68% of SR indicated they understood how the system was operated and maintained.	4	0.20
Percentage (%) of Users who believe they are able to pay for the water service (1.7)	0.1	Users believe they are able to pay for the water service	HS	97% of SR indicated they were able to pay for the water service.	5	0.50
$\sum \left(Wi*S\right)$						1.7

Notes: HS: Household Survey; SR: Survey Respondents; TL: Timeline; WM: Water Monitoring.

provides the detailed results for the assessment of the Population characteristics attribute in the study area. Similar tables for each of the remaining 16 attributes can be found in the Supplementary Material.

3.2.2. Users Acceptability

This attribute had a high score (4.8/5.0, see Table S7), since 83% of users were satisfied with the water quantity, 89% with system reliability, and 92% had not experienced an illness perceived to be related to water. Furthermore, 95% of users were willing to pay a fee for the service. Slightly lower levels of satisfaction were found for water quality during the rainy season (60%) when compared with the dry season (80%).

3.2.3. Accountability and Transparency

This attribute scored high (4.0/5.0, see Table S8), since there was a water committee which met regularly with users, especially for decision-making; the community was informed when money was collected and how it was invested; and they knew the rules for access and use of the water service. For the last aspect, 100% of survey respondents expressed not using water from the system for irrigation. However, the demand for water (740 lpcd) suggested users were not following the water consumption rules they established (i.e., restrictions on the productive use of water).

3.2.4. Collective Action

This attribute scored 2.6/5.0 (see Table S9). The salient aspects were that people felt they were listened to and their opinions were respected (86%); people were willing to participate in activities related to the system (89%); and leaders had initiatives for system improvement. For instance, system construction was initiated by an external initiative and it was not finished. In this scenario, local leaders collected funds among the community, finished the system construction, and assumed its operation without any external support.

3.2.5. Conflicts

This attribute had a score of 2.9/5.0 (see Table S10). Although most respondents expressed an absence of conflicts between users (98%), local leaders acknowledged a latent conflict with people using water from the system for irrigation. Furthermore, human activities in the páramo and the mandate of the national government to protect paramo ecosystems [59], together with the increasing demand by downstream users of water provided by the páramo [26], posed an unresolved conflict.

3.2.6. Policies, Rules, and Norms

This attribute had a low score in the studied system (1.0/5.0, see Table S11), since the requirements set by Colombian regulations for water service providers were not met. The leaders ignored some of these requirements or the process to fulfil them, and, crucially, the system lacked the right to access water.

3.2.7. Administration, Operation, and Maintenance (AO&M)

Seventeen indicators were included in this attribute. The AO&M attribute had a medium score (2.7/5.0, see Table S12). Positive characteristics were the existence of a functional water committee, with actively involved members and with female participation. However, the committee was not legally registered and lacked bylaws. Committee members had taken over the management due to a lack of interest from others to assume responsibilities beyond performing occasional maintenance tasks or making sporadic economic contributions. Therefore, the committee members did not rotate or receive any compensation for their work, and had no training to perform their duties. The water committee lacked an office, a system for keeping records, strategies for asset reposition, and for receiving and addressing user complaints. Rules for access and use of the water service were not enforced, monitored, or sanctioned. Some 60% of respondents indicated they had participated in O&M, at least once, while around one third ignored basic information on system functioning. Due to the character of a voluntary service, the frequency of O&M was not adequate.

3.2.8. Post-Construction Support

This attribute scored low (1.2/5.0, see Table S13), since the community sporadically received financial resources from government institutions, but this has never been accompanied by technical/administrative/financial support or training. Since the community had little trust in external institutions, most respondents explained they preferred non-external intervention.

3.2.9. Access to Water

This attribute performed well (4.3/5.0, see Table S14), since 91% of premises in the community had household connections. The remaining households were not served due to pressure limitations, but individual surface sources were available.

3.2.10. Appropriate Technology

The appropriate technology attribute scored high (5.0/5.0, see Table S15), since the system was gravity-fed, it did not need any source of non-renewable energy or complex equipment or parts. Furthermore, parts were easily available from the nearest urban center, located 55 km away via a paved road.

3.2.11. Infrastructure

The infrastructure attribute had a low score (1.1/5.0, see Table S16), because several failures were identified. For instance, the intake area lacked a fence, allowing animals to enter and defecate near the source, which was not treated. Although the system was about five years old, there was evidence of deterioration. The pipeline was above the surface level and was being worn by animals, poor water drainage, and human activities. Furthermore, users’ water needs were not compatible with the design. Initially, the system was designed and built for domestic uses, but it had been repurposed for both domestic and productive activities. Thus, the infrastructure fell short and the community was forced to adapt an artisanal transmission pipe, making both the intake structure and the grit chamber ineffective.

3.2.12. Reliability

This attribute scored high (4.7/5.0, see Table S17), because 97% of survey respondents indicated the system provided water during the dry and rainy season, 24 h a day, every day (83%). Although, some people experienced failures in water provision at least once in the previous year (55%), mostly due to a pipe breakage, they stated failures were typically solved in two or less days.

3.2.13. Water Quality

Water quality scored low (1.0/5.0, see Table S18), since E. coli counts were present in 100% of samples at the source and at household taps, in both the dry and rainy seasons. Regarding Turbidity, 50% of the samples from the source and 100% from the households fell outside the quality standard in the dry season, while in the rainy season the non-compliance was 75% and 100%, respectively.

3.2.14. Financial Knowledge

This attribute scored low (1.0/5.0, see Table S19), because the water committee lacked a budget for system revenues and expenses; they ignored the total costs of providing a service (e.g., staff, water rights, taxes, O&M, depreciation, reposition); they had not received financial training; there was no money for recurring expenses, AO&M, or investments; and repairs and improvements were made with sporadic contributions. There were some remaining savings from extraordinary contributions, but those were limited since typically the exact amount of money needed for system investments were requested from users. As explained before, all work was performed by volunteers without financial compensation.

3.2.15. Funding

This attribute scored low (1.7/5.0, see Table S20), since there was no fee for the water service. Instead, money was collected through contributions when failures occurred, which was sufficient for some investments and minor repairs, but not for the improvements required by the system in terms of infrastructure and AO&M.

3.2.16. Risks to Service Provision

This attribute had a relatively high score (3.5/5.0, see Table S21). The source provided 18 l/s and the peak demand in the system was 3 l/s, which means only 17% of available water was being abstracted. However, service provision was at risk due to grazing activities around the intake, grit chamber, and storage tank. Livestock had free access to these areas, creating a potential pollution threat. In addition, in most of its length the main pipeline was above the surface level, and at some points, the network went through places where there was a landslide risk.

3.2.17. Environmental Impact of Technology

This attribute had good performance (4.7/5.0, see Table S22), because it was a gravity-fed system, occupying a small amount of land. The system only used 17% of the available flow at the source in the dry season, leaving water in the channel for downstream uses and the environment. The system did not produce any atmospheric emissions, solid waste, or noise, and did not use non-renewable energy. The only environmental impact was the generation of wastewater once a month when infrastructure components were washed.

3.3. Insights from Applying the Tool

Figure 2 provides a synthesis of the performance of the case study system in relation to each of the 17 attributes explained in Section 3.1.1 and whose results are summarized in Section 3.2. A scale of 1 to 5, with 5 being fully met and 1 being not met is used. The radar diagram allows a simple representation to simultaneously compare the results for all the attributes considered, showing the attributes with high, intermediate, and weak performance. This diagram provides an easy to understand but powerful communication strategy, which can help the community rapidly identify areas for improvement.

When addressing sustainability in RWSS, the literature has emphasized the importance of understanding challenges for different contexts [46]. The results from this study show that challenges to sustainable water provision in the assessed system are like those described elsewhere [5,9,40,53,55,60] and in Colombia [61,62] in relation to the lack of training, knowledge, and compliance with legal standards and regulations, weak finances to properly develop system administration, operation, and maintenance, and a lack of post-construction support. As in other places, productive uses of water, typical in rural areas, were ignored in the system design, which created technical and managerial problems. Although accounting for productive and domestic uses is considered a key factor for sustainability in the multiple uses of water literature [63,64], aspects of productive uses are typically excluded as attributes or indicators in the literature regarding sustainability of RWSS (e.g., Dwivedi and Bhadauria [16] and Molinos-Senante et al. [17]).

A distinctive feature of the system studied was the abundance of water, which could be threatened by population growth and increasing demand for productive activities. The poor quality of water, even in the highest elevations of the system, should be a priority at different decision-making levels. In addition, a conflict was identified between local people, the environmental authority, and downstream users who depend on water from the páramo. This conflict represents a challenge over the decisions regarding livelihoods, land use, and water resources. Disputes over divergent valuations by different stakeholders (farmers, the government, and downstream users) of the Berlin páramo, have been extensively discussed by Duarte-Abadía and Boelens [26], who argue that in relation to environmental arguments, páramo inhabitants are being marginalized and considered water polluters who threaten the ecosystem and water provision downstream.

In contrast to studies in which several RWSS are analyzed to understand or generate recommendations at the district [16] or country level [17], this research presents an in-depth case study, where data collection activities include the use of secondary information from official databases [17], RWSS records, semi-structured interviews with system managers, focus groups, technical inspections, and household surveys [16], complemented with water quantity and quality measurements. All the information was collected and analyzed with a team consisting of people from the system studied and researchers. Thus, the evaluation approach included community participation which provided spaces to discuss and identify improvements and develop action plans. This approach could lead to the appropriation of the results, mobilization of actions, and greater probability that the community will implement the changes required to improve the situation under analysis [65].

The tool could be used in a RWSS with different levels of compliance with legal and institutional aspects, and post-construction support. This represents an advantage over other tools, validated in systems that are often benefiting from government programs, and therefore ignore some key attributes (e.g., policies, rules, and norms [16] and post-construction support [17]). In contrast, many systems in Colombia are managed by community organizations that mobilize themselves [61]. Thus, these key aspects cannot be ignored when assessing the sustainability of these systems. This situation can also be found in other parts of the world. Despite the fact that the case study used for validation is not under serious water stress, the tool is able to evaluate water efficiency across different attributes (e.g., indicators 1.3, 11.1, 11.7, 11.9, 16.1, 16.4, and 17.3), which allow the tool to be used in water scarce areas. The tool can also be used in contexts similar to that of the case study (e.g., community-managed, gravity-fed systems with household connections). In general, available tools (such as the one developed in this paper) can be used to assess individual systems [5,15], with some focusing on systems accessing a specific type of water source, such as groundwater [16].

Furthermore, the tool allows a holistic assessment of sustainability since it includes attributes and indicators that sometimes are not under the direct managerial possibilities of community-based organizations (i.e., population characteristics), but have high influence on a system’s sustainability. Assessing these indicators through a participatory process could increase the understanding of local leaders and the community on the performance of the RWSS from a systemic perspective. Consideration of these aspects generates evidence that could mobilize the community to pursue strategies to improve different dimensions of a community’s quality of life, which would include improving the service provided by the RWSS. This community-based approach is crucial, since in many cases RWSS managers also lead other development projects in their community related to education, health, production, etc. These other projects can directly impact the sustainability of a RWSS, so using a tool that can help communities advance a more holistic approach to their development is likely to result in more impactful results than those that can be achieved when using tools that have a more limited scope. In addition, the tool advanced in this paper includes indicators associated with multiple-use water services (MUS), which is an approach to rural water planning that is increasingly associated with improved rural water system sustainability [64].

Finally, the tool incorporates attributes such as conflicts, risks to service provision, and the environmental impact of technology, which have been overlooked in comparable tools (e.g., [16,17]), but are critical when considering increasing pressures associated with climate change, water scarcity, and conflicts between users, communities upstream/downstream, and water providers and regulators [66].

This tool can be used to assess different dimensions of RWSS sustainability, especially for gravity-fed systems with household connections that are managed by participatory forms of governance. The assessment process and results could provide communities with a baseline against which they can identify opportunities to enhance their system and benchmark continuous improvement efforts. The proposed tool can also promote collective action, where a community could mobilize its own resources to address the problems identified. In addition, at the time of an evaluation, efforts can be made to connect a community with external institutions that may be able to provide financial resources for repairs that fall beyond what a community can afford. These external institutions may also be able to support a community through capacity development.

4. Conclusions

Despite decades of research on the sustainability of rural water supply systems, maintaining a reliable water supply remains a development challenge in rural, remote, and marginalized communities in developing countries.

Most studies assessing sustainability in rural water supply systems use a low number of indicators, presumably for pragmatism, the costs associated with data collection, or the use of indicators that are considered proxies to assess attributes. The attributes and indicators selected for the proposed assessment tool were found to be critical to the measurement of sustainability, collectable, and relevant for decision making. The tool developed provides a holistic way to assess the sustainability of rural water supply systems in a local context.

Most MCA studies have emphasized assessments and comparisons across different systems. In the analysis of the RWSS, the quantitative scale of sustainability and the empirical results of the assessment, when shared with community members proved to be a useful communication tool on system sustainability. In addition, the tool can be used to monitor the status of the scheme and as a reference point for decisions. The tool can also be used in similar contexts, providing a baseline for administrators to holistically assess the state of a system, devise improvements, and monitor behavior over time. The tool can help community providers prioritize their actions and investments, look for support for aspects beyond their immediate capabilities, and self-mobilize for improvements that can be performed without external support.

From a policy and managerial perspective, the results show the importance of integrating multiple attributes into a single dimension when assessing the sustainability of RWSS and how collaborating closely with a community can help ensure the findings can be linked with actionable system improvements. Developing studies comparing systems at a district or country level could also help governmental agencies devise policies and identify post-construction support strategies based on the needs of RWSS managed by community based-organizations.

The tool could be improved by establishing thresholds or benchmarks for the indicators. Furthermore, community members could assign weights to attributes, which can result in greater appropriation of the results and mobilisation for actions that generate positive changes in the systems assessed.