Teaching Multi-Criteria Decision Making Based on Sustainability Factors Applied to Road Projects

Abstract

Currently, there is a need for civil engineering programs to train their students in subjects associated with sustainability. Additionally, civil engineers in their work must constantly make decisions, so their training is necessary. Therefore, the goal of this research is to present a methodology for teaching multi-criteria decision-making methods in the context of civil engineering and road infrastructure projects using sustainable factors. To achieve the objective of this study, a decision-making simulation activity has been designed based on a five-step research process: (1) definition and contextualization of the case study; (2) design and planning of the simulation activity; (3) implementation of this activity; (4) evaluation of indicators; and (5) statistical analysis of metrics. The teaching methodology used is of a practical-theoretical type and allows for the step-by-step teaching of three multi-criteria decision making (MCDM) methods that, according to the literature review, are widely used in the architecture, engineering, and construction (AEC) industry. This teaching activity is intended for undergraduate students and professionals in the AEC industry who require a decision-making tool that promotes transparency in problem-solving and who have no knowledge of MCDM. The results obtained in this research show that the method perceived by the group of students with the least difficulty was the weighting rating and calculating (WRC), because of its previous familiarity and use in academic environments. It is recommended that MCDM methods be taught in semester courses to students in training so that they can develop a deep understanding of these tools and can demonstrate their usefulness for decision making where there are many variables to consider, where there are many decision-makers, and for the incorporation of sustainable factors for project evaluation.

1. Introduction

The civil infrastructure contributes to progress and improves the quality of life of people; however, it is also an industry that generates destruction on the environment and nature. On the one hand, infrastructure represents an engine for the development and progress of countries, promoting economic growth and improving the quality of life of people, as well as contributing to reducing poverty, marginalization, and increasing competitiveness [1]. On the other hand, construction is one of the industries that most destroys nature because of the high need for materials, space to construct, and energy to supply both the construction process and the operation of the infrastructure [2,3]. It seems evident that this is the reason why it is not possible to stop construction in order to stop the negative effects that these projects may cause to the environment and society. However, it is possible to seek to integrate sustainability criteria in order to reduce the impacts of infrastructure projects [4,5,6].

Sustainable criteria in civil infrastructure projects can be incorporated through the use of multi-criteria decision-making methods [7,8,9,10,11]. The decision-making process has been defined as the study of identifying and choosing alternatives based on the values and preferences of the decision making person. This process allows one to visualize the problem with many features and make a choice through a process that is as transparent as possible, which allows one to address the uncertainty of the problem-solving process [12,13]. In order to identify each of these features of the problem, it is necessary to carry out a process of research and collection of useful information to facilitate the decision-making process and to properly determine the preferences of the decision-makers. Multi-criteria decision making (MCDM) is a research and operations management field that allows for decisions to be made considering multiple criteria, which may be opposed to each other, and to address the different perspectives and preferences of decision-makers. Multi-criteria methodologies allow the combination of different dimensions, objectives, actors, and scales involved in the decision-making process without sacrificing quality, reliability, and consensus in the results [14].

In the civil engineering profession, civil engineers are systematically exposed to decision making during the development of infrastructure projects, considering their whole life cycle. It is not surprising that decision-making methods have gained an important field in the training of future civil engineers, increasing their applications in the practice of the profession and increasing their use in research associated with the areas of civil engineering. This is demonstrated by the publication of [15,16], where they declare that the use of MCDM has increased steadily since 1990 and determined that civil engineering ranks tenth in research associated with MCDM in the areas of construction building technology, transportation, water resources and other topics in engineering. In turn, the application of MCDM methods associated with sustainable decision-making in civil engineering has increased, specifically in the areas of sustainable construction and construction technology [17].

Globally, a wide range of methodologies and systems have been developed to evaluate the performance and sustainability of transport infrastructure projects such as the Greenroads Rating System, GreenLITES, BE2ST-in-Highways, INVEST, ILAST Rating System, STEED y STARS. In view of this, Chile, one of the areas that seeks to incorporate sustainability criteria in all stages of the life cycle, are undertaking road projects by updating the design manual of road works (Road Manual). By the year 2017, Chilean infrastructure consisted of approximately 80,500 km, whose administration resides in a public entity [18], and this number has been increasing. The reason for this is that every year a large amount of government resources is destined to the construction of pavement networks that make it possible to increase connectivity, improve the quality of life of the people and local development. In addition to other purposes, such as reducing transportation costs, improving supply and transport networks of products.

Currently, there is a knowledge gap in the curricula of the training programs for professionals in the areas of architecture, engineering, and construction (AEC). This gap consists of the fact that currently university curricula in the area of construction do not cover sustainability concepts, according to a study carried out by Pellicer, Sierra, and Yepes in 2015, where they have collected several studies that raise the need to incorporate sustainable education issues to future professionals in the area and other studies where the degree of understanding of these issues by students is evaluated. They have concluded that there is still room for improvement regarding active learning methods that value multiple criteria regarding sustainability, particularly those focused on construction [19]. Additionally, there is a need for multi-criteria decision-making methods to be taught at the undergraduate level, that have historically only been taught at the graduate level, according to a study carried out by Navarro, Martí and Yepes in 2018, where they conducted a review of the main methodologies for multi-criteria decision-making that are taught in the course “Predictive and optimization models for concrete structures” offered at the Masters in Concrete Engineering of the Universitat Politècnica de València [20]. Therefore, the objective of this research is to present a methodology for teaching multi-criteria decision-making methods in the context of civil engineering and road infrastructure projects using sustainable factors.

2. Background

2.1. Multi-Criteria Decision Making

The multi-criteria decision making (MCDM) methods correspond to a research and operations management field that allows decisions to be made based on multiple criteria, which can be opposed to each other, and to address the different points of view and preferences of decision-makers. There are several MCDM methods, among them the most used in engineering are the analytic hierarchy process (AHP), preference ranking organization method for enrichment evaluation (PROMETHEE), elimination et choice translating reality (ELECTRE), analytic network process (ANP) and the technique for order of preference by similarity to ideal solution (TOPSIS), among others [7,15,20,21].

From the existing MCDMs, the following were selected for this work: the weighting rating and calculating (WRC) because of its extended use in the AEC design practice [10]; choosing by advantage (CBA) since, according to Arroyo [22] it allows for the creation of transparency and consensus, better than that of other methods and the analytic hierarchy process (AHP) because of its prevalence in the AEC decision-making literature. The particularity of each methodology lies in the way it transforms the measurements and perceptions of the group of experts into a unique and quantifiable scale.

2.1.1. Weighting Rating and Calculating (WRC)

WRC is a method of decision making, also known as weighted summation, and has been widely employed in architectural, engineering, and construction industries [23]. An example of this is the study conducted by Sabapathy and Maithel (2013), which used WRC to create a ranking of wall material choices in India, it was also used for the selection of subcontractors in the construction industry [24], and Arroyo has used it to make comparisons with the CBA method [22,25]. To apply the WRC method, it is necessary to know the definitions of the following concepts [10,26]:

• Factors: An element, part, or component of a decision. When assessing sustainability, factors should represent economic, social, and environmental aspects.
• Criterion: A decision rule or a guideline. A “must” criterion represents conditions each alternative must satisfy. A ”want” criterion represents the preferences of one or multiple decision-makers. This guide is similar to the one proposed by the Kepner-Tregoe decision analysis technique [27].
• In its methodology, the first step is to identify the selection alternatives from which to choose. Second, you must identify the factors and evaluation criteria for these alternatives, which are key for the decision-maker in the process of making his or her decision. Third, the factors should be given weights (wi) directly; these indicate the importance of each factor to the decision-maker and should be expressed as a percentage. As a restriction, the sum of the weights should be 100%. Fourth, the alternatives should be rated for each factor (ui); this rating will depend on the characteristics of each alternative, and a verbal scale with an associated score can be used to facilitate the choice of rating. In the last step, the value of each alternative is calculated as follows: (∑ wi x ui), where wi is the weight of the factors, and ui is the factor score for each alternative [10,24]. Figure 1 shows the structure of the WRC method.

2.1.2. Choosing by Advantage (CBA)

The CBA method was proposed by Jim Suhr around the 1970s and is based on the advantages of the alternatives and the importance of these advantages. According to Arroyo [28], the method has been used in the Lean community, but very few applications with an environmental perspective can be observed. Some exceptions are, for example, Grant and Jones [29], who used the method to select green roof designs, and Arroyo, who has used the method on several occasions with a sustainable approach [10,23,30,31,32,33,34]. To apply the CBA method, it is necessary to know the definitions of the following concepts:

• Alternative corresponds to two or more people, items, or plans from which one will be chosen. It can also be understood as the possible options or actions available and from which one wants to choose.
• Factors are elements, parts, or components of a decision, those that contain data necessary to make a decision, and that affect the preference of the alternatives.
• Attributes are the characteristics, quantity, or quality of an alternative
• Criteria correspond to standards, rules, tests (trials), or any standard on which a desired or required decision or judgment can be based
• The advantage is the benefit, gain, or improvement, the difference between the attributes of two alternatives.

The CBA methodology consists, in the first step, of identifying the alternatives of selection, which must have remarkable advantages over each other. Second, the factors to be evaluated are identified, which are key to the decision-maker. Third, the desired or required (mandatory) criteria for each factor must be defined to guide the decision-making process. Fourth, the attributes of each alternative must be summarized for each factor; that is, research must be conducted to identify how the alternatives behave and what their characteristics are. Fifth, the advantages of each alternative with respect to the alternative that presents the most unfavorable attribute must be defined according to the defined criteria. Sixth, the importance of each advantage must be decided, on a scale from 0 to 100, with 100 being the most important; this is a subjective stage since it depends on value judgments. Finally, the cost of each alternative and its respective total importance must be evaluated, which corresponds to the sum of the importance assigned in each factor, and thus the decision must be made [10,24]. Figure 2 shows the structure of the CBA method.

2.1.3. Analytic Hierarchy Process (AHP)

The AHP methodology was proposed in 1980 by Thomas Saaty and consists of a multi-criteria decision-making system, which is based on the judgment of a group of experts in a certain area who contribute their subjective knowledge consisting of paired comparisons of attributes, finally providing a relative measure between the options that are evaluated in the decision-making process [35]. According to Sierra [36], the AHP method has been widely used in the construction field and other areas; it is used for the assessment of the sustainability of infrastructures.

The methodology of the AHP process is represented in Figure 3 and consists of five fundamental steps that begin with the definition of the problem, clearly identifying the objective or decision to be made and the different alternatives and criteria. Second, the hierarchy must be built with various levels, which is headed by the objective, the next level contains the fundamental criteria (factors) that will be taken into account for the evaluation, and there may be more levels with sub-criteria until the last level is reached, which contains the decision elements from which to select, or alternatives [37]. The method is based on the paired comparison of elements of the same level, assigning a score according to the use of a rating scale, as proposed by Saaty, shown in

Table 1. The scale of assessment proposed by Saaty in 1990. Adapted from: [36].

Value	Short Definition	Definition
1	Equal importance	Criterion A is equally important as criterion B
3	Moderate importance	Experience and judgment slightly favor criterion A over criterion B
5	Strong importance	Experience and judgment strongly favor criterion A over criterion B
7	Very strong importance	Criterion A is much more important than criterion B
9	Extreme importance	The greatest importance of criterion A over criterion B is beyond doubt

. The third step is achieved by comparing the elements, based on expert judgment, and constructing matrices of judgments of each criterion for the different alternatives, and the elements of the matrix are defined according to the importance that one element has with respect to another. Fourth, the ratio scale or priority vector of each criterion must be calculated by normalizing the judgment matrices and then obtaining the average value of the values of each row of the matrix. Fifth, the value of each alternative is calculated by the weighted sum of the priority vector of each criterion with respect to the different alternatives and the priority vector when comparing all the criteria with each other (which defines the importance of one criterion with respect to another).

2.2. Sustainability Factors

The concept of sustainability was defined in the Brundtland Report, as the ability to meet the needs of the present without compromising the ability of future generations to meet their own needs [38]. There are other definitions of this concept, such as “the integration of economic growth and environmental protection to meet the needs of the present without compromising the well-being of future generations”—World Business Council on Sustainable Development (1992) [39]. The most complete conceptualization of this concept applied to development is the one that was presented at the United Nations Conference of Rio de Janeiro, 1992 in which “sustainable development is defined as the style of development that seeks to achieve the satisfaction of the fundamental needs of the entire population, through rational management of natural resources, promoting their conservation, recovery, improvement, and appropriate use, so that both this generation and future ones have the possibility of enjoying them with physical and psychological balance, thus ensuring their quality of life and the survival of human beings and the planet”. [39]. Thus, sustainability can be understood as the balance of three fundamental dimensions: the environment, society, and the economy, as represented in Figure 4.

The efforts of the United Nations to develop a global action plan for sustainable development have become a unifying theme for governments, organizations, and businesses [42] to achieve the adoption of this new development model, which encompasses all areas of industry, including the AEC. Several studies quantify the impacts of the construction area on the planet. It is estimated that construction uses approximately half of the resources that man consumes from nature [3], and it is considered that 25% of the world’s waste is construction and demolition waste [43]. The construction and operation of buildings represented the majority of global energy use in 2019 (36%) and energy-related CO₂ emissions (39%) [44]. In addition, about 50% of global resources are consumed by the construction industry [45], of which, 40% of the world’s primary materials, equivalent to three billion tons per year, are used in construction [2], between 13 and 20% of the world’s drinking water [22,46,47] and 10% of the Earth’s surface [46].

If current patterns remain unchanged, construction expansion will destroy or at least disturb natural habitats and wildlife on more than 70% of the Earth’s surface by 2032, mainly due to increased population, economic activity, and urbanization [48]. For this reason, there has been an awareness of the need to follow an agenda oriented towards achieving sustainability, and tools have been created to evaluate and qualify the degree of sustainability in civil infrastructure. Among them are the rating systems ENVISION from the USA, the Civil Engineering Environmental Quality (CEEQUAL) from the UK, and the Infrastructure Sustainability (IS) Rating Tool from Australia.

On the other hand, efforts to incorporate sustainability indicators in the evaluation of road projects have been increasing in recent years. Currently, there are certification systems and web platforms for rating good practices, for example, the Greenroads Rating System, GreenLITES, INVEST, and ILAST Rating System. In addition, many researchers propose various ways of incorporating sustainability into paving projects such as Echaveguren [49], Chamorro [50], Ordoñez and Meneses [51], among others. To realize the holistic sustainable management of transportation system projects, more than the three pillars of sustainability shown in Figure 4 must be considered. During the project life cycle, technical, economic, environmental, social, financial, political, and institutional aspects must be considered together [52], as is shown in Figure 5.

To use the decision-making methods, it is necessary to establish the criteria and evaluation factors that will be used. In this research, a list of factors in the three dimensions of sustainability has been considered, which can be seen in Figure 4. This list of sustainable factors was specifically defined for the evaluation of road projects, based on bibliographical studies and the opinion of a group of experts in the area of engineering, construction, and academia [53]. The list of sustainable factors used for this research is presented in

Table 2. Environmental factors for the evaluation of basic road projects. Adapted from [53].

Factors	Description	Aspects to Be Evaluated and Indicators
Ecosystem area affected by the project	Quantify the total area of ecosystems with high ecological value affected by the project.	Number of hectares of protected ecosystems and habitats or of high ecological value.
Endangered species living in the area	Identify natural habitats with endangered species along the road.	Intervention surface of native forest and/or protected species habitat.
Use of materials from the area	Selection and use of local materials to minimize the impact on the environment of transporting materials. Moreover, contribute to income sources for the local economy.	Percentage of materials coming from the area.

,

Table 3. Social factors for the evaluation of basic road projects. Adapted from [53].

Factors	Description	Aspects to Be Evaluated and Indicators
Improve the quality of life of the community	The project provides significant benefits to the affected communities that reduce or eliminate negative impacts. That is, it has a net positive impact, considering aspects such as health and safety of people.	-Number of people living near the road who have health problems. -Number of ppm of dust in suspension along the road.
Beneficiary population/population adjacent to the road	Quantify the number of people who benefit from conserving the road and those who live near the road. To determine the impacts generated in the community.	-Number of residents who benefit from the road (connectivity). -Concentration of neighboring population that benefits (dispersed, medium, concentrated). -Number of residents who live adjacent to the road (less than 100 m from the road).
Community organization level	Weighing the political influence that the community applies.	Level of organization and degree of the political activity of the benefited community (high, medium, or low).
Improve accessibility, road safety, and signaling on construction sites	Maintenance of acceptable levels of safety for road users, be they drivers, passengers, cyclists and/or pedestrians.	-Number of events per year in which the road is flooded or muddy. -Number of accidents per year registered on the road.
Improve community access and mobility	The location of the project improves community access and mobility between inland locations and/or brings people closer to the city and, therefore to health, education and commercial services.	- The road allows access to other interior villages -The road allows connection to urban centers and basic public services.
Promote the use of public transport	Improve the accessibility of public transport, contributing to the mobility of the community.	In case the improvement of the road allows access to public transport.
Presence of health service and/or historical place or cultural interest	Presence of places with high value for the community.	Presence of religious temple, headquarters, clinic, historical places and/or cultural interest in the road or adjacent sector to the route.
Presence of historical, cultural and/or archaeological heritage	Preserve historical and cultural sites important to the community.	The number of places of interest per kilometer of the route.

and

Table 4. Economic factors for the evaluation of basic road projects. Adapted from [53].

Factors	Description	Aspects to Be Evaluated and Indicators
Daily vehicle flow	Number of vehicles on the route.	Level of daily vehicular flow along the route (low, regular, medium, or high)
Periodicity of maintenance and replenishment of the route	Decrease the maintenance and material replacement work involved in preserving the route.	- Time interval between route maintenance. -Time interval between replacement of material in the route.
Typical road use before upgrading/associated travel time	It seeks to determine the use that users give to the route, such as trips to the city to access public services, agricultural transport, among others.	The number of users using the route according to the type of user (pedestrian, public transport passenger, cyclist, driver).
External monetary contributions	Consider the possible economic contribution of outsiders for the realization of the conservation project.	Monetary percentage of the investment as an external contribution to the project.
Local economic assessment/impact on local productivity	Quantify the economic impact of road paving in terms of increased connectivity to natural resource areas.	Travel time with the project (improved road) by the number of trucks on the road.
Predominant economic activities in the area	To value the possible economic impacts that the paving of the road has on the local economy.	Define the main activity of the area (agriculture, livestock, forestry, tourism, mining, urban, among others)
Affected farming area	Decrease the impacts that the suspended particles have on the surrounding farms.	Number of hectares of farms surrounding the route (less than 300 m)
Increased connectivity between communities	Determine the number of connected communities	The number of communities connected.
Ratio of local material use	Decrease the transport of materials to optimize the resources allocated to the project.	Distance to material production facilities.
Type of carpet	To value the impact according to the type of carpet that exists before the conservation.	Type of carpet before upgrading soil or gravel.
Road geometry	Identify the type of geometry that exists on the route to identify the level of difficulty of unpaved road traffic.	Classification of the route (flat, wavy, mountainous, with slopes, among others).
Average road width	Determine the width of the route to be paved to know if it complies with safety ranges.	The width of the roadway to be paved is less or more than 5 m.
Degree of consolidation of sanitation	Identify the degree of sanitation that the route has so that those that already have sanitation are more economical projects.	The presence of sanitation works on the route, the degree can be zero, medium, or high.
Network connected to the road network	The route to be conserved is linked to the road network of the state/country.	Membership of the route to the communal network enrolled in the road network (Yes/No answer).

.

3. Research Process

To achieve the objective of this study, a decision-making simulation activity was designed based on a five-step research process: (1) definition and contextualization of the case study; (2) design and planning of the simulation activity; (3) implementation of this activity; (4) evaluation of indicators; and finally (5) statistical analysis of metrics.

3.1. Contextualization

The simulation activity was carried out in the course “Highway Engineering” during the second semester of the 2018 academic year corresponding to the Civil Engineering degree of the Pontifical Catholic University of Valparaiso, Chile. Highway Engineering is a fourth-year course of the 6 years of the career with an academic load of 3 h of lecture classes per week, 1.5 h of workshop activities, and 4.5 h of personal study. The course curriculum covers the structural design of pavements, considers design and construction methods of asphalt pavements, asphalt treatments, and concrete pavements. A total of 53 students were registered in this class.

In the context where students were learning and studying the design of pavements, they were also required to have notions about the evaluation of road projects, methods of decision making, and how to incorporate sustainable indicators for their evaluation. This activity was voluntary, and the workshop schedule was used to carry it out in sessions of 90 min each.

3.2. Design and Planning of Simulation Activity

We designed the procedure of the simulation experiment for decision making, trying to compare the three selected multi-criteria methods and identified the one with the greatest learning advantages. In order for students to visualize how to use MCMD, they chose three real road projects of similar characteristics whose information was obtained from a virtual platform where public institutions in Chile bid for projects to be executed with government funding. Three basic road projects were selected for this case study, and students were given a summary of each one.

Basic road projects are a special type of conservation of soil or gravel roads to avoid the construction-conservation-reposition cycle to which these roads were subjected. This cycle consists of conservation due to the loss of cohesion and replacement due to the loss of material and reduction in the thickness of the roadbed [18]. The Basic Roads Program is characterized by a high restriction of resources and low modification of the existing condition of the route. In addition, they represent benefits such as increasing the years of the useful life of the route, thus providing greater road safety, achieving growth with equity by triggering local development and increasing the value of the properties, contributing to the development of the country, overcoming poverty, and supporting the productive rural economy [54], and finally, to eliminate negative effects on people’s health and generate sustainable development by reducing environmental pollution (by eliminating dust).

We looked to create a system for prioritization of basic road projects based on sustainable factor-based decision-making methods. Prioritizing these road projects aims to designate annual public resources in the most objective way possible. Therefore, the following basic road projects have been considered for evaluation in this research:

• Project 1—Basic Road for Conservation (CBC) Route T—308, Valdivia, Chile: Conservation project through the application of a primer and asphalt seal type Cape Seal (simple asphalt surface treatment plus asphalt grout) of the 6.47 km route with 6 m wide roadway. The route benefits the inhabitants of Chancoyán, a rural sector of the commune of Valdivia, whose population is approximately 60 people.
• Project 2—Basic Road for Conservation, Route F—348, Valparaiso, Chile: It contemplates the execution of simple surface treatment, sanitation, and road safety works on the 3.32km long route with a 5 m wide roadway. The route benefits mainly the population adjacent to the route, who live mainly in plots and farms with areas of crops that can be affected, as well as the health of people, by the current condition of the route, which shows high concentrations of suspended material by vehicle traffic.
• Project 3—Basic Road to Curipaillaco, SR Route, Tirúa, Chile: The project considers a double surface treatment with polymer on a bidirectional roadway, cleaning of the strip, cutting excavation, earth moving, road safety works, road demarcation, and sanitation works. The route to be treated has a length of 5.17 km and presents several sections with strong slope and closed curves; the folder of rolling is of granular material, is in regular conditions, and has an average width of 5.5 m. The route benefits mainly the population adjacent to the route, who live mainly in large plots and farms with areas of cultivation and, besides, there are areas of native vegetation with high ecological value.

3.3. Application of Learning Activities

The simulation was implemented during the workshop hours of the Highway Engineering course, in which the context of the present research was explained, and then the steps shown in Figure 6 were followed. First, a summary sheet was handed out for each working group, containing the main characteristics of basic road projects that are currently underbidding or were recently awarded. These characteristics are based mainly on technical aspects, and economic, social, and environmental aspects can be inferred or deduced from each one, and based on these aspects, participants must evaluate and make their decision. Attached to this sheet is a step-by-step explanation of each of the decision-making methods, so secondly, a preliminary explanation of each multi-criteria method, how they work, and what are the relevant steps in which they must decide.

Then the participants must make the first decision as a group, so they are presented with the WRC method and an Excel spreadsheet where they find the tables they must fill in to evaluate each project according to environmental, social, and economic factors. Through a process of discussion, where each one presents their opinion, they must reach a group consensus about the project selection and, once this discussion stage is concluded, they proceed to measure the opinion indicators on the WRC method that they have just used. Participants are then introduced to the second multi-criteria method for decision-making called CBA. At this point, they are explained with the corresponding Excel spreadsheet, how the method works, and the boxes that must be modified to make the decisions. The process mentioned above is repeated, where group consensus is reached, and then the indicators are measured. Then, the participants are presented with the latest multi-criteria method for decision making called AHP. They are again presented with the Excel spreadsheet, where they must make the matrix comparison corresponding to this method. When the participants have decided, they are told that they must reach a group consensus about this selection.

Finally, the Excel spreadsheets are compiled with all the group decisions, and the participants give their opinions and final comments about the experience they have just carried out, the proposed methods, etc.

3.3.1. Example Use WRC Method

To make the first decision, the WRC method must be used, for which the Excel spreadsheet must be completed in a similar way to the example presented in

Table 5. Example WRC method.

Factors	Weight Factor	Project 1	Score 1	Project 2	Score 2	Project 3	Score 3
		Qualification		Qualification		Qualification
Ecosystem area affected by the project	0.3	Very good	10	Good	8	Good	8
Endangered species living in the area	0.4	Regular	6	Very good	10	Good	8
Use of materials from the area	0.3	Deficient	4	Regular	6	Regular	6
Total Score	100%	-	6.6	-	8.2	-	7.4

. The qualification column corresponds to a verbal scale to qualify the characteristics of each project associated with each factor. This scale is associated with a score according to the values shown in

Table 6. Rate alternatives for each factor.

Qualification	Very Deficient		Deficient		Regular		Good		Very Good
Score	1	2	3	4	5	6	7	8	9	10

, which will be determined by consensus by the decision-makers.

3.3.2. Example Using CBA Method

To make the second decision, the CBA method must be used, for which the Excel spreadsheet must be completed in a way similar to the example presented in

Table 7. Example CBA method.

Factor	Criteria		Project 1		Project 2		Project 3
Ecosystem area affected by the project	The lower the number of hectares of protected ecosystems and habitats or of high ecological value, the better	Attribute	MB		B		B
		Advantage	Id:1	Project far from protected ecosystems	Id:2	-	Id:3	-
		Imp	80		-		-
Endangered species living in the area	The smaller the area of native forest and/or protected species habitat, the better	Attribute	R		MB		B
		Advantage	Id:4	-	Id:5	There are no nearby areas	Id:6	There are areas a few km
		Imp	-		80		60
Total Importance			80		80		60
Total Project Cost			$636M		$259M		$772M

. Where the attribute box corresponds to defining, from the verbal scale presented in Table 6, the attributes of each project. While the advantage of an attribute concerning the less desired attribute and the Imp box must decide the importance of all the advantages previously defined and must be justified qualitatively in the advantage box. For this purpose, the values presented in

Table 8. Value of importance of each advantage.

Importance	100	90	80	70	60	50	40	30	20	10	0
Identification (Id)			1–5		6

should be used.

To make the final decision on which project to select, the total importance of each project versus the total cost of each project can be plotted so as to visualize which one has greater advantages over the others.

3.3.3. Example Using AHP Method

To make the third decision, the AHP method must be used. Students are given the model of the hierarchy with the decision-making criteria, as shown in Figure 7.

Then, comparisons are made, according to the values shown in Table 1, between elements of the same level for the construction of the trial matrices.

Table 9. Judgment matrix and average vector.

Sub-Criteria A1: Ecosystem Area Affected by the Project
	P1	P2	P3	Standardized matrix			Average Vector
P1	1	7	5	0.74	0.78	0.71	0.75
P2	1/7	1	1	0.11	0.11	0.14	0.12
P3	1/5	1	1	0.15	0.11	0.14	0.13
Total	1.34	9	7
Sub-Criteria A2: Endangered Species Living in The Area
	P1	P2	P3	Standardized matrix			Average Vector
P1	1	1/7	1/5	0.08	0.1	0.05	0.07
P2	7	1	3	0.54	0.68	0.71	0.64
P3	5	1/3	1	0.38	0.23	0.24	0.28
Total	13	1.48	4.2
Criteria: Environmental Dimension
	A1	A2		Standardized matrix			Average Vector
A1	1	3		0.75	0.75		0.75
A2	1/3	1		0.25	0.25		0.25
Total	1.33	4

shows the trial matrix on the right side, where the row corresponds to alternative A and the columns to alternative B. Thus, in the box P1, P2, it has been determined that project 1 has “very strong importance” (value seven) concerning project 2. Therefore, the box P2, P1, is filled with the multiplicative inverse of this value, leaving 1/7.During the decision-making simulation, students had to make comparisons between alternatives (projects) with respect to each sub-criteria, comparisons between sub-criteria (list of factors presented in Table 2, Table 3 and Table 4) concerning the respective sustainability dimension, and comparison between criteria (sustainability dimension). As an example, two comparisons between alternatives with respect to two sub-criteria (A1 and A2) of the environmental dimension and a comparison between these same sub-criteria have been presented in Table 9.

Once the comparisons are made, the average vectors are calculated for each alternative and for each sub-criteria. It is worth mentioning that in the simulation experiment, the Excel spreadsheet was delivered programmed so that the matrices and vectors were self-completed after the comparisons were made. As an example, Table 9 shows the calculation of the average vector obtained from the average of the values of the standardized matrix row. The standardized matrix equals the values of the trial matrix divided by the total values (sum of the columns).

From these matrices, the value of each project alternative can be calculated as the weighted sum of the average vectors, as presented in

Table 10. Value of each project alternative.

Weighting Factor	0.75	0.25
Alternatives	A1	A2	Total
Project 1	0.75	0.07	0.58
Project 2	0.12	0.64	0.25
Project 3	0.13	0.28	0.17

, where it has been expressed as a weighting factor to the average vector of the environmental dimension sub-criteria.

This total value per project should be multiplied by an environmental dimension weighting factor corresponding to the average vector obtained when comparing the three dimensions of sustainability. In this way, the final rating of the projects with respect to the environmental component can be obtained.

3.4. Assessment of Indicators

The decision-making simulation experiment aims to determine the multicriteria method that is simplest and/or performs best in project evaluation. In addition, with the multi-criteria methods, the aim is to be able to parameterize and weigh the dimensions of sustainability in the most objective way possible, through the evaluation of qualitative variables, converting them into numerical data that allow the comparison between different selection alternatives. On the other hand, having no knowledge of decision-making methods can guarantee impartiality in the measurement of indicators and, at the same time, can extrapolate the results on the perceived difficulty of the methods to any possible participant.

There were three indicators to be measured in this simulation, through the application of a survey at the moment the participants finish the decision-making process. The first two indicators were chosen from the experiment conducted by Arroyo et al. [26] and were: personal satisfaction and personal frustration. Measuring the time to reach consensus has been ruled out due to the length of the list of sustainable factors being measured in each of the decision-making methods. In addition, it was chosen to measure the perceived difficulty at the time of applying the method, since to define a decision-making methodology to be applied by anyone interested in the topic, one should choose the one that presents a less general difficulty.

3.4.1. Personal Satisfaction

The personal satisfaction indicator is measured on a scale of one to ten, with one being the least satisfied and ten the most satisfied. This indicator is measured once the participants have made their respective decisions. After the discussion and argumentation process to reach mutual consensus, the indicator is measured with respect to how satisfied they are with the group decision regarding their individual decision or opinion during the discussion process.

3.4.2. Personal Frustration

The indicator of personal frustration is measured on a scale of one to ten, with one being the lowest degree of frustration and ten being the highest. This indicator is measured once the participants make their respective decisions, after the process of discussion and argumentation to reach mutual consensus, and at that point, the indicator is measured with respect to how frustrated (disappointed) they feel with the group decision regarding their individual decision or opinion during the discussion process.

3.4.3. Perceived Difficulty

The indicator of the perceived difficulty of the method is measured on a scale of one to ten, with one being the lowest degree of difficulty and ten being the highest. This indicator is measured once the participants have applied the decision-making methods and reached a mutual agreement; that is, they have gone through the entire step-by-step process of WRC, CBA, and AHP, respectively.

3.4.4. Weight of Sustainability Dimensions

Another indicator that can be obtained from the simulation experiment is the importance or percentage of weighting associated with each dimension of sustainability, depending on the decision-making method being used.

In the case of the WRC method, students were asked to give a weight to each dimension of sustainability so that it would add up to 100%,

The CBA method was the only one where students were not directly asked to obtain the weight of each of the dimensions of sustainability. For this reason, the weight of the dimension is obtained from the importance of the advantages of the attributes that each project presents with respect to the factors and evaluation criteria associated with the sustainability dimension. Equation (1) represents how the weight of each dimension was obtained.

DW_CBA = ∑I_D/I_T

(1)

where DW is the Dimension Weight; I_D is the importance of the factors associated with that dimension, and I_T is the total importance of the project.

In the AHP method, the dimension weight is obtained from the average vector of the comparison between criteria (three sustainability dimensions). As shown in the example of the AHP method, the last step is to make the paired comparison between the criteria (Figure 7) and build the trial matrix, then the normalization of the matrix is achieved and the average vector that represents the weight of each of the dimensions is obtained.

3.5. Statistical Metrics Analysis

To determine whether there are significant differences between the responses of different groups of individuals, non-parametric statistical tests were performed using Real Statistics. In the framework of the present research, we chose to perform the Mann–Whitney U hypothesis statistical test and the Kruskal–Wallis test for independent samples; the first one serves to compare two independent random samples from two populations, while the second one allows comparing n random samples. For these tests, a significance value (𝛼) equal to 0.05 is assumed, which is equivalent to a 95% confidence level. Therefore, if p-value > 0.05, there is no significant difference between the responses.

Besides, a second comparison criterion (Z > +/−1.96 or H > 5.99) was used for the statistics of the hypothesis tests, which is given by the critical values associated with the normal and chi-square distributions for Mann-Whitney’s and Kruskal-Wallis’ U-tests, respectively.

4. Results and Discussion

During the decision-making simulation experiment, the groups were to decide between three basic road projects according to a list of factors and sustainable criteria that allowed them to evaluate these projects. Figure 8 shows that project two was most commonly chosen by all the groups and that the CBA method shows the greatest stability. In contrast, the AHP and WRC methods have almost the same distribution of the percentage of project choice, where the choice varies between project one and project two.

From the ten working groups that participated in the simulation experiment, 80% chose project two with the CBA method; this may be because this method included in its procedure the creation of a graph that allowed one to visualize the total importance that they give to the projects (through scores) versus the total cost of the project. Moreover, since project two was the most economical of the three, this may be the reason why the groups ended up choosing it. However, even though the other methods did not assess the project cost variable, there is still a strong predominance of project two choice, although the choice of project one also increases compared to the CBA method.

4.1. Indicator Results

In this section, the results of the survey for the three measured indicators (personal satisfaction, personal frustration, and perceived difficulty) are detailed and analyzed. This is in order to determine the most understandable, stable, and accepted method by any user who must learn to use it.

4.1.1. Personal Satisfaction

The first indicator measured was personal satisfaction. This indicator, as mentioned above, is measured in terms of how satisfied each participant is with the group’s decision regarding their individual decision in the discussion process to reach a consensus. It is measured on a scale of one to ten, with one being the least satisfied and ten the most satisfied.

The results are presented in Figure 9, which shows the median of the students’ personal satisfaction ratings in each of the ten working groups that were formed for the three decision-making methods.

To observe statistical parameters and thus better determine the behavior of the decision-making methods, the decision was made to make box plots with the median of the responses of the working groups, which are shown in Figure 10.

It can be observed that the WRC method generates a higher satisfaction (level eight) and is the least variable of the methods. Additionally, there is not much difference in the satisfaction generated by the CBA and AHP methods; however, the latter generates a greater dispersion of responses. However, the only method that generated aberrant data was the WRC; this could be because during the award process, large differences of opinion were generated between the working group or because they are the type of participants who tend to respond according to the average of the scale values. The results of the statistical tests are summarized in

Table 11. Statistical test results associated with personal satisfaction.

Comparison between MCDM	Statistic Parameter	p-Value	Ho	Are There Any Significant Differences?
WRC vs. CBA	U = 41/Z = −0.68	0.51	Not rejected	NO
WRC vs. AHP	U = 39/Z = −0.83	0.42	Not rejected	NO
CBA vs. AHP	U = 45/Z = −0.38	0.73	Not rejected	NO
WRC vs. CBA vs. AHP	H = 0.89	0.64	Not rejected	NO

, where it can be seen that no significant differences are observed in any comparison between the decision-making methods.

4.1.2. Personal Frustration

The second indicator measured was students’ personal frustration with each of the decision-making methods. This indicator, as mentioned above, is recorded after participants reach a consensus and measures how frustrated each participant is with the group decision regarding their opinions in the discussion process. The occupied measurement scale was from one to ten, with one being the lowest degree of frustration and ten being the highest. The results are presented in Figure 11, which shows the median of students’ personal frustration in each of the ten working groups formed for the three decision-making methods.

As with the previous indicator, box plots are made with the median of the responses of the working groups, which are shown in Figure 12. It is again the AHP method that generates the greatest dispersion of responses and also generates the greatest degree of frustration in the students. On the other hand, the WRC and CBA method diagrams are very similar in both degree of frustration and dispersion of responses; however, the WRC method continues to gain a little. The results of the statistical tests are summarized in

Table 12. Statistical test results associated with personal frustration.

Comparison between MCDM	Statistic Parameter	p-Value	Ho	Are There Any Significant Differences?
WRC vs. CBA	U = 39.5/Z = −0.79	0.44	Not rejected	NO
WRC vs. AHP	U = 26.5/Z = −1.78	0.078	Not rejected	NO
CBA vs. AHP	U = 35.5/Z = −1.1	0.29	Not rejected	NO
WRC vs. CBA vs. AHP	H = 3.45	0.18	Not rejected	NO

, where it can be observed that in no comparison between the decision-making methods are their differences significant.

4.1.3. Perceived Difficulty

The last indicator measured was the perceived difficulty in using each decision-making method, this indicator, as mentioned above, is measured once the participants have applied the methods proposed in the simulation experiment, in this case, WRC, CBA, and AHP. Its measurement scale is from one to ten, with one being the lowest degree of difficulty and ten being the highest degree of perceived difficulty.

The results are presented in Figure 13, which shows the median of perceived difficulty by students in each of the ten working groups that were formed for the three decision-making methods. Once again, in order to observe relevant statistical parameters and to be able to better compare the three occupied methods, the box diagrams with whiskers are made, as shown in Figure 14.

As in the previous indicators, the AHP method shows a great dispersion in the responses and has a greater difficulty according to the students’ perception. Next comes the CBA method, which has a medium difficulty (level 5.8 approximately) and has a low dispersion of data. Finally, there is the WRC method, with a difficulty of approximately 3.6 with a higher dispersion than the previously mentioned method. The results of the statistical tests are summarized in

Table 13. Statistical test results associated with perceived difficulty.

Comparison between MCDM	Statistic Parameter	p-Value	Ho	Are There Any Significant Differences?
WRC vs. CBA	U = 14/Z = −2.72	0.0069	Rejected	YES
WRC vs. AHP	U = 16/Z = −2.57	0.011	Rejected	YES
CBA vs. AHP	U = 36.5/Z = −1.02	0.323	Not rejected	NO
WRC vs. CBA vs. AHP	H = 10.18	0.0068	Rejected	YES

, where it is observed that in three of the comparisons, there are significant differences between the decision-making methods. In this case, the better behavior of the WRC method is evident with respect to the other two, both at the level of less difficulty perceived by the users and by less dispersion among the responses.

Finally, it can be stated that there is a statistically significant difference between the perceived difficulties of the method obtained when comparing the methods exposed in the simulation. Therefore, it can be concluded that the WRC method is perceived as simpler in its use and development when contrasted with the CBA method, and this affirmation is statistically supported when checking the existence of significant differences between these methods.

It is assumed that the WRC method is perceived as an easier method since for the participants of the simulation, it was a method closer to reality. This is because university courses usually use weighted averaging methods to obtain the final course grade and establish passing criteria. Another example of the use of the WRC method is during the evaluation process of public competition applications to determine the selection of candidates. In fact, more than half of the participants unconsciously developed a step structure similar to that of the WRC method; that is, they weighted the characteristics in a subjective way and then rated that characteristic for each option they chose. In contrast, the CBA method had an entirely new step structure, making it more difficult for participants to assimilate and subsequently execute the steps. It is felt that the perceived difficulty of both methods should be equalized or at least approached once these methods are put into practice by the working groups.

4.2. Sustainability Dimensions

Another parameter that could be visualized through the application of the decision-making simulation experiment was the importance or percentage of weighting associated with each dimension of sustainability, according to the method used. To make the final decision on which project to choose, the groups had to assign each environmental, social, and economic dimension a weight, importance or weighting. These percentages have been represented as boxed plots for each dimension of sustainability, as shown in Figure 15.

From the diagram in Figure 15, it can be seen that the WRC method has the greatest homogeneity of responses, and the AHP method has the greatest distribution of importance associated with each factor. Between the CBA and WRC methods, a similar median weighting is observed in the three dimensions of sustainability, which does not vary by more than 10%. In contrast, the AHP method shows aberrant data that does not allow such a clear median to be visualized among the responses.

Finally, according to the statistical parameters and from the opinions collected from the participants of the simulation process, it was determined that the WRC method is the one that has the greatest stability, less dispersion in the responses, and at the same time, represents greater satisfaction and less difficulty on the part of the users. Of the opinions of the participants, the most outstanding is that the AHP method represents the comparison and filling of too many matrices when contemplating the evaluation of projects with so many sustainable factors, so the method can contribute to making a decision in a more objective way. However, it is not favorable to the optimization of time and to make for a more efficient process. On the other hand, participants noted that the characteristics of the projects that appear in the bases and technical specifications are quite similar, so with the CBA method, many of the factors were left without associated importance (score), because one project did not represent an evident advantage over the others. Furthermore, on this same point, when project evaluation considered so many sustainable factors, the fact of having to express in words the advantages of the projects according to the definition of the attributes associated with them, the process was slowed down, and again the objective of optimizing decision making was not met.

5. Conclusions

Through this research, a simulation decision-making activity was applied as a methodology for teaching three MCDM in the context of civil engineering and road infrastructure projects using sustainable factors. This activity was developed in the highway engineering course of the civil engineering career and allowed one to determine which of the three proposed multi-criteria methods performed better in that experience. To determine the best method, three individual indicators were measured, analyzed and compared through the application of hypothesis tests. The results were partial, as the WRC method performed better than the CBA and AHP method on one of the three individual indicators (perceived difficulty of the method) and had better results on the three indicators combined. However, in the other two indicators, no significant differences between responses could be seen, so any busy method produces similar personal satisfaction and frustration for the users of the experiment.

It is concluded that the WRC method is the appropriate method for the prioritization system of basic road projects since it is perceived as a simpler method that generates a lower dispersion of responses. This would contribute to the optimization of the project selection and evaluation process when a large number of projects must be evaluated simultaneously and/or are evaluated with respect to a large list of sustainable factors. The teaching methodology used is of a practical-theoretical type and allows for the step-by-step teaching of three MCDM methods that, according to the literature review, are widely used in the AEC industry. This teaching activity is intended for undergraduate students and professionals in the AEC industry who require a decision-making tool that promotes transparency in problem-solving and who have no knowledge of MCDM.

This research was limited to two sessions of 90 min for the teaching of three MCDMs, so it is observed that the method that has better results is due to its simplicity and the familiarity that students feel with the steps necessary to make the final decision. Therefore, it is recommended that these decision-making methods be taught in a semester course, where the understanding of the MCDMs by all the students can be visualized and tested. From the results of the indicators of frustration and perceived difficulty, it can be perceived that the activity was challenging for students in the training stage, where they evaluated real basic road projects with sustainable criteria that they were not familiar with. Thus, the next step would be to incorporate the sustainable criteria into other undergraduate training subjects to create an approach to these concepts and how to apply them during professional development. From the results of the indicator weight of the dimensions of sustainability, we observe an evident dispersion of the percentage associated with the weights according to the methods applied and that students give more weight to economic factors. In this way, it could be thought that future students can continue to give greater importance to economic aspects of environmental and social aspects. This may be due to the type of project evaluated, which has the characteristic of low budget projects for its execution, or because students have a greater knowledge of project evaluation with economic criteria. A next step could be to apply MCDM methods to other types of projects to evaluate the variation in the weights assigned to each dimension of sustainability, and thus conclude whether it is necessary to incorporate subjects related to environmental and social aspects in infrastructure projects into the curriculum of civil engineers.