ENHANCING DECISION-MAKING IN WATER RESOURCES MANAGEMENT: AN INNOVATIVE ASSESSMENT OF EXPERT CONSISTENCY AND COMPETENCE

 

Bojan Srdjevic ¹ , Zorica Srdjevic ¹ , Milica Ilić ¹ , Senka Ždero ¹

 

¹ Faculty of Agriculture, Department of Water Management, University of Novi Sad, Group for Systems Analysis and Decision-Making, Trg Dositeja Obradovića 8, 21000 Novi Sad, Serbia

 

1. INTRODUCTION

Policy making and strategy are two distinct but closely related concepts. They play crucial roles in water resources in various fields, including government, business, and organizations. While they are distinct concepts, they are often interconnected and complementary. Before creating a comprehensive plan of action designed to achieve specific goals or objectives in water resources and related sectors (which commonly refers to defining strategy), the policy-making process involves formulating, implementing, and evaluating a set of principles, guidelines, or rules designed to address specific issues defined in the strategy. Both are aimed at solving problems and commonly are the responsibility of governments, organizations, or institutions to guide decision-making and actions.

Policy-making is a high-level approach that involves making choices about how to allocate resources, deploy capabilities, and respond to challenges or opportunities. In various instances of policy creation, and especially before its implementation, group decision-making (GDM) is widely applied. For instance, many examples can be found in the literature related to decision-making processes within water users associations such as small, medium, and large irrigators, or within urban water distribution regulatory bodies Srdjevic et al. (2022).

Groups of individuals such as delegates, experts, or responsible regulators can be of various sizes. One of the commonly used multi-criteria methods to support GDM processes in water resources science and practice is the analytic hierarchy process (AHP) Saaty (1980). This method is developed for individual applications but in the last few decades, its application has been extended to group scenarios due to its simplicity in implementation and its intuitive clarity. This is important because individuals participating in the group have different educational backgrounds and interests, emulating different sector-inspired priorities, etc. Yet, challenges persist in using AHP, including reaching a consensus and assessing the quality of individual judgments.

This paper presents a procedure for straightforwardly evaluating the quality of individuals acting as decision-makers in a group based on their consistency while making judgments and detecting their compatibility with the rest of a group once deriving the final decision is underway. If a complete hierarchy of the decision problem is assumed with a goal at the top, criteria set at the level below, and a set of alternatives at the bottom of the hierarchy, the AHP is very convenient to use and measure specific consistency and compatibility indicators as individual and group performance quality. Part of the procedure is to use the multi-criteria method CP (Compromise Programming) Zeleny (1982) and evaluate the individual quality of performance of the members of a group based on their demonstrated consistencies while judging decision elements. Based on AHP results, this method enables ranking of the group members by their quality due to its ability to measure distance and its effectiveness in handling decision matrices where alternatives' ratings are compared against specific criteria. The main challenge with this method lies in assigning normalized weights to criteria, which can either be arbitrary (based on the problem's nature), or precise if determined through methods like Shannon's entropy or the CRITIC method Srđević et al. (2020). Our approach involves assessing the distance between individual AHP decisions and the AHP-based group decision with the CP using competency-related performance indicators as criteria. An example is provided to illustrate the procedure based on the combined use of AHP and CP.

As an introduction to the topic, a question could be raised ‘What is group decision-making?’ Among many definitions, an appropriate one is that it is a concept of giving a topic to a targeted group of people where the individuals share or accept that their opinions will be collectively managed toward the decision. While there are many open questions in this area, this research aims to address some specific problems, improve solutions, and justify approaches. One challenge is selecting appropriate performance indicators, which is a multidimensional problem that can be visualized using tools such as multidimensional scaling to identify possible outliers in the group.

 

It is important to note that after the group members have established their priority vectors, the organizer of the decision-making process has several options for determining the outcome. One approach is to use the priority vectors derived by each individual and aggregate them to create a group priority vector. Another approach is to perform an adjacency assessment of the individual vectors, grouping members based on the distances among their vectors, and deriving sub-group priority vectors. This approach may result in a different group vector than if all individuals belong to a single group. Additionally, this assessment can identify members who significantly deviate from the group decision, leading to their exclusion from the process or assigning low weight to their individual decisions. Managing the process with outliers in the group is a complex issue that requires careful consideration of both subjective and objective factors.

In the approach we propose, the AHP is used as a primal paradigm of solving the water resources decision-making problem by an individual. Then, the context is extended to the group framework, where an individual becomes a member of the group and is subjected to evaluations regarding his/her quality of decision-making performance. Individual hierarchy-wise consistency of each member is represented by multiple performance indicators. To measure individual agreement with the group, the hierarchy-wise deviation of each member from the group is represented by deviation performance indicators. In other words, individual outcomes are merged into a single, group, the outcome in a consensual or another manner (e.g., by additive or geometric aggregation). Evaluation of consistency/deviation performance of the members of the group and their ranking in the newly established multi-criteria framework is managed by the use of the CP method.

The methodology is applied to assess the significance of six Ramsar sites in Serbia, which face threats from substantial alterations in water regimes, droughts attributed to climate change, and contentious human interventions in associated infrastructure and ownership conditions, such as alterations in land usage. It is found that this methodology provides a structured and efficient approach, enabling assessment of the individual performance of involved participants in the group to aim at deriving a single and competent group decision. In its conclusive implementation phase, the methodology can assist in the identification of suitable experts who will play a pivotal role in defining policy and crafting strategic solutions for investment and management endeavors related to the safeguarding and future progress of Ramsar sites in Serbia.

 

2. RELATED WORK

Group decision-making is challenging due to differences in members' backgrounds, attitudes, communication skills, and willingness to adjust. Moreover, the behavior of decision-makers during the process is crucial, as it can be lengthy and repetitive, requiring them to revise their judgments as new information becomes available. Throughout the process, decision-makers must demonstrate their ability to assess causality, importance, preferences, and goals while considering the available data and any other limitations or constraints.

Measuring the quality of the decision-making performance in a group is a delicate matter that involves subjective characteristics such as cognition, reasoning, inference, and deduction which is difficult to aggregate into a reliable judgment outcome. This can lead to errors and complications in an already complex decision-making process. However, the AHP has proven to be an efficient tool for handling such challenges and deriving trustworthy solutions to well-structured problems. In the AHP, the decision-making problem is structured as a hierarchy, with the goal at the top, criteria, and sub-criteria (if any) on the level below the goal, and alternatives on the bottom level. The decision-maker then uses a ratio scale to compare decision elements (criteria, sub-criteria, and alternatives) at a given level by their importance concerning adjacent decision elements at upper levels. These judgments are inserted into local comparison matrices, known as 'multiple-preference relations’ to indicate the importance rather than causality of compared decision elements. The selected prioritization method is then used to derive weights from all local matrices and the final synthesis of local vectors produces the final vector with global weights of alternatives versus the goal.

Srđević et al. (2020) defined a framework based on the group AHP for identifying the most desirable technologies for constructed wetland segmentation. A method is defined for aggregating the evaluations provided by the members of the group into the new metrics by calculating different consistency and statistical conformity measures. The three two-dimensional metrics and the one three-dimensional metric are created to determine the distances of the members from the reference points corresponding to full consistency and statistical conformity. The mapping of members is performed into the consistency/conformity scatter plots to enable visualizing the outlier(s), that is, the members who have different opinions about which technologies to apply in wetland segmentation than all the members on average. The scatter plots are intended to guide the decision-making process towards grouping participants into subgroups, thus heading towards consensus in both subgroups and global groups.

The basic idea in multicriteria method CP (Compromise Programming) is to identify an ideal, or so-called utopian, solution as a point of reference for the decision maker Zeleny (1982). The basic assumption is that any DM seeks a solution as close as possible to the ideal point that realistically represents human preferences. To achieve this closeness, a distance function is used with several possible metrics applied. The concept of distance in CP is not used in its geometric sense, but as a proxy measure for human preferences Romero & Rehman (2003). The idea of a distance metric or a family of distance functions is essential for the CP technique to work. Applications of this method can be found in rich literature related to water resources, e.g., in Marzieh et al. (2021), Sarband et al. (2021), Ekhtiari & Zandieh (2022), and Marzieh et al. (2021).

Srdjevic et al. (2002) presented a comparative analysis of alternatives using multicriteria decision-making methods PROMETHEE, TOPSIS, and CP in agriculture. An illustrative example is given to demonstrate how these methods can be used in parallel with the scalarization technique and standard LP algorithm to solve the problem related to water resource allocation scenarios. Two simple techniques for their comparison are also suggested.

Sarband et al. (2021) discussed different aspects of integrated water resources management and methods for assessment of water allocation scenarios performing the spatial multi-criteria analysis. Localized impacts of water allocation scenarios across the Aras basin in Iran are analyzed regarding their spatial variations. The authors pointed out that conventional multi-attribute decision-making is not capable of capturing and implementing spatial variations because their use often results in simplification or even oversimplification. Therefore they proposed a new framework involving a compromise programming approach, a fuzzy analytical hierarchy process, and distributed economic, social, and environmental indicators. It is claimed that the use of such a framework enables a more detailed evaluation of integrated localized impacts and spatial trade-offs. Based on the results of this research it is concluded that using lumped indicators rather than distributed indicators imposes significant uncertainties in the evaluation process. Applying CP, distributed indicators, and fuzzy weights simultaneously extended the interval with stable ranking and allowed the determination of regional overall priorities and optimal spatial water allocation.

One possibility of adapting the CP method is presented in Jarraya-Horriche et al. (2022) to choose a suitable site for the artificial recharge of the groundwater aquifer in Bena Arous in Tunisia. Considering the localized impacts of water allocation as criteria, the different sites as alternative solutions are validated with equal and several schemes of different weights assigned to criteria.

Khademi et al. (2022) used a stochastic version of CP for evaluating alternative urban water resource allocation scenarios.  The key goal of this research was to create an integrated decision-making-based multistage scenario-based interval-stochastic programming model to serve as an urban water management supporting tool. The model is developed for a given case study area and two key objectives are implemented in the model, the economic value of the benefit gained and social satisfaction level. Various constraints are considered under uncertainty and the chance constraint method is applied to model all constraints imposed in the model.

Freitas et al. (2022) reported on a compromise programming approach in developing a composite indicator to measure sustainable water use in Portugal. They identified the main factors affecting human life, economic activities, and ecosystems’ survival, and proposed monitoring indicators considering these factors at the municipality level. Research led to the conclusion that tourism activity, income level, and young age population have a significant negative effect on sustainable water use and that municipal revenue has a positive effect. It was also shown that irrigated farming does not have a significant negative effect on sustainable water use, as well as population density, elderly population, and education level.

An interesting discussion on various aspects of confidence and its matching in group decision-making environments is given by Bang et al. (2017). Authors analyze probabilities of correcting individual opinions within a group and levels of adaptation which enables combining opinions optimally and establishing their confidence according to a common metric. It is shown that matching individuals' communicated confidence can be more effective when group members have similar levels of expertise. It is also shown that matching is more robust when group members do not have insight into mutual relative levels of expertise. One of the conclusions of this research is that confidence matching can cause miscommunication among group members about recognition of who is more likely to be correct and that the herding behavior can be a reason why groups sometimes fail to make good decisions.

Humphreys & Jones (2006) elaborated on theoretical aspects of group decision-making processes from their start to the end, from the level of freedom of decision-makers to think about translating their desires into action, to the level of group-agreed imaginable courses of action as candidates for implementation. Reported research seeks to identify a synthesis of theories that influence decision-making within organizations. It proposes a comprehensive system of 'Group Decision Authoring and Communication Support (GDACS)' which enables the extension of visual language authorship to support decision-making. The suggested approach tends to ensure good group decisions by working iteratively in the development of a collective narrative within a group. Eventually, it will come up to active engagement of individuals and final implementation of the decision – a process called 'Collaborative Authored Outcomes'. In many instances, the described approach is similar to modern concepts of participatory decision-making supported by user-friendly computerized tools and multi-media platforms.

Generating solutions to multiple criteria group decision-making problems that are satisfactory to the decision-makers can be achieved in many different ways. Globally speaking this can be done by consensus or by aggregation methods, in some cases by voting. For instance, Fu et al. (2020) proposed a new method to examine how much is the group satisfied after alternatives are assessed and ranked based on differences between the decision-makers and the group. In this research, it is demonstrated how to analyze group satisfaction and group consensus based on differences in alternatives’ grades versus group (alternatives) grades using Spearman’s rank correlation coefficient.

In GDM applications of the AHP method, of particular importance are distances of individual priority vectors from the group vector and possible violations of the rules (such as transition) and consistencies of judgments while deriving such vectors. For further reference, note that the result of individual AHP applications is the priority vector of alternatives versus goal, derived after synthesis of local priority vectors computed for criteria versus goal and then alternatives versus criteria. The number of elements in each priority vector is equal to the number of alternatives n. In a group context, there are m priority vectors w_i (i = 1, 2,...,m) for m individuals, which can be aggregated in one – the group vector w^G.

Distances of individual priority vectors from the group vector can be measured in many different ways, for instance by using distance functions such as Manhattan, Euclidean, Cosine, Jaccard, Dice, RMD (root-mean-square deviation), etc. More on distances can be found in Saaty (1980), Mikhailov (2000), Cha (2007), Deza & Deza (2009), Chiclana et al. (2013).

Our research shows that the application of one of the mentioned distance functions in group decision-making problems does not produce significant differences in the measurement of individuals’ agreement with the group consensus. A similar hypothesis has been proven by Chiclana et al. (2013) for the five firstly mentioned distance functions above. Notice that in this paper we used the first two functions in a different context than in referenced studies. Manhattan distance is used as a group measure, that is, to measure the conformity of each individually derived priority vector from the group vector. The Euclidean distance is used as an individual measure only, that is, for measuring total deviations of individual judgments at all hierarchy levels with derived local priority vectors.

Worth mentioning is that along with AHP most often used for assessing the quality of the estimates of priority vectors are generalized L2 Euclidean distance (ED) Barzilai (1997), and minimum violations (MV) criterion Golany & Kress (1993), for instance, ED is the total distance between all judgment elements in comparison matrix at a given level of the hierarchy and related ratios of the weights contained in the vector w derived from this matrix by some prioritization method. The ED is a universal error measure, and it does not depend on the prioritization method used to derive vector w. The MV measure sums up all violations associated with the priority vector w and judgments contained in the comparison Golany & Kress (1993). More on these two measures will be given in the next section.

 

3. MATERIALS AND METHODS

In Srđević et al. (2021), six endangered Ramsar areas UNWTO (2012) in the northern part of Serbia are assessed as complex environmental systems. An exposure of areas to the risk is then evaluated with the AHP by seven experts. Worth mentioning is that sites are multifunctional environmental systems, and all at risk mostly because of undesired climate change, but also because of a lack of normative documents regarding their protection and management, and, in a way, a general misunderstanding of their importance for society. To detect possible future actions for improving the situation, sites are evaluated and ranked by importance against a set of eight meaningful criteria. The experts in the subject area individually evaluated criteria and alternatives (Ramsars) and geometric averaging of individually obtained weights of Ramsars produced a group result: their weights from the risk point of view. The final ranking of sites by importance was an easy task once aggregated (group) weights were obtained.

Ramsar are assessed within standard AHP-group decision-making framework for their vulnerability to adverse effects from human activities and natural phenomena such as climate change. Detailed descriptions of these sites can be found in Srđević et al. (2021), while summarized below:

A1 – Gornje Podunavlje: Positioned along the left bank of the Danube River in northwestern Serbia, this Special Nature Reserve boasts a diverse array of ecosystems. Renowned for its biodiversity and cultural significance, it has been designated as a significant national scientific and recreational zone owing to its remarkable natural resources.

A2 – Koviljsko-Petrovaradinski Rit: Covering around 6,000 hectares along the middle course inundation area of the Danube River, this Ramsar site comprises a complex of marshes and forest ecosystems. Offering ample tourist attractions and insights into national heritage, it holds significant value.

A3 – Obedska Bara: Reflecting a vestige of the former meander of the international Sava River, this Ramsar site boasts diverse ecosystems. Its elevated dryland area intersects water depressions teeming with various mammal, fish, and reptile species.

A4 – Carska Bara: Extending over nearly 17 km2, this special nature reserve, also known as Imperial Pond, is Serbia's largest individual bogland. Accumulating a substantial deposit of dead plant material, it stands as a significant wetland.

A5 – Zasavica: Situated in the southern part of Vojvodina Province, west-central Serbia, this wetland Ramsar site spans across the Sava River. Alongside Obedska Bara, it serves as a crucial wildlife sanctuary in the country.

A6 – Slano Kopovo: Nestled in the central part of Vojvodina Province, this lake basin remains one of the few ponds in the Pannonian Basin to have evaded drainage. During periods of drought or semi-drought, its water level decreases, leaving behind salt layers that impart a desert-like appearance to the area.

 

The criteria set utilized by experts for evaluating risk-related impacts across Ramsar sites, as outlined in Srđević et al. (2021), are as follows:

C1 – Habitat protection level

C2 – Biodiversity

C3 – Water regime

C4 – Purposes

C5 – Geographical location

C6 – Tourism and educational potential

C7 – Water quality

C8 – Cultural heritage

 

Seven individuals participated in a group and performed the required evaluation and ranking of Ramsars by the AHP method. They are unanimously identified as members M1-M7 with only associated information on their educational and professional background:

M1 – Agriculture and water management (PhD candidate 1)

M2 – Environmental engineering and management (PhD candidate 2)

M3 – Natural resources protection and management (M.Sc., Expert 1)

M4 – Systems analysis and natural resources management (Univ. Prof. 1)

M5 – Water Resources management (PhD, Expert 2)

M6 – Systems analysis and natural resources management (Univ. Prof. 2)

M7 – Natural resources protection and management (M.Sc., Expert 3)

 

The evaluation method is divided into five steps as follows:

Step #1

Solve given decision-making problem by each member of the group and obtain individual priority vectors of alternatives concerning problem objectives and goals. During this process measure the consistency of the members and record individual indicators of consistent performance.

This step can be realized in many different ways. Without losing generality, and without going into too many details, it can be assumed that the decision-making problem is previously defined through an agreement of participants in the group (members), reached by consensus or by another means. Commonly, the problem can be hierarchized with the goal, objectives, and alternatives as key elements of the problem. A sufficiently good paradigm, which will be used in illustrating way how to realize the next steps as well, is the Analytic Hierarchy Process and the use of Saaty’s 9-point scale (Table 1) for pairwise comparisons of decision elements and creation of local comparison matrices of type A (Eq. (1)).

 

                                                                                                           (1)

 

Table 1

Table 1 Saaty's Importance Scale
Definition	Assigned value
Equally important	1
Weak importance	3
Strong importance	5
Demonstrated importance	7
Absolute importance	9
Intermediate values	2,4,6,8

 

Completion of the AHP is achieved after synthesis of all local priority vectors are synthesized by a simple addition procedure in a top-down direction. The outcome of the method is a global priority vector of alternatives versus goal, derived regarding priorities of criteria (and before that across sub-criteria if they exist).

During the AHP application by each member of the group, it is possible to compute locally and globally (hierarchy-wise) various measures of consistency indicating the quality of fitting original judgments to corresponding priorities from the local vectors computed by the prioritization method. For instance, if prioritization is performed by the eigenvector method (EV), the consistency ratio CR proposed by Saaty (1980) can be computed for matrix A. It was suggested that the preferable tolerance of this ratio should be up to 0.10. CR values for all comparison matrices are synthesized using a weighting scheme to obtain unique consistency parameters for the hierarchy. This value can be denoted as hCR, where h corresponds to a ‘hierarchy-wise’ indication.

Besides the CR index, which is established exclusively for the AHP method, there are two more measures of consistency, commonly used within the AHP framework, minimum violation criterion (MV), and total Euclidean distance (ED). The MV sums up violations of initial judgments associated with the priority vector w for the comparison matrix (1). ‘Conditions of violation' penalize possible order reversals such as this: if the j-th alternative is preferred to the i-th one (i.e., a_ji> 1), but the derived priorities are such that w_i > w_j, then there is a 'violation', or element preference reversal Golany & Kress (1993). MV consistency measure for the whole hierarchy can be denoted as hMV and obtained by summing MVs at all levels of the hierarchy.

The third important indicator is Euclidean distance (ED) which measures the total distance between all judgment elements in the comparison matrix A at a given level of the hierarchy and related ratios of the weights (w_i/w_j) contained in the vector of weights derived by the eigenvector method. On a hierarchy–wise level, the total Euclidean distance hED can be obtained by summing distances obtained at all levels of the hierarchy.

Note that MV and ED consistency measures can be used in case of any prioritization method for assessing consistency as reported by many researchers (e.g., Mikhailov & Singh (1999), Srđevic (2005), Kou & Lin (2014).

It is important to note that the performance indicators listed here have been chosen because they are commonly paired with the Analytic Hierarchy Process (AHP) to quantify the consistency of decision-makers. These indicators are easily understandable to decision-makers, aligning with their cognitive perceptions. Moreover, they can function as a regulatory tool throughout the entire decision-making process. For example, these indicators are measurable at every level of the problem hierarchy and can validate the transitivity rule, which is fundamental in assessing elements despite the limitations posed by using a 9-point scale or any other scale to compare the importance sets of decision elements (criteria and alternatives).

 

Step #2

Synthesize individual priority vectors of alternatives into the group priority vector. Use the group vector as a reference vector and compute deviation, correlation, and other indicators of individual agreement and/or disagreements with the rest of the group. This set of measures can be denoted as group-related indicators of individual deviation (dispersion) performance and recorded analogously as consistency indicators in Step #1.

 

Individually derived global priority vectors (alternatives vs goal) can be geometrically aggregated to obtain the global group priority vector given by Eq. (2).

 

i =1, ..., n                                                                                        (2)

 

where m stands for the number of members in a group,  for the priority of the ith alternative for the jth member, a_j for the ‘weight’ of the jth member, and  for the aggregated group priority value. The weights a_j should be additively normalized before their use in (2) and the final additive normalization of priorities  is required.

Once the group vector w^G ( ,…, ) is derived, it can be considered as the reference vector for the members of a group. Deviation of each vector from the reference one can be computed as Conformity (CO), known also as the Manhattan distance. In a global multilevel hierarchy context where more than one matrix exists, and this is the case here, this measure in a hierarchy context is expressed as:

 

CO^j= ,   j = 1, ..., m                                                                                         (3)

 

Superscript G stands for the reference priority vector obtained by aggregation (2). Conformity defined in this way indicates the global similarity of the individual priority vector with the reference group vector. CO performance indicator only applies after all computations in AHP are concluded, which is different from using consistency indicators CR, MV, and ED. Analogously to consistency indicators, conformity of each member of the group is also an indicator of performance in a hierarchy-wise sense and can be denoted as hCO.

Individual agreements and/or disagreements with the rest of a group can also be measured statistically, for instance by comparing ranks of corresponding elements of global priority vectors and global reference vectors for the group. The Spearman’s rank correlation coefficient SC can be used as a specific dispersion/deviation performance indicator calculated as

 

,  j =1,..., m                                                                                                    (4)

 

D_i is a rank difference between the rank of the element from vector w^j for a jth member in the group and the rank of the corresponding element in the reference vector w^G. The number of ranked elements in two vectors is n. The coefficient SC describes the positive or negative correlation between vectors w^j and w^G, and can have a value in the range [−1,1]; a value of −1 is obtained if the elements in two vectors have opposite ranks (ideal negative correlation); the value +1 shows that the elements are fully matched (ideal positive correlation); if SC is zero, the ranks do not correlate. Note that in group decision-making applications, Spearman’s rank correlation coefficient can be understood as a statistical distance measure or statistical conformity of ranks obtained by the decision-makers with reference ranks for an ‘average’ decision-maker. The use of Spearman's rank correlation coefficient assumes that the number of decision-makers is not too small; however, in practice, this performance indicator appeared to be a confident statistical measure even in cases when the number of decision-makers (members in the group) is at least five. For the sake of completeness, it worth mentioning is that SC is a relative measure, not an absolute measure. This means that the two vectors can differ significantly in rank preference but be relatively close in absolute preference. In the case of larger groups, possible misguidance is generally eliminated.

Similar to consistency and conformity indicators (hCR, hMV, hED, and hCO), Spearman's correlation coefficient can be denoted as hSC because it is a hierarchy-wise value, calculated after all AHP computations are concluded.

 

Step #3

Create the decision matrix of size m× n with rows representing members of the group M₁, ..., M_m (as ‘alternatives’), and n columns representing performance indicators (as 'criteria') identified in Steps #1 and #2. Matrix entries in each row should correspond to given member scores regarding performance indicators of consistency and deviation.

In this step, the decision matrix has to be criteria with criteria representing consistency and deviation performance indicators and alternatives representing members of the group. Typically, data for this step are provided in a multidimensional format. A decision matrix can be a large table where the rows represent decision-makers and the columns represent performance indicators. The size of the matrix is not restricted in either direction, that is, columns (for criteria) or rows (for members). Any cross-referencing and visualization during the preparation of the matrix may help to generate insights and impact of matrix elements.

To complete the decision matrix it is required to associate weights to criteria with a sum to 1. Entries of the matrix are performance indicators by members. If the AHP is used with the eigenvector prioritization method, and five consistency and deviation measures are calculated across the complete hierarchy (h) for each member of the group (M_i, i = 1,...,m), the decision matrix X given by (5) is:

 

(w₁    w₂           ...   w₅)

hCR  hED        ...  hSC

                                                                                               (5)

 

Step #4

Apply the TOPSIS method (standard, behavioral, or fuzzified) on a decision matrix created in Step #3 to rank members of the group by their overall performance, regarding individual consistencies and level of agreement with the group and its final prioritization of alternatives. Before the TOPSIS application, specify the weights of all performance indicators.

 

In this step, any multi-criteria method can be used to rank members of the group by their consistency/deviation performance. It usually requires specifying weights of performance indicators, as shown in Eq. (5) with values summing to 1. In the example presented in the next section, the TOPSIS method enabled the required ranking. Worth mentioning is that before the TOPSIS application, instead of the subjective weighting of performance indicators, the entropy principle could be applied to determine the objective weights of these indicators. In the presented case study entropy concept has not been applied because the group consisted of seven experts. It is applicable only if the number of members would be statistically sufficiently large, say 20 or more (members).

 

Step #5

To follow possible outcomes of the process described in Steps #1 – #4, perform sensitivity analysis by allocating different weights to performance indicators clusters (a – consistency, and b – deviation), between clusters and inside the clusters. In all preference schemes, the sum of all weights must be equal to 1.

Depending on the weighting scheme applied to performance indicators, repeated applications of TOPSIS enable evaluation of the performance quality of the members of the group if the focus is put on demonstrated consistency while judging the initial decision-making problem, or the focus is on their conformity and statistical agreement with the rest of the group. In the given example, preference schemes can differently weight groups of related parameters (consistency – hCR, hMV, and hED; deviation – hCO and hSC) and explore opportunities for sub-grouping members, avoiding some of worse performance from the further decision-making process, etc. Simple sensitivity analysis may help to improve the quality of the decision-making process itself, and especially validate the quality of performance of each group member.

The described procedure represented by Steps #1-#5 is easy to further generalize by adding or replacing performance indicators before the final assessment of the quality of group members starts. Other combinations of multi-criteria methods can be used for deriving individual solutions, aggregation of solutions to obtain group solutions, and performing the final evaluation of group members as alternatives by demonstrated consistencies and deviations. It is important to highlight that the coherence of preferences plays a crucial role in shaping the logic and rationality behind the outcomes of decisions made. While this research employs preference consistency as a yardstick to assess decision-maker’s performance, it doesn't necessarily assume that all preferences adhere to this consistency. This intricate matter could serve as a focal point for further exploration within the realm of group decision-making frameworks.

If the combination of multi-criteria methods AHP-CP is used, the methodology can be applied locally at any level within the hierarchy. For instance, if only criteria level is considered, criteria are evaluated concerning a goal; then, hierarchy-wise performance indicators (hCR, hMV, hED, hCO, and hSC) relate to only criteria set and can be denoted as (cCR, cMV, cED, cCO, and cSC). The same is valid if alternatives are evaluated versus criteria, and then performance indicators obtain appropriate local prefixes, in general (lCR, lMV, lED, lCO, and lSC).

The only particularity when AHP is used for individual assessments is that consistency parameter CR should be used only if the prioritization of decision elements is performed by the additive normalization (AN) or eigenvector (EV) method Saaty (1980), Crawford & Williams (1985), Srđevic (2005). Parameter CR should be replaced by GCI along with the LLS prioritization method Crawford & Williams (1985), Aguaron & Moreno-Jimenez (2003), Aguaron et al. (2019), m with the FPP method Mikhailov (2000), or CCI with the CMM method Kou & Lin (2014).

 

4. RESULTS

Ramsar are assessed within standard AHP-group decision-making framework for their vulnerability to adverse effects from human activities and natural phenomena. The final result of the group decision-making process is summarized in two parts of Table 2. The left-hand side of the table contains the weights of Ramsars as individually derived by AHP; the last row contains geometrically averaged individual weights assuming equal importance of the members of a group. The right-hand side of the table presents overall, that is hierarchy-wise performance indicators of the members in a group: (a) hCR – consistency ratio; (b) hED – total Euclidean distance, (c) hMV – minimum violation criterion, (d) hCO – conformity with group weights, and (e) hSC – Spearman's correlation coefficient as a rank distances indicator. A description of all performance indicators is given by Mikhailov (2000) and elsewhere in pertinent literature sources. Prefix h stands to indicate the reference to the complete three-level hierarchy (goal – criteria – Ramsars).

Table 2

Table 2 Weights of Ramsars and Hierarchy-Wise Indicators of Group Members’ Performance (Srđević et al. 2021)
Members/ Weights	Ramsars (Alternatives)						Performance indicators
	A1 w1	A2 w2	A3 w3	A4 w4	A5 w5	A6 w6	hCR	hED	hMV	hCO	hSC
M1	0.235	0.143	0.111	0.220	0.144	0.147	0.028	38.607	1.0	0.312	0.086
M2	0.271	0.224	0.148	0.176	0.122	0.059	0.073	67.426	0.0	0.144	0.943
M3	0.322	0.174	0.172	0.121	0.071	0.140	0.088	51.006	15.0	0.204	0.829
M4	0.260	0.170	0.200	0.125	0.166	0.079	0.080	65.045	18.0	0.108	0.886
M5	0.267	0.194	0.201	0.087	0.187	0.064	0.039	35.697	0.0	0.158	0.886
M6	0.306	0.209	0.123	0.123	0.170	0.069	0.080	46.284	8.0	0.156	0.829
M7	0.246	0.199	0.290	0.114	0.108	0.044	0.284	81.346	30.0	0.245	0.829
Geometrically averaged weights	0.279	0.191	0.175	0.137	0.137	0.081	Average* 0.10	Average* 55.06	Average* 10.3	Average* 0.19	Average* 0.76
Type of criterion							min	min	min	min	max

 

Table 2 can be considered as a base for creating a specific decision matrix within a multi-criteria decision-making framework. Namely, if performance indicators are considered as criteria set, and the members of a group as ‘alternatives’, then following such a setup members of the group were ranked by the CP method to get insight into the quality or competence of the group members.

If the right-hand side of Table 2 is copied into Table 3, then by associating different sets of weights to five criteria (hCR, hED, hMV, hCO, and hSC), it is possible to analyze what happens if the focus is put on consistency (hCR, hED, and hMV), or group conformity and statistics of ranks (hCO and hSC).

 

 

Table 3

Table 3 Decision Matrix
‘ALTERNATIVES’	CRITERIA (Performance indicators)
(Group members)	hCR	hED	hMV	hCO	hSC
M1	0.028	38.607	1.0	0.312	0.086
M2	0.073	67.426	0.0	0.144	0.943
M3	0.088	51.006	15.0	0.204	0.829
M4	0.080	65.045	18.0	0.108	0.886
M5	0.039	35.697	0.0	0.158	0.886
M6	0.080	46.284	8.0	0.156	0.829
M7	0.284	81.346	30.0	0.245	0.829
Type of criterion	min	min	min	min	max

 

Two preference situations are explored to indicate members’ performance. The first one considers one criterion at a time (single criterion framework) and the second one is a set of scenarios where different weights are associated with all criteria (multi-criteria framework). According to data presented in Table 3, individual AHP applications indicated the different global quality of performance of group members (shaded entries of the table) regarding a single hierarchy-wise criterion at a time.

Based on data in Table 3, in Table 4 the ranking of members is given in columns corresponding to each performance indicator and illustrated in Figure 1.

Table 4

Table 4 Overall Ranking the Members in the Group by Borda Count Method Based on their Single Criteria Rankings
Group members	Ranking the members of a group for each performance indicator					Borda	Borda
	hCR	hED	hMV	hCO	hSC
M1	1	2	3	7	7	20	5
M2	3	6	1-2	2	1	13.5	2
M3	6	4	5	5	4	24	6
M4	4-5	5	6	1	2-3	19	3
M5	2	1	1-2	4	2-3	11	1
M6	4-5	3	4	3	5	19.5	4
M7	7	7	7	6	6	33	7

 

Figure 1

Figure 1 Ranks of Group Members for Different Indicators

 

If the ranks are summarized in each row (last column of Table 4), the result can be explained in Borda Count terms D'Angelo et al. (1998), Srdjevic (2007): The best candidate is member M5, second best is M2, and the third is M4. The worst is M7 at the seventh (last) position.

In a multi-criteria framework, three different priority schemes are applied to moderate the importance of performance indicators. Members have been ranked accordingly by using the CP method and the results are shown in Table 5. Recall that the CP method normalizes performance indicators in each column of Table 3, therefore not giving a priori preferences to initial values of members' performance.

Table 5

Table 5 Ranking the Members in the Group for Different Priorities of Performance Indicators.
Priority scheme	Performance indicator					Ranking of group members
	hCR	hED	hMV	hCO	hSC
#1	0.20	0.20	0.20	0.20	0.20	M5 – M6 – M2 – M3 – M4 – M1 – M7
#2	0.30	0.30	0.10	0.15	0.15	M5 – M6 – M3 – M4 – M1 – M2 – M7
#3	0.40	0.40	0.10	0.05	0.05	M5 – M1 – M6 – M3 – M4 – M2 – M7

 

The first scheme assumes equal importance of performance indicators. The top three ranked members of a group are M5, M6, and M2 in that order.

The second scheme equally shares 60% of total importance to consistency (hCR) and dissipation (hED) (30% each), 10% is allocated to the rank reversal criterion (hMV), and the remaining two distance criteria (hCO and hSC) received 15% each. In this case, with the rising importance of consistency, the ranking of the top three group members is M5, M6, and M3.

The third scheme again raised participation of consistency and dispersion while the weights of distance indicators were decreased. The ordering of the top three members is slightly changed: M5, M1, and M6. Member M1 in this scenario precedes M6 due to his/her better performance on the first two indicators hCR and hED (Cf. Table 3).

 

5. DISCUSSION

Analyzed preference schemes show that members M4, and M7 are never positioned at the top three positions regarding the quality of performance; except in the case of priority scheme 2, the same occurs with member M3. Therefore, they might be excluded from the group aggregations in the final stage of the decision-making process.

However, the evaluation process performed in the initial phase of the AHP method assumed mutual independence of eight criteria for validating exposure to the risks of six Ramsar sites. Respecting the logic that comparisons of criteria to derive their weights could be critical at later stages of the decision-making process, it might be useful to re-check the independence of adopted criteria and let decision-makers repeat criteria comparisons before Ramsars are checked by importance against criteria. Because at a later stage of the decision-making process, some criteria might be excluded from the evaluation, and it may also happen with performance indicators, the position of experts in the group may also change in a way that some other members can become candidates for exclusion from the decision-making process. In this regard, the Decision-making Trial, and Evaluation Laboratory (DEMATEL) method Gabus & Fontela (1972), Fontela & Gabus (1976), Si et al. (2008) can be suggested to structurally model and analyze the cause-and-effect relationships among the criteria in a problem hierarchy. DEMATEL may confirm the existence of a relationship or interdependence among criteria once they are adopted for analyzing Ramsar's-related risks. Although DEMATEL may also help to find out the relative level of relationships within the criteria set, this last feature of DEMATEL is not necessary to engage because the AHP comparisons of criteria are sufficient and trustful.

Note that DEMATEL was originally developed as a method aimed at searching for integrated solutions to the fragmented and antagonistic phenomena of world societies Gabus & Fontela (1972), Fontela & Gabus (1976). In general, DEMATEL explores causal relations of the factors describing any system, including technical and natural systems subjected to decision-making processes. The method is based on graph theory and is very effective in understanding the total relations between system factors and components, including the division of factors/components into causes and effects.

The connection of DEMATEL with the AHP and other theories and methods in decision-making can be found in many directions. One interesting could be to assess mutual cause-effect relations between criteria before AHP comparisons of their mutual importance start. 

 

6. CONCLUSION

The evaluation of water- and environment-related decisions in group settings can be influenced in various ways during different phases of the process. After the problem has been defined and resolved by the decision-makers, individual decisions must be combined into a single group decision. There are several methods to achieve this outcome, and one possible approach is to use the analytic hierarchy process (AHP) to derive priority vectors of alternatives versus the goal by individual members of the group and measure their consistency. Once individual priority vectors are aggregated into a unique group vector, the deviation of each vector from group one can be determined, providing performance indicators for members of the group. These indicators can be evaluated in multi-criteria decision-making scenarios, where members are alternatives and the criteria for evaluation and ranking are the selected performance indicators.

For the final assessment of AHP-derived individual solutions, the CP method is proposed. In this method, weights of criteria, i.e. performance indicators, can be subjectively or objectively defined. For subjective weighting, the focus can be on individual consistency or deviation from the group solution. For objective weighting, the entropy principle can be applied. Sensitivity analyses can be used to contrast members and provide feedback in repeated parts or the complete decision-making process. However, the controversial options of selecting top-ranked individuals, excluding odd individuals, or clustering members into smaller groups are beyond the scope of this study.

An example application of the approach in water resources and environmental planning and management provided in this paper demonstrates how the proposed method can be used to rank a group of experts who evaluated changes in water regimes in environmentally endangered sites in Serbia based on eight criteria. The study aims to outline the decision-making processes as "competent environments," emphasizing the evaluation of involved group members (experts) based on their consistency in judgment and alignment with other members.

Our research highlights the importance of conducting a comparative analysis between our approach and alternative methodologies which may include other methods and approaches available. A posterior sensitivity analysis also highlights the importance of individual consistency and the possibility of reducing the group size. Additionally, our sensitivity analyses demonstrate the consistency of our results, affirming the efficiency and robustness of the AHP-TOPSIS-based methodology.

In summary, the evaluation of environmental decisions in a group setting can be complex, but the AHP and CP methods provide a structured and efficient approach to assess individual performance and derive a single group decision. Certainly, it is acknowledged that various methods can yield different rankings in decision-making processes. We firmly believe that the approach presented here, rooted in established multi-criteria methods suitable for real-life decision-making scenarios in water and environment planning and management holds qualities that are easily adaptable and capable of generating reliable solutions that inspire trust. Future research can expand on these methods and explore other tools such as the DEMATEL method to enhance the decision-making process and assessments of planning and management strategies. This method can be used to assess criteria independence before applying the AHP method and to evaluate the independence of performance indicators used for ranking members in the group after the AHP application is completed.

 

CONFLICT OF INTERESTS

None. 

 

ACKNOWLEDGMENTS

None.

 

REFERENCES

Aguaron, J., & Moreno-Jimenez, J.M. (2003). The Geometric Consistency Index: Approximated Thresholds. European Journal of Operations Research, 147(1), 137-145. https://doi.org/10.1016/S0377-2217(02)00255-2   

Aguaron, J., Escobar, M.T., Moreno-Jiménez, J.M., & Turón, A. (2019). AHP-Group Decision Making Based on Consistency. Mathematics, 7, 242. https://doi.org/10.3390/math7030242

Bang, D., Aitchison, L., & Moran, R. (2017). Confidence Matching in Group Decision-Making. Nature Human Behavior, 1. https://doi.org/10.1038/s41562-017-0117

Barzilai, J. (1997). Deriving Weights from Pairwise Comparison Matrices. Journal of Operations Research Society, 48 (12), 1226-1232. https://doi.org/10.1057/palgrave.jors.2600474

Cha, S. H. (2007). A Comprehensive Survey on Distance/Similarity Measures Between Probability Density Functions. Journal of Mathematical Models and Methods in Applied Science, 4(1), 300-300.

Chiclana, F., Tapia Garcia, J.M., Moral, M.J., & Herrera-Viedma, E. (2013). A Statistical Comparative Study of Different Similarity Measures of Consensus in Group Decision-Making. Information Science, 221, 110-123. https://doi.org/10.1016/j.ins.2012.09.014

Crawford, G., & Williams, C. (1985). A Note on the Analysis of Subjective Judgment Matrices. Journal of Mathematical Psychology, 29(4), 387-405. https://doi.org/10.1016/0022-2496(85)90002-1

D'Angelo, A., Eskandari, A., & Szidarovszky, F. (1998). Social Choice Procedures in Water Resources Management. Journal of Environmental Management, 52(3), 203-210. http://dx.doi.org/10.1006/jema.1997.0156

Deza, M.M., & Deza, E. (2009). Encyclopedia of Distances. Springer, Berlin Heidelberg. https://doi.org/10.1007/978-3-642-00234-2_1

Ekhtiari, M., & Zandieh, M. (2022). A Multi-Objective Bi-Level Stochastic Programming for Water Sustainable Supply and Allocation Problem. Journal of Industrial and Systems Engineering, 14(2), 1-31.

Fontela, E., & Gabus, A. (1976). The DEMATEL Observer. Battelle Geneva Research Center, Geneva, Switzerland.

Freitas, M.B.C., Xavier, A., Fragoso, R., & Antunes, C. (2022). A Composite Indicator to Measure Sustainable Water use in Portugal: A Compromise Programming Approach. Journal of Environmental Management, 311. https://doi.org/10.1016/j.jenvman.2022.114791

Fu, C., Chang, W., & Yang, S. (2020). Multiple Criteria Group Decision-Making Based on Group Satisfaction. Information Science, 518, 309-329. https://doi.org/10.1016/j.ins.2020.01.021

Gabus, A., & Fontela, E. (1972). World Problems, an Invitation to Further Thought Within the Framework of DEMATEL. Battelle Geneva Research Center, Geneva, Switzerland.  

Golany, B., & Kress, M. (1993). A Multicriteria Evaluation of Methods for Obtaining Weights from Ratio Scale Matrices. European Journal of Operations Research, 69, 210-220. https://doi.org/10.1016/0377-2217(93)90165-J

Huang, I.B., Keisler, J., & Linkov, I. (2011). Multi-Criteria Decision Analysis in Environmental Science: Ten Years of Applications and Trends. Science of the Total Environment, 409(19), 3578-3594. https://doi.org/10.1016/j.scitotenv.2011.06.022

Humphreys, P., & Jones, G. (2006). The Evolution Of Group Decision Support Systems To Enable Collaborative Authoring of Outcomes. World Futures, 62 (3), 193-222. https://doi.org/10.1080/02604020500509546

Jarraya-Horriche, F., Brahim, H.B., & Slama, F. (2022). The Compromise Programming Method to Choose Artificial Recharge Sites in Mornag Aquifer (Tunisia). Arab Journal of Geosciences 15, 1418. https://doi.org/10.1007/s12517-022-10695-1

Khademi, M., Sharifi, S., Salimi,M., Pansera, B.A., & Ferrara, M. (2022). An Interval-Stochastic Compromise Programming for Urban Water Resource Allocation. AAPP - Atti della Accademia Peloritana dei Pericolanti Classe di Scienze Fisiche, Matematiche e Naturali, 100(1), A4 (2022).

Kou, G., & Lin, C. (2014). A Cosine Maximization Method for the Priority Vector Derivation in AHP. European Journal of Operations Research, 235, 225-232. https://doi.org/10.1016/j.ejor.2013.10.019

Marzieh, M., Kourosh, B., Hossein, Y., & Sina, Z. (2021). A Scenario-Based Management of Water Resources and Supply Systems Using a Combined System Dynamics and Compromise Programming Approach. Water Resources Management (In Press). https://doi.org/10.1007/s11269-021-02942-z

Mikhailov, L. (2000). A Fuzzy Programming Method for Deriving Priorities in the Analytic Hierarchy Process. Journal of Operations Research Society, 51, 341-349. https://doi.org/10.1057/palgrave.jors.2600899

Mikhailov, L., & Singh, M.G. (1999). Comparison Analysis of Methods for Deriving Priorities in the Analytic Hierarchy Process. IEEE SMC'99 Int. Conf. on Systems, Man, and Cybernetics, 1, 1037-042. https://doi.org/10.1109/ICSMC.1999.814236

Moreno-Jimenez, J., Aguaron, J., & Escobar, M.T. (2008). The Core of Consistency in AHP-Group Decision Making. Group Decision and Negotiation, 17(3), 249-265. https://doi.org/10.1007/s10726-007-9072-z

Romero, C., & Rehman, T. (2003). Multiple Criteria Analysis for Agricultural Decisions (ed. by Ch. 5: Compromise Programming. Developments in Agricultural Economics 11, 63-78. https://doi.org/10.1016/S0926-5589(03)80007-9

Saaty, T.L. (1980). The Analytic Hierarchy Process. McGraw-Hill, New York. https://doi.org/10.21236/ADA214804 

Sarband, E.M., Shahab, A., Attari, J., & Ebrahimi, K. (2021). Adaptation of a Compromise Programming Approach for Evaluating the Localized Impacts of Water Allocation. Hydrological Sciences Journal, 66(8), 1275-1287. https://doi.org/10.1080/02626667.2021.1912755   

Si, S.L., You, X.Y., Liu, H.C., & Zhang, P. (2008). DEMATEL Technique: A Systematic Review of the State-of-the-art Literature on Methodologies and Applications. Mathematical Problems in Engineering. https://doi.org/10.1155/2018/3696

Srdjevic, B. (2007). Linking Analytic Hierarchy Process and Social Choice Methods to Support Group Decision-Making in Water Management. Decision Support Systems, 42(4), 2261-2273.  https://doi.org/10.1016/j.dss.2006.08.001

Srdjevic, B., Medeiros, Y. D. P., & Srdjevic, Z. (2022). Empowering Small Stakeholder Groups in Selecting a Long-Term Water Management Plan. Water Policy, 24(7), 1208-1222. https://doi.org/10.2166/wp.2022.075

Srdjevic, B., Srdjevic, Z., & Zoranovic T. (2002). PROMETHEE, TOPSIS, and CP in Multicriteria Decision Making in Agriculture. Annals of Scientific Work 26 (1), 5-23.   

Srđevic, B. (2005). Combining Different Prioritization Methods in the Analytic Hierarchy Process Synthesis. Computers and Operations Research, 32 (7), 1897-1919. https://doi.org/10.1016/j.cor.2003.12.005

Srđević, B., Srđević, Z., Ilić, M., & Ždero, S. (2021). Group Model for Evaluating the Importance of Ramsar Sites in Vojvodina Province of Serbia. Environment, Development and Sustainability, 23, 10892-10909. https://doi.org/10.1007/s10668-020-01093-2

Srđević, Z., Srđević, B., Suvočarev, K., & Galamboš, L. (2020). Hybrid Constructed Wetland Selection as a Group Decision-Making Problem. Water Resources Management, 34, 295-310. https://doi.org/10.1007/s11269-019-02444-z

UNWTO (2012). Secretariat of the Ramsar Convention on Wetlands. Gland, Switzerland & World Tourism Organization.

Zeleny, M. (1982). Multiple Criteria Decision Making. New York: McGraw-Hill.