Multicriteria Group Decision-Making Under Uncertainty Using Interval Data and Cloud Models
MMulticriteria Group Decision-Making Under Uncertainty Using Interval Data and Cloud Models
Abstract
In this study, we propose a multicriteria group decision-making (MCGDM) algorithm under uncertainty where data is collected as intervals. The proposed MCGDM algorithm aggregates the data, determines the optimal weights for criteria and ranks alternatives with no further input. The intervals give flexibility to experts in assessing alternatives against criteria and provide an opportunity to gain maximum information. We also propose a novel method to aggregate experts’ judgements using cloud models. We introduce an experimental approach to check the validity of the aggregation method. After that, we use the aggregation method for an MCGDM problem. Here, we find the optimal weights for each criterion by proposing a bi-level optimisation model. Then, we extend the technique for order of preference by similarity to ideal solution (TOPSIS) for data based on cloud models to prioritise alternatives. As a result, the algorithm can gain information from decision-makers with different levels of uncertainty and examine alternatives with no more information from decision-makers. The proposed MCGDM algorithm is implemented on a case study of a cybersecurity problem to illustrate its feasibility and effectiveness. The results verify the robustness and validity of the proposed MCGDM using sensitivity analysis and comparison with other existing algorithms.
Keywords:
Group Decision-Making, Cloud Model, Bi-level Optimisation, Interval-valued Data, Uncertainty. Introduction
Decision making is an everyday task in human life and it becomes challenging when it includes multiple criteria [1, 2]. Multicriteria decision-making (MCDM) is an important area in decision-making theory [3]. Decisions ideally should not be made by individuals especially when the consequences affect a group of people. In addition, decision-making can often be improved by involving more than one expert [4]. The aim of multicriteria group decision-making (MCGDM) is to find the best option using preferences expressed by a set of decision-makers against a set of criteria [5, 6]. It normally is difficult for decision-makers to give absolutely certain values when examining the options. This difficulty comes from the nature of human thinking and the complication of the options and criteria [7]. Capturing uncertainty in decision-making has been getting much interest recently. There are two principal sources of uncertainty when collecting opinions 1) inter-expert uncertainty which is the variation among different decision-makers, 2) intra-expert uncertainty which comes from change of the mind of an individual decision-maker on the same situations [8]. One way to capture the opinions of decision-makers with their associated uncertainty are linguistic variables allowing computing with words [9]. This concept has attracted much interest and many techniques have been developed to deal with opinions which are expressed by words [10]. In these techniques, the words are typically encoded into quantity using triangular fuzzy numbers [11], generalized trapezoidal fuzzy numbers [12, 13], type-2 fuzzy sets [14], hesitant fuzzy sets [15] or cloud models [16, 17]. Then, quantitative analyses are applied, and the results are decoded into recommendations. However, in these techniques, often limited linguistic terms are offered to decision-makers to express their opinion. So, there is still restricted flexibility for decision-makers in terms of capturing their uncertainty. This issue has been addressed as a limitation of studies using these techniques in [18],
Hadi A. Khorshidi and Uwe Aickelin .e. the linguistic terms are required to be limited and pre-defined, but providing more options may improve the performance of decision-making. In these techniques, each linguistic term is translated into a specific quantity. Whereas in reality, people may have different perceptions over a linguistic term in different situations [19]. Also, in an investigation on Analytic Hierarchy Process (AHP), researchers show that Fuzzy AHP by translating the 9-level evaluation into fuzzy number does not introduce much difference [20]. The proposed MCGDM is firstly motivated to overcome the limitations of studies using linguistic terms and give more flexibility to decision-makers to express their opinions. In this study, we use intervals to capture uncertainty in decision-makers’ opinions. Using intervals, decision-makers can rate alternatives based on criteria flexibly and express their level of uncertainty to the precise extent they want. So, the proposed method does not give limited linguistic terms to decision-makers and does not need encoding of the linguistic terms into limited uncertain quantities. As a result, we gain maximum information from surveys. In several studies that used interval-values for decision-making such as [8, 21], it is assumed that experts’ ratings are uniformly distributed within the interval. However, in real world scenarios, experts do not have a similar level of uncertainty for the whole range of the interval. Moreover, these intervals are a better fit for uncertain data sourced from measurement errors and random observations. Intervals with Gaussian membership functions can be a better representation of experts’ ratings. So the second motivation for the proposed MCGDM method is to capture uncertainty of the intervals using Gaussian membership functions. To address this motivation, we need to use cloud models to aggregate interval-valued data collected from experts.. Cloud models are introduced and have been extensively studied as a transformation model of uncertainty between the qualitative concept and the quantitative description [22]. In this study, we intend to investigate whether we can propose a method, using cloud models to aggregate interval-valued data, which is both precise in capturing information and relatively simply formulated using less computation time. Thus, the proposed method can provide an opportunity for a faster and more dynamic aggregation which is useful for many applications such as online ratings. Another limitation of existing MCGDM algorithms is that the weights for criteria and/or decision-makers should be known [7, 23]. For many MCGDM problems, decision-makers need to give weights to criteria separately [3, 7, 23]. The third motivations for the proposed MCGDM is to assign the weights automatically through the aggregation process. Thus, we introduce a bi-level optimisation model which can derive the weights for criteria based on the interval-valued data collected from decision-makers. In our proposed MCGDM algorithm, decision-makers can express their opinions flexibly with different levels of uncertainty, and the alternatives are examined with no further information from decision-makers. Preliminaries
Interval-valued Data
There are many situations in real life where the use of interval-valued data is more suitable than single crisp values. These situations occur due to different reasons. For instance, some phenomena cannot be expressed by a single value, e.g., the number of times that a patient smokes in a day. Therefore, we require some tolerance in measuring attributes especially in medical and engineering data. Moreover, often there are several observations for an attribute, e.g., testing blood sugar of a patient on different days or a group of people rating their online shopping’s experience [24, 25]. Interval-valued data have been studied as an efficient representation for uncertain and imprecise information for different applications such as modelling survey data [21], prediction [26] and clustering [27]. There have been studies [14, 21] to aggregate interval-valued data into type-2 fuzzy sets. These studies assume that intervals have a uniform distribution. In addition, the aggregation process may require a large amount of computation and memory since the number of parameters is proportional to the number of surveyed intervals [28]. Here, we describe the aggregation method called interval agreement approach (IAA) [21]. Then, we use it in our comparative investigation. Suppose I=[a, ] is an interval and we have a set of intervals as
ℐ ={𝐼 ! , 𝐼 " , . . . , 𝐼 } . The membership function of the set ( 𝜇 ℐ ) which aggregates intervals based on where they are in agreement is obtained using Eq. (1). 𝜇 ℐ = ∑ 𝑧 " /&⋃ ⋃ . .. ⋃ &𝐼 ! ∩. . .∩ 𝐼 " , $ " % " &’$("&) $ % ! &’$("&’ ! %’ , $"%’ , (1) where 𝑧 % = 𝑖/𝑛 is the degree of membership and is equal to one for the values in which all the intervals are in agreement. To simplify Eq 1 for coding, let 𝐸 ℐ ={𝑒 ! , 𝑒 " , . . . , 𝑒 " } be the set of the sorted values of endpoints, and 𝐹 ℐ = {𝑓 ! , 𝑓 " , . . . , 𝑓 " } be a set of indicators where 𝑓 % are +1 and -1 if 𝑒 % is a lower and upper endpoints respectively [8]. So, the membership function can be obtained using Eq. (2). 𝜇 ℐ (𝑒) = 0 1/𝑛, 𝑒 = 𝑒 ’ 𝜇 ℐ (𝑒 "(’ ) + 𝑓 " /𝑛, 𝑒 = 𝑒 " 𝑎𝑛𝑑 𝑖 = 2, . . .2𝑛 𝜇 ℐ (𝑒 " ), 𝑒 " ≤ 𝑒 < 𝑒 "&’ 𝑎𝑛𝑑 𝑖 = 1, . . . ,2𝑛 − 1 (2) Cloud Models
A cloud model is a cognitive model to represent uncertainty. It contains the properties of probability distributions and fuzzy membership functions. The cloud model reflects the correlation between randomness and fuzziness to construct a map between quality and quantity. The distribution of membership for a cloud model within the domain is named as a membership cloud or simply a cloud. The cloud is composed of cloud drops, in a way that the higher certainty degree of a cloud drop indicates that the cloud drop has higher probability of appearance [22, 23]. There are similarities between cloud models and type-2 fuzzy sets. They both have been developed to add a higher level of uncertainty to membership functions. However, they are different in terms of fundamental theory and motivations to represent the uncertainty. Type-2 fuzzy sets capture the uncertainty of membership functions by defining upper and lower membership functions and use analytic mathematical methods to analyse the uncertainty. Cloud models use discrete cloud drops to express the uncertainty motivated by the random distribution of samples, without defining a membership function. Cloud models can effectively integrate statistics with fuzzy logic (you can refer to [29] for more details on a comparative investigation between these two). The advantage of cloud models is that a cloud can be characterized by only three parameters, which means low computation and memory requirements, while type-2 fuzzy sets have higher computational complexity [28, 29]. These three parameters are expectation ( Ex ), entropy ( En ) and hyper-entropy ( He ). Ex is the centre value of the cloud drops in a cloud model. In other words, it is the representative of the cloud model with the highest probability of appearance. En measures the uncertainty and represents the domain of the cloud. He quantifies the dispersion of cloud drops and the randomness of the membership. In other words, it is the uncertainty degree of the entropy [23, 30]. Normal cloud models are based on a normal distribution (probability theory) and a Gaussian membership function (fuzzy logic) [30]. Normal cloud models have been used successfully in different applications such as knowledge discovery [31], image processing [32], decision making [33], food industry [34], risk evaluation [23]. Definition 1 [22]: Let U be the universe of discourse and T be a concept in U . If 𝑥 ∈ 𝑈 is a random instance of concept T , which satisfies 𝑥~𝑁 6𝐸𝑥, 𝐸𝑛 ′ " , 𝐸𝑛 ′ ~𝑁(𝐸𝑛, 𝐻𝑒 " ) and certainty degree that x belongs to concept T is calculated by Eq. (3). 𝑦 = 𝑒 ( (& ′ $ , (3) So, a cloud drop can be represented as (𝑥, 𝑦) and the normal cloud would be 𝑦< = (𝐸𝑥, 𝐸𝑛, 𝐻𝑒) . Once the three parameters of a normal cloud model are determined, we can generate the normal cloud [30] using Algorithm 1. Algorithm 1:
Normal cloud generator (CG)
Input:
Parameters Ex , En , He and number of cloud drops N Output: N cloud drops and their certainty degree. 1. for i from 1 to N do 𝐸𝑛 *" ← generate a random value which follows a Normal distribution with mean En and variance 𝐻𝑒 ) 𝑥 " ← generate a random value which follows a Normal distribution with mean Ex and variance 𝐸𝑛 *) 𝑦 " ← calculate the certainty degree /* Using Eq. (3) */ end for Fig. 1 shows a normal cloud and its numerical characteristics which is generated by (0, 1, 0.1) and 5000 cloud drops. Figure 1]
Definition 2 [23, 35]: Suppose a normal cloud 𝑦< =(𝐸𝑥, 𝐸𝑛, 𝐻𝑒) in the domain of U and a positive non-zero value 𝜆 , then their multiplication is as Eq. (4). 𝜆𝑦C = &𝜆𝐸𝑥, √𝜆𝐸𝑛, √𝜆𝐻𝑒,, (4) Definition 3 [16, 23]: Given two normal clouds 𝑦< ! = (𝐸𝑥 ! , 𝐸𝑛 ! , 𝐻𝑒 ! ) and 𝑦< " = (𝐸𝑥 " , 𝐸𝑛 " , 𝐻𝑒 " ) in the domain of U , these two clouds can be compared through as follows. • Convert the clouds into interval-values as 𝑦< ! ≈A𝑎, 𝑎CD and 𝑦< " ≈ A𝑏, 𝑏CD , where 𝑎 = 𝐸𝑥 ! − 3𝐸𝑛 ! , 𝑎C = 𝐸𝑥 ! + 3𝐸𝑛 ! , 𝑏 = 𝐸𝑥 " − 3𝐸𝑛 " , 𝑏C = 𝐸𝑥 " +3𝐸𝑛 " . • Calculate 𝑆 &’ + ,&’ , using Eq. (5). 𝑆 -. ! ,-. $ = 2&𝑎F − 𝑏, − &𝑎F − 𝑎 + 𝑏F − 𝑏, , (5) • Use the following rules for the comparison. 1. If 𝑆 &’ + ,&’ , > 0 , then 𝑦< ! > 𝑦< " ; 2. If 𝑆 &’ + ,&’ , = 0 and 𝐸𝑛 ! < 𝐸𝑛 " , then 𝑦< ! > 𝑦< " ; 3. If 𝑆 &’ + ,&’ , = 0 , 𝐸𝑛 ! = 𝐸𝑛 " and 𝐻𝑒 ! < 𝐻𝑒 " , then 𝑦< ! > 𝑦< " ; 4. If 𝑆 &’ + ,&’ , = 0 , 𝐸𝑛 ! = 𝐸𝑛 " and 𝐻𝑒 ! = 𝐻𝑒 " , then 𝑦< ! = 𝑦< " . Definition 4 : The distance between two normal clouds 𝑦< ! = (𝐸𝑥 ! , 𝐸𝑛 ! , 𝐻𝑒 ! ) and 𝑦< " =(𝐸𝑥 " , 𝐸𝑛 " , 𝐻𝑒 " ) is measured as Eq. (6). 𝑑(𝑦C ’ , 𝑦C ) ) = H|𝐸𝑥 ’ − 𝐸𝑥 ) | ) + |𝐸𝑛 ’ − 𝐸𝑛 ) | + |𝐻𝑒 ’ − 𝐻𝑒 ) |, (6) The distance measure should meet the following conditions [7]. 1. 𝑑(𝑦< ! , 𝑦< " ) ≥ 0 ; 2. 𝑑(𝑦< ! , 𝑦< " ) = 0 if and only if 𝑦< ! = 𝑦< " ; 3. 𝑑(𝑦< ! , 𝑦< " ) = 𝑑(𝑦< " , 𝑦< ! ) ; 4. For any normal cloud 𝑦< ) = (𝐸𝑥 ) , 𝐸𝑛 ) , 𝐻𝑒 ) ) in U , 𝑑(𝑦< ! , 𝑦< " ) ≤ 𝑑(𝑦< ! , 𝑦< ) ) + 𝑑(𝑦< ) , 𝑦< " ) . Proof : Clearly, the proposed distance measure in Eq. (6) meets the first three conditions. We prove that the proposed distance measure also meets the fourth condition as follows. 𝑑(𝑦$ ! , 𝑦$ " ) = (|𝐸𝑥 ! − 𝐸𝑥 " | " + |𝐸𝑛 ! − 𝐸𝑛 " | + |𝐻𝑒 ! − 𝐻𝑒 " | =(|𝐸𝑥 ! − 𝐸𝑥 + 𝐸𝑥 − 𝐸𝑥 " | " + |𝐸𝑛 ! − 𝐸𝑛 + 𝐸𝑛 − 𝐸𝑛 " | + |𝐻𝑒 ! − 𝐻𝑒 + 𝐻𝑒 − 𝐻𝑒 " | ≤((|𝐸𝑥 ! − 𝐸𝑥 | + |𝐸𝑥 − 𝐸𝑥 " |) " + |𝐸𝑛 ! − 𝐸𝑛 | + |𝐸𝑛 − 𝐸𝑛 " | + |𝐻𝑒 ! − 𝐻𝑒 | + |𝐻𝑒 − 𝐻𝑒 " | ≤(|𝐸𝑥 ! − 𝐸𝑥 | " + |𝐸𝑛 ! − 𝐸𝑛 | + |𝐻𝑒 ! − 𝐻𝑒 | + |𝐸𝑥 − 𝐸𝑥 " | " + |𝐸𝑛 − 𝐸𝑛 " | + |𝐻𝑒 − 𝐻𝑒 " | ≤(|𝐸𝑥 ! − 𝐸𝑥 | " + |𝐸𝑛 ! − 𝐸𝑛 | + |𝐻𝑒 ! − 𝐻𝑒 | +(|𝐸𝑥 − 𝐸𝑥 " | " + |𝐸𝑛 − 𝐸𝑛 " | + |𝐻𝑒 − 𝐻𝑒 " | = 𝑑(𝑦$ ! , 𝑦$ ) + 𝑑(𝑦$ , 𝑦$ " ) Aggregation
We use interval values to collect the decision-makers’ opinions. These values give decision-makers flexibility to express their opinions, i.e., allow for a degree of uncertainty. Once the values are collected, we need an aggregation method to create a formal representative of the opinions and keep all the information. Here we use a cloud model to aggregate the interval values. This aggregation method can be used for both inter-expert and intra-expert uncertainties. The advantage of using cloud models is to provide a simpler representation for uncertainty (just with three parameters). So, less computations and memory are needed. We assume that each interval-value has a Gaussian membership function which we believe is a better assumption in comparison with the uniformly distributed membership function. Suppose we have an interval-value such as I=[a, b]. This value can be translated into a 99.73% confidence interval (CI) based on “ principle” of the Normal distribution using Eq. (7). J𝑥̅ = (𝑎 + 𝑏)/2𝜎 = (𝑏 − 𝑎)/6, (7) where 𝑥̅ is the mean and 𝜎 is the standard deviation. So, the interval-value of I can be converted into a pair as ( 𝑥̅ , 𝜎 ) which 𝑥̅ is an estimation for the interval and 𝜎 quantifies the confidence (uncertainty) of 𝑥̅ estimate. Example 1:
Suppose we have three decision-makers, named as DM1, DM2 and DM3 respectively. They give their opinion using interval-values as DM1=[3,4], DM2=[1,6] and DM3=[2,5]. The probability density functions (pdf) for these opinions after translation into Gaussian membership function are visualized in Fig. 2. As can be seen, the more the certain opinion, the smaller the range and the higher the density peak. [Figure 2]
Once the interval-values are translated, we use a cloud model to aggregate opinions. The parameters for the aggregated cloud model can be calculated as Eq. (8). ⎩⎪⎨⎪⎧𝐸𝑥 = ’0 ∑ 𝑥̅ 𝐸𝑛 = ’0 ∑ 𝜎 + R ’0 ∑ (𝑥̅ − 𝐸𝑥) )1 𝐻𝑒 = R ’0 ∑ (𝜎 − 𝐸𝑛) )1 (8) here Ex , En and He are the expectation, entropy and hyper-entropy of the aggregated cloud model respectively, and K is the number of decision-makers. The expectation of the aggregated cloud is calculated using the average of the estimation values of the intervals (Eq. 7) to represent the centre values of the cloud drops. The entropy value comes from two sources of uncertainties. First, the uncertainties of intervals obtained by Eq. (7). Second, the variation of intervals’ estimations from the centre of the cloud model. As a result, the aggregated cloud model can cover the whole entropy in the aggregation process. The hyper-entropy calculates how dispersed uncertainties are across decision-makers. For example 1, we can aggregate decision-makers’ judgement using Eq. (8). The aggregation finds a cloud model with Ex =3.5, En =0.5 and He =0.27. Fig. 3 visualizes the output of the aggregation process. To achieve this, we firstly calculated probability values of each decision-maker based on their normal distribution using 𝑒 * (3435),,7, (colored lines). Then, we generated 250 cloud drops based on the aggregated cloud model (black circles). As can be seen, the generated drops are clustered around the areas where decision-makers are in agreement as well as covering the uncertainty of decision-makers. [Figure 3] We also created more examples to show how the aggregation method works for different situations. The details for these examples and the aggregation results are shown in Table 1, and they are visualized in Fig. 4. [Table 1]
In example 2, the cloud drops are concentrated around the middle point and scattered across the domain based on decision-makers’ uncertainty. In example 3, the hyper-entropy is zero and the cloud model is a normal distribution as all decision-makers give the same intervals. The uncertainty of decision-makers in examples 4 and 5 is the same. However, the aggregated entropy and hyper-entropy of example 5 is lower due to the overlap between intervals (decision-makers are more in agreement). Examples 6 and 7 are instances where decision-makers have different estimations and different uncertainty levels. So, they have high aggregated entropy and hyper-entropy values. However, the cloud model in example 7 has a lower hyper-entropy as two of decision-makers are more in agreement. [Figure 4]
Experiments
To validate the performance of the proposed aggregation method, we design some appropriate experiments. In these experiments, we simulate different situations where the number of decision-makers ( d ) varies from 2 to 10. For each situation, we generate P aggregation problems. In each problem, d interval-values are generated randomly. Firstly, we estimate the parameters of the Gaussian membership function equivalent with each interval-value using Eq. (7). Then, we generate R random values using the parameters for each distribution. We create a pool of randomly generated values and calculate the descriptive statistics such as mean, quartile 1, median (quartile 2) and quartile 3 for the pool. Secondly, we aggregated the interval-values into a cloud model using Eq. (8). Based on the aggregated parameters, we generate 𝑅 × 𝑑 random values, and similarly find means and quartile values. Finally, we calculate the similarity between the two inter-quartile ranges (IQR) using a similarity measure based on overlapping ratios ( 𝑆 +, ) [36] formulated as Eq. (9). Note that IQR denotes the range between quartiles 1 and 3. 𝑆 &𝐼𝑄𝑅 : , 𝐼𝑄𝑅 ; , = min X =>?9 * ∩>?9 + ==>?9 * = , =>?9 * ∩>?9 + =|>?9 + | Y, (9) where S𝐼𝑄𝑅 - ∩ 𝐼𝑄𝑅 . S is the size of the intersection between two IQRs, S𝐼𝑄𝑅 - S and |𝐼𝑄𝑅 . | are the size of the IQR of the pool and the cloud respectively. The higher the value of 𝑆 +, , the more the similarity of two IQRs. The steps to implement the experiments are outlined in Algorithm 2. Algorithm 2:
Aggregation experiment
Input:
Constant values for 𝐷 , 𝑃 and 𝑅 Output:
𝐷 − 1 matrices ( 𝑂 B ) includes quartiles and 𝑆 𝐷 ←
Upper bound for number of decision-makers;
𝑃 ←
Number of problems;
𝑅 ←
Number of random values 2. for d from 2 to D do 𝑂 B ← An empty matrix with P rows and 9 columns 4. for p from 1 to P do for q from 1 to d do 𝐼 C ← Generate an interval-value randomly 𝑁𝑜𝑟𝑚 > , ← Find Gaussian membership function for 𝐼 C 𝑃𝑜𝑜𝑙 ←
Generate R random values using 𝑁𝑜𝑟𝑚 > , end for 𝑂 B ← Find mean and quartiles of
𝑃𝑜𝑜𝑙 Aggregate the interval-values 12.
𝐶𝑙𝑜𝑢𝑑 ←
Generate
𝑅 × 𝑑 random values using the aggregated cloud model /* Using Algorithm 1 */ 𝑂 B ← Find mean and quartiles of
𝐶𝑙𝑜𝑢𝑑 𝑂 B ← Calculate 𝑆 /* Using Eq. (9) */ 15. end for end for For this experiment, we set the inputs as
𝐷 = 10 , 𝑃 = 100 and
𝑅 = 50 . Once we have the results, we test the difference between the mean values of the pool and the aggregated cloud using paired t-tests. We checked the normality of data before applying this parametric test. The outputs of statistical tests are shown in Table 2. [Table 2]
The statistical test results show that we cannot reject the null hypothesis (true difference in mean is equal to zero) for mean and quartile 3 in all cases as p-values are greater than 0.05. This proves that the aggregation method can achieve similar mean values. Fig. 5 represents the results of the calculation of 𝑆 +, for each problem within each situation. The red dot denotes the mean of 𝑆 +, and the red line shows an interval width of two standard deviations. We can see the performance of the aggregation method is robust by changing the number of decision-makers. [Figure 5] We also investigate the performance of the aggregation method through a simulation approach. The parameters of a cloud model can be derived from cloud drops in the cloud. This transformation from the cloud drops to the cloud’s parameters is called backwards cloud generator ( 𝐶𝐺 *! ). There are two types of algorithm for backwards cloud generation for situations where the certainty degree is known [22]. However, we do not know the degree of certainty and hence require a different approach. The backwards cloud generator without knowing the certainty degree is described in Algorithm 3. Algorithm 3:
Backwards normal cloud generator ( 𝐶𝐺 (’ ) Input: 𝑥 " values of N cloud drops Output:
Parameters Ex , En , He of the cloud model 1. 𝐸𝑥 ← calculate the mean of 𝑥 " 𝑆 ) ← calculate the variance of 𝑥 " 𝐸𝑛 ← calculate R D) × ’E ∑ |𝑥 " − 𝐸𝑥| E"%’ 𝐻𝑒 ← calculate
H|𝑆 ) − 𝐸𝑛 ) | In other words, we use the generated pool (in Algorithm 2, steps 3-9) as the input for Algorithm 3 and derive the cloud’s parameters. So, we obtain the cloud parameters using simulation. However, we need much more computation and memory. Consequently, we can obtain the quartiles and calculate 𝑆 +, from Algorithm 2. Fig. 6 compares the average of 𝑆 +, calculated within for each scenario. [Figure 6] As can be seen from Fig. 6, both the proposed aggregation method and the backwards cloud generator can retain a high percentage of information from the pool. This shows that using cloud models is a suitable way to aggregate information from interval-valued data. In addition, the results show that the proposed aggregation method performs better than the 𝐶𝐺 *! in almost all situations (except when number of decision-makers is 5). Moreover, the proposed aggregation method can simply estimate the parameters of the cloud model from the interval-valued data using Eq. (8), while the 𝐶𝐺 *! needs a pool of cloud drops to derive the parameters. Overall, the proposed aggregation method performs well in capturing uncertainty in the opinions of decision-makers, which is very helpful in MCGDM. In addition, we compare the performance of the proposed aggregation method with other non-cloud models. Thus, we can ascertain how well the proposed method can gain information from intervals having Gaussian membership functions. We perform the comparison with two other methods. For the first alternative, we aggregate the intervals into an interval with Gaussian membership function by averaging the mean and standard deviation values. In other words, the intervals are aggregated into a type-1 fuzzy set. So, this aggregation does not consider the uncertainty of membership functions. The second method is IAA [21] which is described in section 2.1. [Figure 7] The comparison results are presented in Fig. 7. The results show that our proposed aggregation method performs better than two other methods by resulting in higher similarity values with generated inputs. Especially, it performs much better than type-1 fuzzy set aggregation. The results confirm that the proposed aggregation method can capture the uncertainty across the membership functions well. . Proposed MCGDM
In this section, a decision-making model using the proposed aggregation method (from section 3) is developed to prioritise alternatives when criteria are assessed under uncertainty by a group of decision-makers using interval values. For a MCGDM problem, suppose there are K decision-makers 𝐷𝑀 / (𝑘 = 1,2, … , 𝐾) , M criteria 𝐶 (𝑗 = 1,2, … , 𝑀) and N alternatives 𝐴 % (𝑖 = 1,2, … , 𝑁) . The proposed MCGDM model consists of three main stages. The first stage includes collecting uncertain data from decision-makers and aggregating decision-makers’ opinions. In the second stage, the decision matrix is constructed for alternatives and criteria. This is followed by weights for each criterion being determined by an optimisation model. In the third stage, the alternatives are prioritised using an extended TOPSIS method. Each stage consists of detailed steps which are described as follows. Stage 1: data collection and aggregation Evaluate alternatives using interval-values
Due to the complexity and vagueness of human thinking, it is hard for decision-makers to express their opinions without uncertainty. To give more flexibility to decision-makers, we allow the use of intervals to evaluate alternatives based on different criteria. Intervals provide an opportunity for decision-makers to express their opinions with different levels of uncertainty. Aggregate the opinions of decision-makers
The interval-values are aggregated and translated into cloud models using Eq. (8).
Stage 2: decision matrix construction Construct the decision cloud matrix
After aggregation, the information gained from decision-makers is translated into cloud models. For each criterion of each alternative, there will be a cloud model denoted by 𝑦< %0 = c𝐸𝑥 %0 , 𝐸𝑛 %0 , 𝐻𝑒 %0 d . Then, the decision matrix 𝑌f can be constructed as Eq. (10). 𝑌g = 𝐶 ’ 𝐶 ) … 𝐶 F 𝐴 ’ 𝐴 ) ⋮𝐴 E k𝑦C ’’ 𝑦C )’ 𝑦C ’) …𝑦C )) … 𝑦C ’F 𝑦C )F ⋮ ⋮ ⋱ ⋮𝑦C E’ 𝑦C E) … 𝑦C EF m, (10) Determine the weights of criteria
Once the decision cloud matrix is constructed, we can use the hyper-entropy of cloud models to give weights to criteria. The idea is to give less weight to the criteria with higher hyper-entropy (dispersion) across all alternatives. In other words, the criteria where decision-makers have higher level of agreement, are more important in ranking alternatives. This is because those criteria are more reliable to build the decision-making process than the criteria on which the collective opinion of the decision-makers has high uncertainty. These weights help the decision-making process becomes more robust and less sensitive to dispersed opinions of the decision-makers. In addition, finding the level of agreement of decision-makers has always been important in decision-making processes [19]. We use a bi-level optimisation model to find the weights of the criteria as Eq. (11). The optimal weights are obtained by considering all alternatives in one model. 𝑚𝑖𝑛 𝑚𝑎𝑥 " no𝐻𝑒 " 𝑤 − 𝐻𝑒 " - 𝑤 - oq (11) Subject to ∑ 𝑤 = 1 𝑤 ≥ 0, for all j where 𝑤 is the weight for j th criterion and 𝑗 refers to one of the criteria. This bi-level optimisation model can be transformed into a linear programming problem using the following formulations. 𝑚𝑖𝑛 𝜉 (12) Subject to 𝐻𝑒 " 𝑤 − 𝐻𝑒 " - 𝑤 - ≤ 𝜉 , for all 𝑖 and 𝑗 ≠ 𝑗 G 𝐻𝑒 " 𝑤 − 𝐻𝑒 " - 𝑤 - ≥ −𝜉 , for all 𝑖 and 𝑗 ≠ 𝑗 G ∑ 𝑤 = 1 𝑤 ≥ 0, for all j The optimal weights for criteria are obtained by solving problem in Eq. (12). We use the optimal weights to update the decision cloud matrix. Construct the weighted decision cloud matrix
The weighted decision cloud matrix
𝑌fh is constructed by multiplying the elements of the decision cloud matrix 𝑌f with the weights of criteria as Eq. (13). 𝑦Cw " = 𝑦C " × 𝑤 , for all 𝑖 and 𝑗 (13) here 𝑦 Once the weighted decision cloud matrix is available, we can use the extended TOPSIS method to rank alternatives. The extended TOPSIS can deal with cloud models as its values. Define ideal solutions In TOPSIS, we need to define positive and negative ideal solutions. The positive (negative) ideal solution is the best (worst) solution from all alternatives. To define these ideal solutions, we should check whether a criterion is a higher-the-better or lower-the-better one. Then, the ideal solutions can be obtained by Eqs. (15) and (16). 𝐴 & = ~max " 𝑦Cw " : 𝑗 ∈ 𝐽, min " 𝑦Cw " : 𝑗 ∈ 𝐽 ′ |𝑖 = 1,2, … , 𝑁(cid:132) =n𝑦Cw ’& , 𝑦Cw )& , … , 𝑦Cw F& q, (15) 𝐴 ( = ~min " 𝑦Cw " : 𝑗 ∈ 𝐽, max " 𝑦Cw " : 𝑗 ∈ 𝐽 ′ |𝑖 = 1,2, … , 𝑁(cid:132) =n𝑦Cw ’( , 𝑦Cw )( , … , 𝑦Cw F( q, (16) where 𝐴 and 𝐴 * are the positive and negative ideal solutions respectively, 𝐽 refers to the set of the higher-the-better criteria and 𝐽 ′ refers to the set of the lower-the-better criteria. Calculate distance from ideal solutions The distance between each alternative and the ideal solutions can be calculated using Eqs. (17) and (18). 𝑑 "& = ∑ 𝑑&𝑦Cw " , 𝑦Cw , , for all 𝑖 (17) 𝑑 "( = ∑ 𝑑&𝑦Cw " , 𝑦Cw , , for all 𝑖 (18) Determine the ranking score Once the distances from ideal solutions are calculated for all alternatives, a ranking score ( 𝑅𝑆 % ) can be formulated based on how much an alternative is close to the positive ideal solution and distant from the negative ideal solution as Eq. (19). 𝑅𝑆 " = B " B " & B ". , for all 𝑖 (19) The value of the ranking score is between 0 and 1. Bigger value for 𝑅𝑆 % denotes the higher rank for i th alternative. Case Study In this section, we implement the proposed MCGDM model on a real decision-making problem in the cyber-security area to demonstrate its applicability and effectiveness. A questionnaire survey was conducted on the difficulty of security components based on the judgement of a group of cyber-security experts [37, 38]. In this problem, a set of questions are used as criteria to rate 14 attack related security components ( 𝑁 = 14 ) by 38 experts ( 𝐾 = 38 ). The set of questions contains seven questions, which work as criteria ( 𝑀 = 7 ). Table 3 shows these questions and the derived criteria. Table 3 mentions whether a criterion is either a higher-the-better or a lower-the-better one, which are perceived from correlation with the overall difficulty in [38] and intuitively from the questions. [Table 3] Experts are asked to rate each component based on criteria using an interval between 0 and 100. Implementation of the Proposed MCGDM Once the data is collected, we can apply the proposed aggregation method to construct the decision cloud matrix. Table 4 shows the constructed decision cloud matrix. [Table 4] Now, we can determine the weights of criteria using the information in the decision cloud matrix by solving the bi-level optimisation model formulated in Eq. (12). The optimal weights are shown in Table 5. As can be seen, frequency has the highest weight, while interaction has the lowest weight. [Table 5] By determining the weights of criteria, we can construct the weighted decision cloud matrix. Table 6 shows the weighted constructed decision cloud matrix. [Table 6] To define positive and negative ideal solutions, we need to find the maximum and minimum cloud models for each criterion among the alternatives using Algorithm 4. lgorithm 4: Ideal solution finder Input: Set of cloud models for criterion j Output: Indices of the maximum and minimum cloud models for criterion j ( 𝑀𝑎𝑥 and 𝑀𝑖𝑛 ) 1. Convert the cloud models into interval-values as described in Definition 3 𝑀𝑎𝑥 ← 1 and 𝑀𝑖𝑛 ← 1 for i from 2 to N do Calculate 𝑆 -. /0& ,-. " /* Using Eq. (5) */ Compare 𝑦C HIJ and 𝑦C " /* Using 𝑆 -. /0& ,-. " and the rules in Definition 3 */ 6. If 𝑦C " > 𝑦C HIJ , 7. 𝑀𝑎𝑥 ← 𝑖 Then if 𝑦C " < 𝑦C HIJ , 9. Calculate 𝑆 -. /") ,-. " /* Using Eq. (5) */ 10. Compare 𝑦C H"$ and 𝑦C " /* Using 𝑆 -. /") ,-. " and the rules in Definition 3 */ 11. If 𝑦C " < 𝑦C H"$ , 12. 𝑀𝑖𝑛 ← 𝑖 end for Considering whether each criterion is higher-the-better or lower-the-better (demonstrated in Table 3), the ideal solutions are defined as below. 𝐴 ={(11.99,6.83,5.27), (6.01,10.97,9.68), (13.7,4.15,3.2), (4.33,9.69,7.98), (11.77,6.76,5.29), (11.95,5.29,4.02), (2.84,8.2,6.95) 𝐴 * ={(6.97,8.59,6.83), (10.86,5.22,4.25), (3.59,7.5,5.9), (13.25,3.49,2.62), (5.59,10.14,8.4), (6.87,9.69,7.75), (8.05,9.18,7.27) The distances of each alternative from positive and negative ideal solutions can be calculated using Eqs. (17) and (18). Finally, these distance values are used to calculate the ranking scores which can be used to prioritise the alternatives. The values for distances and ranking scores as well as ranking of alternatives are brought in Table 7. [Table 7] As can be seen, the components 𝐴 and 𝐴 stand at the bottom, which means they are mostly vulnerable against the attack. On the other hand, the components 𝐴 !" and 𝐴 have the first and the second lowest levels in the ranking respectively, which denotes that they are in lower risk of being successfully attacked. Comparative Analysis In this section, we perform three comparative analyses on the ranking results based on the case study data. The first comparative analysis is a sensitivity analysis. A key parameter that has impact on the ranking of alternative in our proposed algorithm is the distance measure. Thus, we apply a sensitivity analysis on the distance measure. The second analysis is to compare the ranking performance of the proposed MCGDM algorithm with other group decision-making algorithms which use interval-valued data. To identify the differences in the comparative analyses, we compute the ranking results and use a Spearman rank correlation test. This test is frequently used to determine the correlation between two sets of ranking values [39, 40].The test statistic is measured using Eq. (20). 𝑟 K = ;GL(9 ,9 )M M , (20) where 𝑅 and 𝑅 are two ranking vectors, 𝑐𝑜𝑣(. ) is the covariance of two vectors, and 𝜎 denotes the standard deviation of each vector. The value for 𝑟 is between -1 and 1 which denotes a complete negative and positive relationship between two ranking results, respectively. In other words, the higher the 𝑟 value, the more the similarity of two rankings. The third analysis is to compare the computational time of the proposed MCGDM algorithm with the other group decision-making algorithms using interval values. In this analysis, we replicate the MCGDM procedure for each algorithm 100 times and record the CPU time. Then, compare the algorithms’ computation using descriptive statistics and a statistical test. Sensitivity Analysis In this section, we analyse the sensitivity of the proposed MCGDM algorithm based on different distance measures. We consider two distance measures for cloud models previously introduced in the literature. One is proposed in [23] which treats each cloud model as a vector and finds the distance between two cloud models using Euclidean distance. Another one is proposed in [7] which is constructed based on Hamming distances. The comparison of ranking results is shown in Fig. 8. For each distance measure, we calculate the Spearman test statistic. The test statistics for the ranking difference of the proposed distance measure with distance measures [23] and [7] are 0.891 and 0.767 respectively. Figure 8] As can be seen in the ranking results, the components 𝐴 !" , 𝐴 , 𝐴 ) and 𝐴 " are ranked high and the components 𝐴 and 𝐴 rank at the bottom across different similarity measures. This show that the proposed MCGDM algorithm is robust in terms of using distance measures. Moreover, the Spearman rank-correlation statistics confirm this robustness. They denote that the ranking results from different similarity measures are strongly correlated and similar to the ranking results of the proposed MCGDM. Ranking Performance Comparison As previously mentioned, we compare our MCGDM algorithm with two other group decision-making algorithms [21, 41], which used interval-valued data as inputs, using the case study. In [41], the interval-valued data are firstly normalized. Then, the normalized data across decision-makers are aggregated into interval-valued intuitionistic fuzzy numbers (IVIFNs) using the mean and standard deviation values of the endpoints. so, there would be a decision matrix which the values are IVIFNs. After that, the weighted average IVIFN for each alternative is calculated. Finally, the alternatives are ranked based on the calculated weighted average. In [21], the interval-valued data (with uniform membership function) collected from decision-makers are aggregated into types-2 fuzzy sets (as described in section 2.1). Each FS is de-fuzzified by obtaining the centroid. The weighted average of the centroid of the criteria is used to prioritise the alternatives. Fig. 9 provides a comparison between ranking results of the algorithms. The test statistics for the ranking difference of the proposed MCGDM algorithm with algorithms [41] and [21] are 0.621 and 0.942 respectively. [Figure 9] The Spearman correlation statistics are different between the comparisons. In comparison with [41], the test statistic is low and the alternatives which are ranked at the top or bottom vary. Whereas, in comparison with [21], the correlation statistic is high, and also the components 𝐴 !" , 𝐴 ) and 𝐴 " are ranked at the top and the components 𝐴 and 𝐴 the ranked at the bottom in both rankings. So, the ranking results of the proposed algorithm is similar to [21] and is distant from [41]. By more investigations, We find out that there are several deficiency and drawback in the algorithm of [41]. We believe these drawbacks to be as follows. (1) In the normalization step, if the minimum value of the lower bounds of intervals is zero, the interval values are transformed into [0, 1] intervals. This transformation leads to the loss of a lot of information. (2) In the aggregation process, the assumption is that the mean value of the lower bounds for the normalized intervals (for each criterion and each alternative) is less than 0.5, which is not always true in reality including in the case study. So, the authors formulated the aggregation in a way to control that the mean plus standard deviation (for lower bounds) does not exceed 0.5. However, based on the previous assumption, they did not control the value of mean minus standard deviation. This value exceeds 0.5 several times in the aggregating decision-makers’ ratings, which leads to having an upper bound lower than the lower bound. In our implementation, we fixed the problem by applying a similar control. (3) There are similar assumptions and controls for the upper bound values, so that the mean value of upper bounds for the normalized intervals (for each criterion and each alternative) should be greater than 0.5. (4) Applying these controls lead to losing information during aggregation. On the other hand, the algorithm presented in [21] is valid for interval-valued data with uniform membership function. So, having similar ranking results with [21] and different ranking results from [41] is a validation evidence for our proposed MCGDM algorithm. Computational Time Comparison The statistics for computation time (in milliseconds) of each MCGDM algorithm after 100 times of running are presented in Table 8. [Table 8] The statistics show that the proposed MCGDM needs less CPU time for computations in average. Also, to check the significance of the difference among the algorithms, we used Kruskal-Wallis test which is a non-parametric test (as CPU time data records do not have Normal distribution). The test proves that computation times of the algorithms are significantly different. verall, there are advantages for the proposed MCGDM algorithm in comparison with two others as follows. 1. It does not have the deficiencies mentioned for [41] and its performance is validated through the comparative analysis. 2. Our algorithm has a built-in mechanism to find the optimal weights for criteria objectively and automatically. 3. The aggregation method is formulated in a way that needs less computations, to gain the information from the decision-makers. Conclusion In this paper, we propose a MCGDM algorithm under uncertainty with interval-valued data. In this algorithm, we propose a new aggregation method using cloud models and an extended version of TOPSIS method. The algorithm has an automated mechanism to derive the weights of criteria using the proposed bi-level optimisation model. This paper contributes to the MCGDM literature as listed below. • The proposed method gives more flexibility to decision-makers to express the uncertainty in their opinions. • Using cloud models for aggregation provides a simpler representation and therefore fewer computations and less memory usage. • There is no need for more inputs from decision-makers to weight criteria. We design an experimental setup to examine how well the aggregation method can retain the information from decision-makers. Also, we used the experimental setup to compare the performance of the aggregation method with other methods. Moreover, the algorithm is applied on a practical case of a cyber-security problem to prioritise security components based on their vulnerability against cyber-attacks. A comparative analysis of the ranking results for the proposed MCGDM algorithm is undertaken. The comparative analysis includes sensitivity analysis and comparisons with other existing algorithms. The results show that the aggregation method is robust over various numbers of decision-makers (experimented in section 3.1 and results presented in Fig. 5). Also, it performs well and retains information in comparison with other comparable methods (experimented in section 3.1 and results presented in Fig. 6) and performs better than other methods gaining information from interval-valued data with Gaussian membership function (experimented in section 3.1 and results presented in Fig. 7). The proposed MCGDM algorithm is robust in terms of distance measures (experimented in section 5.2.1 and results presented in Fig. 8). The ranking results of the algorithm are valid in comparison with the existing algorithms (experimented in section 5.2.2 and results presented in Fig. 9). The proposed MCGDM algorithm needs less computation time in comparison with the existing ones (experimented in section 5.2.3 and results presented in Table. 8). The assumption of Gaussian membership functions for intervals could be considered as a limitation of the proposed method. Even though we justified that this membership function is the most appropriate one for intervals, we suggest considering other membership functions or a combination of various membership functions for future research, . So, new aggregation methods should be developed accordingly. The performance of the aggregation methods can be examined using the proposed experimental design. Moreover, the proposed aggregation method can be used for ensemble learning using machine learning methods by considering the uncertainty level in their prediction. Furthermore, the proposed algorithm can be applied to various types of collected uncertain data other than expert opinions. As the interval-valued data for this algorithm are assumed to have Gaussian membership functions, the algorithm fits well to uncertain data which is sourced from measurement errors, learned prediction errors and random observations. The algorithm can be used for consensus problems when the experts are not humans but are intelligent machines or expert systems. References [1] Liao H, Wu X, Mi X, and Herrera F. An integrated method for cognitive complex multiple experts multiple criteria decision making based on ELECTRE III with weighted Borda rule. Omega . 2020;93 102052. [2] Aickelin U, Reps JM, Siebers PO, and Li P. Using simulation to incorporate dynamic criteria into multiple criteria decision-making. Journal of the Operational Research Society . 2018;69 (7):1021-1032. [3] Rezaei J. Best-worst multi-criteria decision-making method. Omega . 2015;53 49-57. Lan J, Sun Q, Chen Q, and Wang Z. Group decision making based on induced uncertain linguistic OWA operators. Decision Support Systems . 2013;55 (1):296-303. [5] Mohammadi M and Rezaei J. Bayesian best-worst method: A probabilistic group decision making model. Omega (United Kingdom). 2019. [6] Pérez IJ, Cabrerizo FJ, Alonso S, Dong YC, Chiclana F, and Herrera-Viedma E. On dynamic consensus processes in group decision making problems. Information Sciences . 2018;459 20-35. [7] Wang J-q, Peng J-j, Zhang H-y, Liu T, and Chen X-h. An Uncertain Linguistic Multi-criteria Group Decision-Making Method Based on a Cloud Model. Group Decision and Negotiation . 2015;24 (1):171-192. [8] Havens TC, Wagner C, and Anderson DT. Efficient modeling and representation of agreement in interval-valued data. in . 2017. [9] Zadeh LA. The concept of a linguistic variable and its application to approximate reasoning—I. Information Sciences . 1975;8 (3):199-249. [10] Herrera F, Alonso S, Chiclana F, and Herrera-Viedma E. Computing with words in decision making: foundations, trends and prospects. Fuzzy Optimization and Decision Making . 2009;8 (4):337-364. [11] Borovička A. New approach for estimation of criteria weights based on a linguistic evaluation. Expert Systems with Applications . 2019;125 100-111. [12] Khorshidi HA, Gunawan I, and Nikfalazar S. Application of Fuzzy Risk Analysis for Selecting Critical Processes in Implementation of SPC with a Case Study. Group Decision and Negotiation . 2015;25 (1):203-220. [13] Zhang WR, Knowledge representation using linguistic fuzzy relations . 1986, University of South Carolina. [14] Wu D, Mendel JM, and Coupland S. Enhanced Interval Approach for Encoding Words Into Interval Type-2 Fuzzy Sets and Its Convergence Analysis. IEEE Transactions on Fuzzy Systems . 2012;20 (3):499-513. [15] Peng JG and Xia G. A systematic fuzzy multi-criteria group decision-making approach for alternatives evaluation. Journal of the Operational Research Society . 2019;70 (9):1490-1501. [16] Wang J-q, Peng L, Zhang H-y, and Chen X-h. Method of multi-criteria group decision-making based on cloud aggregation operators with linguistic information. Information Sciences . 2014;274 177-191. [17] Li CB, Qi ZQ, and Feng X. A multi-risks group evaluation method for the informatization project under linguistic environment. Journal of Intelligent and Fuzzy Systems . 2014;26 (3):1581-1592. [18] Qin J, Xi Y, and Pedrycz W. Failure mode and effects analysis (FMEA) for risk assessment based on interval type-2 fuzzy evidential reasoning method. Applied Soft Computing . 2020;89 106134. [19] Navarro J, Wagner C, Aickelin U, Green L, and Ashford R. Measuring agreement on linguistic expressions in medical treatment scenarios. in . 2016. [20] Chan HK, Sun X, and Chung S-H. When should fuzzy analytic hierarchy process be used instead of analytic hierarchy process? Decision Support Systems . 2019;125 113114. [21] Wagner C, Miller S, Garibaldi JM, Anderson DT, and Havens TC. From Interval-Valued Data to General Type-2 Fuzzy Sets. IEEE Transactions on Fuzzy Systems . 2015;23 (2):248-269. [22] Li D and Du Y. Artificial intelligence with uncertainty . Boca Raton, FL: Chapman & Hall/CRC; 2007. [23] Liu H, Wang L, Li Z, and Hu Y. Improving Risk Evaluation in FMEA With Cloud Model and Hierarchical TOPSIS Method. IEEE Transactions on Fuzzy Systems . 2019;27 (1):84-95. [24] Ren Y, Liu YH, Rong J, and Dew R. Clustering interval-valued data using an overlapped interval divergence. Conferences in Research and Practice in Information Technology Series . 2009;101 35-42. [25] Blanco-Fernández A, Colubi A, and González-Rodríguez G, Linear regression analysis for interval-valued data based on set arithmetic: A review , in Studies in Fuzziness and Soft Computing . 2013. p. 19-31. [26] Chuang CC, Wang CM, and Li CW. Weighted linear regression for symbolic interval-values data with outliers. in Proceedings of the 2010 5th IEEE Conference on Industrial Electronics and Applications, ICIEA 2010 . 2010. [27] Rodríguez SIR and de Carvalho FAT, Fuzzy clustering algorithm based on adaptive euclidean distance and entropy regularization for interval-valued data , in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) . 2018. p. 695-705. [28] Yang X, Yan L, Peng H, and Gao X. Encoding words into Cloud models from interval-valued data via fuzzy statistics and membership function fitting. Knowledge-Based Systems . 2014;55 114-124. [29] Huang HC and Yang X. A Comparative Investigation of Type-2 Fuzzy Sets, Nonstationary Fuzzy Sets and Cloud Models. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems . 2016;24 (2):213-227. [30] Li D, Liu C, and Gan W. A new cognitive model: Cloud model. International Journal of Intelligent Systems . 2009;24 (3):357-375. [31] Li D, Han J, Shi X, and Chan MC. Knowledge representation and discovery based on linguistic atoms. Knowledge-Based Systems . 1998;10 (7):431-440. [32] Zhou Z. Cognition and removal of impulse noise with uncertainty. IEEE Transactions on Image Processing . 2012;21 (7):3157-3167. [33] Yang X, Yan L, and Zeng L. How to handle uncertainties in AHP: The Cloud Delphi hierarchical analysis. Information Sciences . 2013;222 384-404. [34] Wang M, Sun Y, Hou J, Wang X, Bai X, Wu C, Yu L, and Yang J. A comparison of food crispness based on the cloud model. Journal of Texture Studies . 2018;49 (1):102-112. [35] Wu Y, Chen K, Zeng B, Yang M, and Geng S. Cloud-based decision framework for waste-to-energy plant site selection – A case study from China. Waste Management . 2016;48 593-603. [36] Kabir S, Wagner C, Havens TC, Anderson DT, and Aickelin U. Novel similarity measure for interval-valued data based on overlapping ratio. in IEEE International Conference on Fuzzy Systems . 2017. [37] Miller S, Appleby S, Garibaldi JM, and Aickelin U. Towards a More Systematic Approach to Secure Systems Design and Analysis. International Journal of Secure Software Engineering . 2013;4 (1):11 - 30. [38] Miller S, Wagner C, Aickelin U, and Garibaldi JM. Modelling cyber-security experts’ decision making processes using aggregation operators. Computers and Security . 2016;62 229-245. [39] Parkan C and Wu M-L. Decision-making and performance measurement models with applications to robot selection. Computers & Industrial Engineering . 1999;36 (3):503-523. [40] Berenson M. Basic Business Statistics: Concepts and Applications . 4th ed. Pearson Education; 2015. Yue Z. A group decision making approach based on aggregating interval data into interval-valued intuitionistic fuzzy information. Applied Mathematical Modelling . 2014;38 (2):683-698.. 2014;38 (2):683-698.