A meta-evaluation model on science and technology project review experts using IVIF-BWM and MULTIMOORA

Abstract

Abstract Meta evaluation theory and methods were used to evaluate the review experts of science and technology projects. Dozens of meta evaluation criteria can be found within two categories: the objective data of the experts review results, i.e. the coefficient of deviation, the Spearman rank correlation coefficient, the reliability coefficient of binary value; the subjective data of the experts, i.e., the degree of experts participation, the degree of review punctuality, the qualification of experts, and the degree of review seriousness. How these criteria impact each other has few been examined, and how to integrate the objective and subjective criteria need a comprehensive model with considering the fuzzy characteristics of the subjective criteria. This study targeted these two questions. An empirical study was adopted with hundreds of experts taking part in reviewing hundreds of Sci-Tech projects. The impacting relationships among the criteria were analyzed based on the empirical study. In order to deal with the intuitionistic fuzzy data on the subjective criteria and improve the estimation efficiency, an IVIF-BWM (best worst method under interval-valued intuitionistic fuzzy environment) was proposed by combining IVIF and the classical BWM to generate the importance weight for each criterion. The MULTIMOORA (multi-objective optimization by ratio analysis plus the full multiplicative form) was used to determine how to combine these criteria. At last, the proposed meta-evaluation model based on IVIF-BWM and MULTIMOORA was applied in a real case. The case study results supported the accuracy and reliability of the proposed model.