On the role of review papers in the face of escalating publication rates - a case study of research on contaminants of emerging concern (CECs).

Abstract

In the past few decades, there has been a dramatic increase in scientific publications dealing with contaminants of emerging concern (CECs) and the escalating publication rate makes it close to impossible for individual researchers to get an overview of the field. Assuring the relevance and quality of the research conducted in any research field is a crucially important task. The rapidly increasing publication rates imply that review papers will play a progressively more central role to that end. The aim of the present paper is to critically assess whether reviews dealing with contaminants of emerging concern (CECs) are effective vehicles for a healthy dialogue about methodological weaknesses, uncertainties, research gaps and the future direction of the field. We carried out a tiered content-analysis of CEC review papers. Relevant papers were identified through searches in Web of Science (Clarivate), leading to the identification of 6391 original research papers of which 193 are review papers. We find that the majority of CEC reviews are written as if they are comprehensive, even though this clearly is not the case. A minority (~20%) take a critical-analytical approach to the reviewing task and identify weaknesses and research gaps. The following widespread tendencies in CEC research papers are commonly noted as concerning: to equate removal of CECs to decreased concentrations in the effluent ; to focus on parent substances and not concern oneself with degradation products; to focus on most commonly studied substances rather than those of most concern; to not deal with the corollary of our inability to detect or assess the risk for all substances, and to give insufficient attention to uncertainties and the unknown. Several critical-analytical reviews are among the highest cited, which suggests that they have the potential to function as effective vehicles for a healthy dialogue on these topics. On the other hand, it would appear that the concerns expressed in these reviews have a limited impact, as the same concerns are repeated over time. This might be due to a tendency among review authors to express their concerns implicitly, instead of clearly spelling them out. Our study suggests that CEC reviews presently fail to provide adequate and reliable guidance regarding the relevance and quality of research in the field. We argue that the overwhelming number of publications in combination with a lack of quality criteria for review papers are reasons to this failure: it is well documented that choices made during the reviewing process have a major impact on the outcome of a review. These choices include: search engine; the criteria used to include or exclude papers; the criteria used to assess the quality of the data generated in the research papers included; the criteria used for the choice of substances/ organisms/ technologies reported on. The lack of transparent procedures makes it very difficult, if not impossible, to assess the quality of the findings presented or to put those findings in context. In this light, it is noteworthy that criteria for a good review paper are rarely spelled out by peer-reviewed journals or included in instructions on scientific writing. The dramatic increase in publications is a challenge for the entire research community, particularly for research fields that are expected to provide policy-relevant data. We argue that only when peer-reviewed journals start specifying quality criteria for review papers, can such papers be relied upon to provide adequate and strategic guidance on the development of CEC research. We anticipate that our findings and conclusions are valid for many other research fields.