Cristian Scutaru

New quality and quantity indices in science (NewQIS): the study protocol of an international project

Benchmarking systems are important features for the implementation of efficacy in basic and applied sciences. These systems are urgently needed for many fields of science since there is an imbalance present between funding policies and research evaluation. Here, a new approach is presented with an international study project that uses visualisation techniques for benchmarking processes. The project is entitled New Quality and Quantity Indices in Science (NewQIS). The juxtaposition of classical scientometric tools and novel visualisation techniques can be used to assess quality and quantity in science. In specific, the tools can be used to assess quality and quantity of research activity for distinct areas of science, for single institutions, for countries, for single time periods, or for single scientists. Also, NewQIS may be used to compare different fields, institutions, countries, or scientists for their scientific output. Thus, decision making for funding allocation can be made more transparent. Since governmental bodies that supervise funding policies and allocation processes are often not equipped with an in depth expertise in this area, special attention is given to data visualisation techniques that allow to visualize mapping of research activity and quality.

Quality and quantity indices in science: use of visualization tools

Recent articles in EMBO reports have addressed the issues of decreasing research funding (Wolinsky, 2009) and the importance of research evaluation (Bornmann & Daniel, 2009). Both topics are directly connected to one other. In the light of stagnant or even decreasing private and public funding for research, precise rules for funding policies have become indispensable in order to identify good research projects and proposals. The use of benchmarking systems to assess the quality of research is therefore an important feature of implementing an efficient funding policy. Although individual scientometric and bibliometric methods can be used to analyse the research activities of faculties or single scientists, these techniques often do not provide a larger picture of scientific quality and quantity in relation to funding. Without the use of scientometric methods and other techniques to analyse whether existing funding schemes are successful or not, there will be a growing disrespect for funding policies among scientists. To address this, we present an example of how to visualize research quantity and quality across the European Union (EU). Reviews of European research funding and other comments make it clear that the EU needs to improve its research strategy (Fricker, 2007; Kingdon, 2006; Mayor, 2007; Watts, 2008). In this respect, a recent news report stated that “European science is not as intense as hoped” (Clery, 2009). Although research activities in the EU increased between 2000 and 2006, research spending as a …

Models of asthma: density-equalizing mapping and output benchmarking

Despite the large amount of experimental studies already conducted on bronchial asthma, further insights into the molecular basics of the disease are required to establish new therapeutic approaches. As a basis for this research different animal models of asthma have been developed in the past years. However, precise bibliometric data on the use of different models do not exist so far. Therefore the present study was conducted to establish a data base of the existing experimental approaches. Density-equalizing algorithms were used and data was retrieved from a Thomson Institute for Scientific Information database. During the period from 1900 to 2006 a number of 3489 filed items were connected to animal models of asthma, the first being published in the year 1968. The studies were published by 52 countries with the US, Japan and the UK being the most productive suppliers, participating in 55.8% of all published items. Analyzing the average citation per item as an indicator for research quality Switzerland ranked first (30.54/item) and New Zealand ranked second for countries with more than 10 published studies. The 10 most productive journals included 4 with a main focus allergy and immunology and 4 with a main focus on the respiratory system. Two journals focussed on pharmacology or pharmacy. In all assigned subject categories examined for a relation to animal models of asthma, immunology ranked first. Assessing numbers of published items in relation to animal species it was found that mice were the preferred species followed by guinea pigs. In summary it can be concluded from density-equalizing calculations that the use of animal models of asthma is restricted to a relatively small number of countries. There are also differences in the use of species. These differences are based on variations in the research focus as assessed by subject category analysis.

Tobacco Control: Visualisation of Research Activity Using Density-Equalizing Mapping and Scientometric Benchmarking Procedures

Background: Tobacco smoking continues to be a major preventable cause of death and disease and therefore tobacco control research is extremely important. However, research in this area is often hampered by a lack in funding and there is a need for scientometric techniques to display research efforts. Methods: The present study combines classical bibliometric tools with novel scientometric and visualizing techniques in order to analyse and categorise research in the field of tobacco control. Results: All studies related to tobacco control and listed in the ISI database since 1900 were identified by the use of defined search terms. Using bibliometric approaches, a continuous increase in qualitative markers such as collaboration numbers or citations were found for tobacco control research. The combination with density equalizing mapping revealed a distinct global pattern of research productivity and citation activity. Radar chart techniques were used to visualize bi- and multilateral research cooperation and institutional cooperation. Conclusions: The present study supplies a first scientometric approach that visualises research activity in the field of tobacco control. It provides data that can be used for funding policy and the identification of research clusters.

Analysis of research output parameters: density equalizing mapping and citation trend analysis.

BackgroundBurden of disease studies indicate major socio-economic burdens since many years. They should be used for the allocation of funding. However, imbalances are present in funding policies and therefore benchmarking becomes increasingly important in health services research.MethodsThe present study assessed benchmarking approaches. Using large data base analyses, research was analyzed for different health research output parameters. The fields of cardiovascular and respiratory medicine served as models to assess irregular patterns of health research. For visualization, density equalizing mapping procedures were used.ResultsSpecific areas of major research activity were identified for European countries and large differences were found. Spatial distribution of published items for cardiac and cardiovascular systems differed in comparison to the distribution for the respiratory system. In general, large countries dominated the overall number of published items. When qualitative measures such as citation analysis were assessed, differing results were achieved. In this category, mostly Scandinavian countries dominated.ConclusionThe present approach of comparative output benchmarking can be used to assess institutional operating figures at the national and international level and to analyze imbalances in health and research funding.

Scientometric analysis and combined density-equalizing mapping of environmental tobacco smoke (ETS) research.

Background Passive exposure to environmental tobacco smoke (ETS) is estimated to exert a major burden of disease. Currently, numerous countries have taken legal actions to protect the population against ETS. Numerous studies have been conducted in this field. Therefore, scientometric methods should be used to analyze the accumulated data since there is no such approach available so far. Methods and Results A combination of scientometric methods and novel visualizing procedures were used, including density-equalizing mapping and radar charting techniques. 6,580 ETS-related studies published between 1900 and 2008 were identified in the ISI database. Using different scientometric approaches, a continuous increase of both quantitative and qualitative parameters was found. The combination with density-equalizing calculations demonstrated a leading position of the United States (2,959 items published) in terms of quantitative research activities. Charting techniques demonstrated that there are numerous bi- and multilateral networks between different countries and institutions in this field. Again, a leading position of American institutions was found. Conclusions This is the first comprehensive scientometric analysis of data on global scientific activities in the field of environmental tobacco smoke research. The present findings can be used as a benchmark for funding allocation processes.

Inter-disease Comparison of Research Quantity and Quality: Bronchial Asthma and Chronic Obstructive Pulmonary Disease

Background. The two obstructive airway diseases bronchial asthma and chronic obstructive pulmonary disease (COPD) represent major global causes of disability and death. Whereas COPD research was largely underfunded in the 1980s and 1990s, increased funding activities have been initiated since the year 2000. However, detailed scientometric data on the development of research for asthma and COPD have not been generated so far. Methods. The present scientometric study was conducted to establish a database of research quantity and quality in the 20-year period between 1987 and 2006 using the Web of Science information system and the United Kingdom and Germany for comparison of research activities. Results. The information database Web of Science was screened and during the period from 1987 to 2006 a number of 8,874 items related to asthma was published by UK affiliations. Of these, 1,824 were published in cooperation with a total of 86 other countries. This is a ratio of 20.55%. In the same period, 3,341 items were published by German institutions (923 in cooperation with 56 other countries, ratio of 27.63%). Citation analysis demonstrated an average citation of 24.48 per UK article and 17.62 per German article. For COPD, 2,179 items were published by UK affiliations and 689 items by German institutions. Of the UK COPD publications, 570 were published in cooperations with 47 countries (ratio of 22.95 %). By contrast, 218 of the 689 German COPD articles were published with 29 other countries (ratio of 25.49%). When citation analysis was performed, average citation ratios of 18.93 for the UK and 10.61 for German were found. Conclusion. Summarizing this first country-specific comparative benchmarking analysis for obstructive pulmonary diseases it can be concluded that (1) asthma research dominated in the past 20 years; (2) COPD research gained importance in the field since the end of the 1990s; (3) there are large differences present in the research output between the two high-income countries examined.

Hepatitis B: global scientific development from a critical point of view

Hepatitis B is the tenth leading cause of death worldwide. Countries with high endemicity, such as China and Taiwan show high scientific productivity in this field and dominate the top ten list of the most productive authors worldwide, providing four of them. This is remarkable, as the USA and Europe usually maintain leading positions, not only regarding country‐specific scientific productivity, but also top ten ranking of most productive and most cited authors in other important medical sectors. So far, a scientometric analysis of the topic ‘hepatitis B’ has not been generated despite an increased need for it in times of modified evaluation criteria for academic personnel and a subsequent tendency to co‐authorship and author self‐citation. In this study, scientometric methods and large‐scale data analysis were used to evaluate quality and quantity of scientific research dealing with the topic ‘hepatitis B’ and to contribute to distinguish relevant research output. Data were gained from Pubmed and ISI‐Web. In the time span of 1971–2011, 49 166 items were published by 250 countries, of which the USA have been the most productive supplier with 28% of all publications, followed in considerable distance by Germany, China, the United Kingdom, Japan, France, Italy and Taiwan, respectively. The USA have established their position as centre of international cooperation. Their cooperation with China proves to be the most productive one. The most prolific journals in the field of hepatitis b were ‘Hepatology’, the ‘Journal of Hepatology’ and the ‘Journal of Medical Virology’. h‐index, citation rate and impact factor, commonly used for assessment of scientific quality, were determined and discussed critically with regard to distortion by bias of self‐citation and co‐authorship.

Interfield dysbalances in research input and output benchmarking: Visualisation by density equalizing procedures

BackgroundHistorical, social and economic reasons can lead to major differences in the allocation of health system resources and research funding. These differences might endanger the progress in diagnostic and therapeutic approaches of socio-economic important diseases. The present study aimed to assess different benchmarking approaches that might be used to analyse these disproportions. Research in two categories was analysed for various output parameters and compared to input parameters. Germany was used as a high income model country. For the areas of cardiovascular and respiratory medicine density equalizing mapping procedures visualized major geographical differences in both input and output markers.ResultsAn imbalance in the state financial input was present with 36 cardiovascular versus 8 respiratory medicine state-financed full clinical university departments at the C4/W3 salary level. The imbalance in financial input is paralleled by an imbalance in overall quantitative output figures: The 36 cardiology chairs published 2708 articles in comparison to 453 articles published by the 8 respiratory medicine chairs in the period between 2002 and 2006. This is a ratio of 75.2 articles per cardiology chair and 56.63 articles per respiratory medicine chair. A similar trend is also present in the qualitative measures. Here, the 2708 cardiology publications were cited 48337 times (7290 times for respiratory medicine) which is an average citation of 17.85 per publication vs. 16.09 for respiratory medicine. The average number of citations per cardiology chair was 1342.69 in contrast to 911.25 citations per respiratory medicine chair. Further comparison of the contribution of the 16 different German states revealed major geographical differences concerning numbers of chairs, published items, total number of citations and average citations.ConclusionDespite similar significances of cardiovascular and respiratory diseases for the global burden of disease, large input and output imbalances have been revealed in the present study which point to a need for changes in funding policies. The present study supplies data that could be used for decision making in the field of health systems funding.

Traffic medicine–related research: a scientometric analysis

ObjectiveTraffic crashes and related injuries are important causes of morbidity and mortality and impose insofar an important burden on public health. However, research in this area is often under-funded. The aim of this study was to analyse quantity, evolution and geographic distribution of traffic medicine-related research. This multi-sectorial field covers both transport and health care sectors.DesignA scientometric approach in combination with visualizing density equalizing mapping was used to analyse published data related to the field of traffic medicine between 1900 and 2008 within the “Web of Science” (WoS) database.ResultsIn total, 5,193 traffic medicine-associated items were produced between 1900 and 2008. The United States was found to have the highest research activity with a production of n = 2,330 published items, followed by Germany (n = 298) and Canada (n = 219). Cooperation analyses resulted in a peak of published multilateral cooperations in the year of 2003. The country with the highest multilateral activity was the USA. The average number of cited references per publication varied heavily over the last 20 years with a maximum of 27.67 in 1995 and a minimum of 15.08 in 1998. Also, a further in-depth analysis was performed with a focus solely on public health aspects which revealed similar trends.ConclusionsSummarizing the present data it can be stated traffic medicine-related research productivity grows annually. Also, an active networking between countries is present. The data of the present study may be used by scientific organisations in order to gain detailed information about research activities in this field which is extremely important for public health.