Liwen Vaughan

Search engine coverage bias: evidence and possible causes

Commercial search engines are now playing an increasingly important role in Web information dissemination and access. Of particular interest to business and national governments is whether the big engines have coverage biased towards the US or other countries. In our study we tested for national biases in three major search engines and found significant differences in their coverage of commercial Web sites The US sites were much better covered than the others in the study: sites from China, Taiwan and Singapore. We then examined the possible technical causes of the differences and found that the language of a site does not affect its coverage by search engines. However, the visibility of a site, measured by the number of links to it, affects its chance to be covered by search engines. We conclude that the coverage bias does exist but this is due not to deliberate choices of the search engines but occurs as a natural result of cumulative advantage effects of US sites on the Web. Nevertheless, the bias remains a cause for international concern.

New measurements for search engine evaluation proposed and tested

A set of measurements is proposed for evaluating Web search engine performance. Some measurement are adapted from the concepts of recall and precision, which are commonly used in evaluating traditional information retrieval systems. Others are newly developed to evaluate search engine stability, an issue unique to Web information retrieval systems. An experiment was conducted to test these new measurements by applying them to a performance comparison of three commercial search engines: Google, AltaVista, and Teoma. Twenty-four subjects ranked four sets of Web pages and their rankings were used as benchmarks against which to compare search engine performance. Results show that the proposed measurements are able to distinguish search engine performance very well.

Scholarly use of the web: what are the key inducers of links to journal web sites?

Web links have been studied by information scientists for at least six years but it is only in the past two that clear evidence has emerged to show that counts of links to scholarly Web spaces (universities and departments) can correlate significantly with research measures, giving some credence to their use for the investigation of scholarly communication. This paper reports on a study to investigate the factors that influence the creation of links to journal Web sites. An empirical approach is used: collecting data and testing for significant patterns. The specific questions addressed are whether site age and site content are inducers of links to a journals Web site as measured by the ratio of link counts to Journal Impact Factors, two variables previously discovered to be related. A new methodology for data collection is also introduced that uses the Internet Archive to obtain an earliest known creation date for Web sites. The results show that both site age and site content are significant factors for the disciplines studied: library and information science, and law. Comparisons between the two fields also show disciplinary differences in Web site characteristics. Scholars and publishers should be particularly aware that richer content on a journals Web site tends to generate links and thus the traffic to the site.

Bibliographic and Web citations: what is the difference?

Web citations have been proposed as comparable to, even replacements for, bibliographic citations, notably in assessing the academic impact of work in promotion and tenure decisions. We compared bibliographic and Web citations to articles in 46 journals in library and information science. For most journals (57%), Web citations correlated significantly with both bibliographic citations listed in the Social Sciences Citation Index and the ISIs Journal Impact Factor. Many of the Web citations represented intellectual impact, coming from other papers posted on the Web (30%) or from class readings lists (12%). Web citation counts were typically higher than bibliographic citation counts for the same article. Journals with more Web citations tended to have Web sites that provided tables of contents on the Web, while less cited journals did not have such publicity. The number of Web citations to journal articles increased from 1992 to 1997.

Web citation data for impact assessment: A comparison of four science disciplines

The number and type of Web citations to journal articles in four areas of science are examined: biology, genetics, medicine, and multidisciplinary sciences. For a sample of 5,972 articles published in 114 journals, the median Web citation counts per journal article range from 6.2 in medicine to 10.4 in genetics. About 30% of Web citations in each area indicate intellectual impact (citations from articles or class readings, in contrast to citations from bibliographic services or the authors or journals home page). Journals receiving more Web citations also have higher percentages of citations indicating intellectual impact. There is significant correlation between the number of citations reported in the databases from the Institute for Scientific Information (ISI, now Thomson Scientific) and the number of citations retrieved using the Google search engine (Web citations). The correlation is much weaker for journals published outside the United Kingdom or United States and for multidisciplinary journals. Web citation numbers are higher than ISI citation counts, suggesting that Web searches might be conducted for an earlier or a more fine-grained assessment of an articles impact. The Web-evident impact of non-UK/USA publications might provide a balance to the geographic or cultural biases observed in ISIs data, although the stability of Web citation counts is debatable.

Relationship between links to journal Web sites and impact factors

The study found a significant correlation between the number of external links and the journal impact factor for LIS journals. Journals with higher journal impact factor scores tend to attract more links to their Web sites. The study also investigated issues pertaining to data collection methods for webometrics research. It showed that the choice of search engine for data collection could affect the conclusion of a study. Data collected at different time periods were found to be fairly stable. The use of multiple rounds of data collection was shown to be beneficial, especially when the result from a single round of data is borderline significant or inconclusive.

A new look at evidence of scholarly citation in citation indexes and from web sources

A sample of 1,483 publications, representative of the scholarly production of LIS faculty, was searched in Web of Science (WoS), Google, and Google Scholar. The median number of citations found through WoS was zero for all types of publications except book chapters; the median for Google Scholar ranged from 1 for print/subscription journal articles to 3 for books and book chapters. For Google the median number of citations ranged from 9 for conference papers to 41 for books. A sample of the web citations was examined and classified as representing intellectual or non-intellectual impact. Almost 92% of the citations identified through Google Scholar represented intellectual impact — primarily citations from journal articles. Bibliographic services (non-intellectual impact) were the largest single contributor of citations identified through Google. Open access journal articles attracted more web citations but the citations to print/subscription journal articles more often represented intellectual impact. In spite of problems with Google Scholar, it has the potential to provide useful data for research evaluation, especially in a field where rapid and fine-grained analysis is desirable.

Links to commercial websites as a source of business information

Websites of Chinas top 100 information technology (IT) companies were examined. Link count to a companys website was found to correlate with the companys revenue, profit, and research and development expenses. This suggests that Web hyperlinks to commercial sites can be a business performance indicator and thus a source of business information. This information is useful for Web business intelligence and Web data mining. As a comparison to IT companies, Chinas top 100 privately owned companies were also studied. No relationship between link count and the business performance measure was found for these companies due probably to the heterogeneous nature of this group. Data collection issues for webometrics research were also explored in the study.

A modeling approach to uncover hyperlink patterns: the case of Canadian universities

Hyperlink patterns between Canadian university Web sites were analyzed by a mathematical modeling approach. A multiple regression model was developed which shows that faculty quality and the language of the university are important predictors for links to a university Web site. Higher faculty quality means more links. French universities received lower numbers of links to their Web sites than comparable English universities. Analysis of interlinking between pairs of universities also showed that English universities are advantaged. Universities are more likely to link to each other when the geographical distance between them is less than 3000 km, possibly reflecting the east vs. west divide that exists in Canadian society.

Exploring website features for business information

Two previous webometrics studies found a relationship between the number of inlinks to a commercial site and the companys business performance measures. Thus inlink counts to commercial sites could be a potential source of business information. However, those studies examined top ranking information technology companies in the U.S. and China. Whether the above-mentioned relationship holds for all companies regardless of ranking and in other countries is unknown. The study reported in this paper investigated this question. The study includes all information technology companies in the U.S. and Canada and gathered both business performance data and website data for these companies. It found significant correlation between business performance measures and inlinks to the company websites. The correlation was still significant even after the size of the company and the website age were accounted for. The conclusion is robust to the search engine used for data collection. Data collection issues for webometrics research were also explored.