Gove N. Allen

Academic data collection in electronic environments: defining acceptable use of internet resources

Academic researchers access commercial web sites to collect research data. This research practice is likely to increase. Is this appropriate? Is this legal? Such commercial web sites are maintained to achieve business objectives; research access uses site resources for other purposes. Web site administrators may, therefore, deem academic data collection inappropriate. Is there a process to make research access more open and acceptable to web site owners and administrators? These are significant issues. This article clarifies the problems and suggests possible approaches to handle the issues with sensitivity and openness. Research access to commercial web sites may be manual (using a standard web browser) or automated (using automated data collection agents). These approaches have different effects on web sites. Researchers using manual access tend to make a limited number of page requests because manual access is costly to perform. Researchers using automated access methods can request large numbers of pages at a low cost. Therefore, web site administrators tend to view manual access and automated access very differently. Because of the number of accesses and the nonbusiness purpose, automated research requests for data are sometimes blocked by site administration using a variety of means (both technological and legal). This paper details the pertinent legal issues including trespass, copyright violation, and breech of contract. It also explains the nature of express and implied consent by site administration for research access. Based on the issues presented, guidelines for researchers are proposed to reduce objections to research activities, to facilitate communication with web site administration, and to achieve express or implied consent. These include notification to web site administration of intended automated research activity, description of the research project posted as a web page, and clear identification of automated requests for web pages. In order to encourage good research practices with respect to automated data collection, suggestions are made with respect to disclosing methods used in research papers and for self regulation by academic associations.

Is Query Reuse Potentially Harmful? Anchoring and Adjustment in Adapting Existing Database Queries

Reusing database queries by adapting them to satisfy new information requests is an attractive strategy for extracting information from databases without involving database specialists. However, the reuse of information systems artifacts has been shown to be susceptible to the phenomenon of anchoring and adjustment. Anchoring often leads to a systematic adjustment bias in which people fail to make sufficient changes to an anchor in response to the needs of a new task. In a study involving 157 novice query writers from six universities, we examined the effect of this phenomenon on the reuse of Structured Query Language (SQL) queries under varying levels of domain familiarity and for different types of anchors. Participants developed SQL queries to respond to four information requests in a familiar domain and four information requests in an unfamiliar domain. For two information requests in each domain, participants were also provided with sample queries (anchors) that answered similar information requests. We found evidence that the opportunity to reuse sample queries resulted in an adjustment bias leading to poorer quality query results and greater overconfidence in the correctness of results. The results also indicate that the strength of the adjustment bias depends on a combination of domain familiarity and type of anchor. This study demonstrates that anchoring and adjustment during query reuse can lead to queries that are less accurate than those written from scratch. We also extend the concept of anchoring and adjustment by distinguishing between surface-structure and deep-structure anchors and by considering the impact of domain familiarity on the adjustment bias.

The effects of state-based and event-based data representation on user performance in query formulation tasks

Ad hoc query formulation is an important task in effectively utilizing organizational data resources. To facilitate this task, managers and casual end-users are commonly presented with database views expressly constructed for their use. Differences in the way in which things, states, and events are represented in such views can affect a users ability to understand the database, potentially leading to different levels of performance (i.e., accuracy, confidence, and prediction of the accuracy of their queries). An experiment was conducted over the Internet involving 342 subjects from 6 universities in North America and Europe to investigate these effects. When presented with an event-based view, subjects expressing low or very low comfort levels in reading entity-relationship diagrams expressed confidence that better predicted query accuracy although there were no significant differences in actual query accuracy or level of confidence expressed.

Ontological Foundations for Active Information Systems

While passive information systems simply record and report on the observed states of things in the world, active information systems participate in the determination and ascription of state to things. They infer conclusions based on the application of rules that govern how things in the real world are affected when defined and identified events occur. The ontological foundations for active information systems must include constructs to represent such causal rules. Conceptualizing things and events as distinct ontological categories with existence and properties and representing them as entities at the conceptual level is sufficient for this purpose. The properties of an event include data values inherent in the event and rules that define how the states of affected things are changed when the event occurs. In this manner the state-history of a thing is represented by the sequence of events that have affected it. Future states of a thing can be predicted based on proposed or conjectured events. Such a conceptualization enables a parsimonious mapping between an active information system and the real world system it is intended to model.

Modeling Temporal Dynamics for Business Systems

Research in temporal database management has viewed temporal dynamics from a structural perspective, posing extensions to the entity-relationship (E-R) model to represent the state history of time-dependent attributes and relationships. We argue that temporal dynamics are semantic rather than structural and that the existing constructs in the E-R model are sufficient to represent them. Practitioners have long used E-R models without temporal extensions to design systems with rich support for temporality by modeling both things and events as entities — a practice that is consistent with the original presentation of the E-R model. This approach supports methodologies that leverage narrative and human cognitive processing capabilities in the development and verification of data models. Furthermore it maintains modeling parsimony and facilitates the representation of causality — why a particular state exists.

How well do shopbots represent online markets? A study of shopbots’ vendor coverage strategy

Consumers often use shopbots to search for information when making purchase decisions in Internet markets. Although they have varying sensitivity to shopbot bias, consumers generally prefer accurate market representation. However, in choosing the accuracy of market representation, shopbots must balance the desires of consumers with the costs of providing their services and with the desires of the vendors, who are often the largest source of their revenue. In this paper, we study how accurately shopbots represent a market and analyze the strategies shopbots adopt to achieve market representativeness. We theoretically identify two important drivers in shopbot vendor coverage strategy – how many vendors it covers (shopbot size) and which vendors it covers (shopbot affiliation) – and analytically show how the drivers affect shopbot market representativeness. We report the results of a large-scale study in which we collected 2.2 million vendor price listings from eight shopbots and develop metrics for measuring shopbot size, shopbot affiliation, and shopbot market representativeness. We found that (1) shopbots do not represent markets equally well; (2) size drives a shopbots market representativeness positively whereas affiliation drives a shopbots market representativeness negatively; (3) shopbots follow differnet vendor representative strategies to pursue market representativeness.

Research Frontiers in Object Technology

Object technology has been widely acclaimed as offering a revolution in computing that will resolve a myriad of problems inherent in developing and managing organizational information processing capabilities. Although its foundations arose in computer programming languages, object technology has implications for a wide range of business computing activities including: Programming, Analysis and Design, Information Management, and Information Sharing. We examine six fundamental research frontiers in each activity: Common Business Classes; Organizational Barriers; Applications and Tools; Reuse and Object Management; Standards, Testing, and Metrics; and Technology Investment. The cross product of the business computing activities with these fundamental research frontiers yields a taxonomy within which to position the research needed to realize the promises offered by object technology.

Using transaction-level data to diagnose knowledge gaps and misconceptions

The role of assessment in learning is to evaluate student comprehension and ability. Assessment instruments often function at the task level. What is rarely considered is the process students go through to reach the final solution. This often allows knowledge component gaps and misconceptions to go undetected. This research identified higher levels of knowledge component gaps and misunderstandings when assessing transaction-level knowledge component data than task-level final solution data. Final solution data showed little evidence that students had any misunderstanding or knowledge gaps about the use of absolute references. However, when analyzing these data at the transaction level we found evidence that far more students struggled than the analysis of the final solutions suggested.

Transaction-level learning analytics in online authentic assessments

This paper presents a case for the use of transaction-level data when analyzing automated online assessment results to identify knowledge gaps and misconceptions for individual students. Transaction-level data, which records all of the steps a student uses to complete an assessment item, are preferred over traditional assessment formats that submit only the final answer, as the system can detect persistent misconceptions. In this study we collected transaction-level data from 996 students enrolled in an online introductory spreadsheet class. Each student’s final answer and step-by-step attempts were coded for misconceptions or knowledge gaps regarding the use of absolute references over four assessment occasions. Overall, the level of error revealed was significantly higher in the step-by-step processes compared to the final submitted answers. Further analysis suggests that students most often have misconceptions regarding non-critical errors. Data analysis also suggests that misconceptions identified at the transaction level persist over time.