Frederick A. Peck

Strategy hubs: next-generation domain portals with search procedures

Current search tools on the Web, such as general-purpose search engines (e.g. Google) and domain-specific portals (e.g. MEDLINEplus), do not provide search procedures that guide users to form appropriately ordered sub-goals. The lack of such procedural knowledge often leads users searching in unfamiliar domains to retrieve incomplete information. In critical domains such as in healthcare, such ineffective searches can have dangerous consequences. To address this situation, we developed a new type of domain portal called a Strategy Hub. Strategy Hubs provide the critical search procedures and associated high-quality links that enable users to find comprehensive and accurate information. This paper describes how we collaborated with skin cancer physicians to systematically identify generalizeable search procedures to find comprehensive information about melanoma, and how these search procedures were made available through the Strategy Hub for healthcare. A pilot study suggests that this approach can improve the efficacy, efficiency, and satisfaction of even expert searchers. We conclude with insights on how to refine the design of the Strategy Hub, and how it can be used to provide search procedures across domains.

Exploring the distribution of online healthcare information

Motivated by the importance of retrieving comprehensive healthcare information, we analyzed how information about 12 concepts related to a widely available healthcare topic is distributed across 145 high-quality webpages. The analysis reveals that the distribution of the concepts follows a power law where a few pages contain many concepts, while the majority contains less than half the concepts. The analysis also reveals the existence of general, specialized, and sparse pages, in addition to the large number of pages that users must visit before they have access to all the concepts. These results provide insights into expert search procedures, and motivate the design of future search systems that guide users in the retrieval of comprehensive information.

Strategy-Based Instruction: Lessons Learned in Teaching the Effective and Efficient Use of Computer Applications

Numerous studies have shown that many users do not acquire the knowledge necessary for the effective and efficient use of computer applications such as spreadsheets and Web-authoring tools. While many cognitive, cultural, and social reasons have been offered to explain this phenomenon, there have been few systematic attempts to address it. This article describes how we identified a framework to organize effective and efficient strategies to use computer applications and used an approach called strategy-based instruction to teach those strategies over five years to almost 400 students. Controlled experiments demonstrated that the instructional approach (1) enables students to learn strategies without harming command knowledge, (2) benefits students from technical and nontechnical majors, and (3) is robust across different instructional contexts and new applications. Real-world classroom experience of teaching strategy-based instruction over several instantiations has enabled the approach to be disseminated to other universities. The lessons learned throughout the process of design, implementation, evaluation, and dissemination should allow teaching a large number of users in many organizations to rapidly acquire the strategic knowledge to make more effective and efficient use of computer applications.

Using Learning Progressions to Design Vertical Scales that Support Coherent Inferences about Student Growth

The concept of growth is at the foundation of the policy and practice around systems of educational accountability. It is also at the foundation of what teachers concern themselves with on a daily basis as they help children learn. Yet there is a disconnect between the criterion-referenced intuitions that parents and teachers have for what it means for students to demonstrate growth and the primarily norm-referenced metrics that are used to infer growth. One way to address this disconnect would be to develop vertically linked score scales that could be used to support both criterion-referenced and norm-referenced interpretations, but this hinges upon having a coherent conceptualization of what it is that is growing from grade to grade. In this paper, a learning-progression approach to the conceptualization of growth and the subsequent design of a vertical score scale is proposed and illustrated in the context of the Common Core State Standards for Mathematics.

Exploring the Use of Natural Language Systems for Fact Identification: Towards the Automatic Construction of Healthcare Portals

In prior work we observed that expert searchers follow well-defined search procedures in order to obtain comprehensive information on the Web. Motivated by that observation, we developed a prototype domain portal called the Strategy Hub that provides expert search procedures to benefit novice searchers. The search procedures in the prototype were entirely handcrafted by search experts, making further expansion of the Strategy Hub cost-prohibitive. However, a recent study on the distribution of healthcare information on the web suggested that search procedures can be automatically generated from pages that have been rated based on the extent to which they cover facts relevant to a topic. This paper presents the results of experiments designed to automate the process of rating the extent to which a page covers relevant facts. To automatically generate these ratings, we used two natural language systems, Latent Semantic Analysis and MEAD, to compute the similarity between sentences on the page and each fact. We then used an algorithm to convert these similarity scores to a single rating that represents the extent to which the page covered each fact. These automatic ratings are compared with manual ratings using inter-rater reliability statistics. Analysis of these statistics reveals the strengths and weaknesses of each tool, and suggests avenues for improvement.

Struggling for Legitimacy: Trajectories of Membership and Naturalization in the Sorting Out of Engineering Students

Situated learning theory proposed the notion of legitimate peripheral participation as central to newcomers’ trajectories toward membership in communities of practice. Left underdeveloped were questions of how legitimacy is conferred or denied. Leigh Star’s work suggests ways of addressing these questions by considering how identifications of participants are made on the basis of classification systems. This article examines how classification of “calculus-readiness,” tied to an identification of engineering with mathematics, shapes the activity of students, staff, and faculty involved in a diversity program at a U.S. college of engineering, and is used to organize trajectories for students in the program.

Towards a Model of Information Scatter: Implications for Search and Design

Recent studies suggest that users often retrieve incomplete healthcare information because of the complex and skewed distribution of facts across relevant webpages. To understand the causes for such skewed distributions, this paper presents the results of two analyses: (1) A distribution analysis discusses how facts related to healthcare topics are scattered across high-quality healthcare pages. (2) A cluster analysis of the same data suggests that the skewed distribution can be explained by the existence of three page profiles that vary in information density, each of which play an important role in providing comprehensive information on a topic. The above analyses provide clues towards a model of information scatter which describes how the design decisions by individual webpage authors could collectively lead to the scatter of information as observed in the data. The analyses also suggest implications for the design of websites, search algorithms, and search interfaces to help users find comprehensive information about a topic.