Gregory Schymik

Strategies for Document Management

Keyword search has failed to adequately meet the needs of enterprise users. This is largely due to the size of document stores, the distribution of word frequencies, and the indeterminate nature of languages. The authors argue a different approach needs to be taken, and draw on the successes of dimensional data modeling and subject indexing to propose a solution. They test our solution by performing search queries on a large research database. By incorporating readily available subject indexes into the search process, they obtain order of magnitude improvements in the performance of search queries. Their performance measure is the ratio of the number of documents returned without using subject indexes to the number of documents returned when subject indexes are used. The authors explain why the observed tenfold improvement in search performance on our research database can be expected to occur for searches on a wide variety of enterprise document stores.

An Interactive Search Method Based on User Preferences

This paper presents a general method for interactively searching for objects (alternatives) in a large collection the contents of which are unknown to the user and where the objects are defined by a large number of discrete-valued attributes. Briefly, the method presents an object and asks the user to indicate his or her preference for the object. The method allows preference indications in two basic modes: (1) by assignment of objects to predefined preference categories such as high, medium, and low preference or (2) by direct preference comparison of objects such as “object A preferred to object B.” From these preference statements, the method learns about the users preferences and constructs an approximation to a value or preference function of the user (additive or multiplicative) at each iteration. It then uses this approximate preference function to rerank the objects in the collection and retrieve the top-ranked ones to present to the user at the next iteration. The process terminates when the user is satisfied with the list of top-ranked objects. This method can also be used to solve general multiattribute discrete alternative problems, where the alternatives are known with certainty and described by a set of discrete-valued attributes. Test results are reported and application possibilities are discussed.

Architecting a Dimensional Document Warehouse

This paper examines how the concepts of dimensional data warehouses can be applied to document retrieval and storage. It then shows how the specifics of dimensional document warehouses differ from dimensional data warehouse and how these differences make it impractical to use existing engines for building and analyzing data cubes (such as SQL servers analysis manager) in order to build and analyze a document warehouse. The paper further shows that readily available software can be used to build an engine to analyze a dimensional document mart. All of the steps required to design, build, and analyze a dimensional document mart are described and illustrated. Design features are suggested for improving the recall and precision of searches from dimensional document marts

Enabling self-service BI: A methodology and a case study for a model management warehouse

The promise of Self-Service Business Intelligence (BI) is its ability to give business users access to selection, analysis, and reporting tools without requiring intervention from IT. This is essential if BI is to maximize its contribution by radically transforming how people make decisions. However, while some progress has been made through tools such as SAS Enterprise Miner, IBM SPSS Modeler, and RapidMiner, analytical modeling remains firmly in the domain of IT departments and data scientists. The development of tools that mitigate the need for modeling expertise remains the “missing link” in self-service BI, but prior attempts at developing modeling languages for non-technical audiences have not been widely implemented. By introducing a structured methodology for model formulation specifically designed for practitioners, this paper fills the unmet need to bring model-building to a mainstream business audience. The paper also shows how to build a dimensional Model Management Warehouse that supports the proposed methodology, and demonstrates the viability of this approach by applying it to a problem faced by the Division of Fiscal and Actuarial Services of the US Department of Labor. The paper concludes by outlining several areas for future research.

The benefits and costs of using metadata to improve enterprise document search

The literature shows that there are many problems with enterprise document search. Studies reveal that typical knowledge workers spend between 10% and 20% of their time searching for documents they never find. While many argue that metadata can improve enterprise document search, in reality few organizations use metadata. This represents a missed opportunity. This article describes the results of two experiments that use simulation to evaluate the actual impact of metadata on the costs and benefits of enterprise search. The first study provides quantitative evidence of the increase in recall and precision that stems from the use of metadata-enhanced document searches. The second study demonstrates that simple metadata structures can be nearly as effective as complex ones, implying that the cost of creating and maintaining metadata is likely to be lower than generally thought. This is the first study to provide explicit quantitative evidence of the gains that can be achieved from the use of metadata, and one of only a handful of studies that examines the cost of creating and maintaining metadata

Query Expansion in Enterprise Search

Although web search remains an active research area, interest in enterprise search has not kept up with the information requirements of the contemporary workforce. To address these issues, this research aims to develop, implement, and study the query expansion techniques most effective at improving relevancy in enterprise search. The case-study instrument was a custom Apache Solr-based search application deployed at a medium-sized manufacturing company. It was hypothesized that a composition of techniques tailored to enterprise content and information needs would prove effective in increasing relevancy evaluation scores. Query expansion techniques leveraging entity recognition, alphanumeric term identification, and intent classification were implemented and studied using real enterprise content and query logs. They were evaluated against a set of test queries derived from relevance survey results using standard relevancy metrics such as normalized discounted cumulative gain (nDCG). Each of these modules produced meaningful and statistically significant improvements in relevancy.

Strategies for Improving the Efficacy of Fusion Question Answering Systems

Web search engines typically retrieve a large number of web pages and overload business analysts with irrelevant information. One approach that has been proposed for overcoming some of these problems is automated Question Answering (QA). This paper describes a case study that was designed to determine the efficacy of QA systems for generating answers to original, fusion, list questions (questions that have not previously been asked and answered, questions for which the answer cannot be found on a single web site, and questions for which the answer is a list of items). Results indicate that QA algorithms are not very good at producing complete answer lists and that searchers are not very good at constructing answer lists from snippets. These findings indicate a need for QA research to focus on crowd sourcing answer lists and improving output format.