Donald J. Berndt

The Catch data warehouse: support for community health care decision-making

The measurement and assessment of health status in communities throughout the world is a massive information technology challenge. Comprehensive Assessment for Tracking Community Health (CATCH) provides systematic methods for community-level assessment that is invaluable for resource allocation and health care policy formulation. CATCH is based on health status indicators from multiple data sources, using an innovative comparative framework and weighted evaluation process to produce a rank-ordered list of critical community health care challenges. The community-level focus is intended to empower local decisionmakers by providing a clear methodology for organizing and interpreting relevant health care data. Extensive field experience with the CATCH methods, in combination with expertise in data warehousing technology, has led to an innovative application of information tehnology in the health care arena. The data warehouse allows a core set of reports to be produced at a reasonable cost for community use. In addition, online analytic processing (OLAP) functionality can be used to gain a deeper understanding of specific health care issues. The data warehouse in conjunction with Web-enabled dissemination methods allows the information to be presented in a variety of formats and to be distributed more widely in the decision-making community. In this paper, we focus on the technical challenges of designing and implementing an effective data warehouse for health care information. Illustrations of actual data designs and reporting formats from the CATCH data warehouse are used throughout the discussion. Ongoing research directions in health care data warehousing and community health care decision-making conclude the paper.

Communication Challenges in Requirements Elicitation and the Use of the Repertory Grid Technique

Requirements elicitation is a central and critical activity in the systems analysis and design process. This paper explores the nature of the challenges that confront analysts and their clients during requirements elicitation. A review of the literature highlights communication as a persistent locus of concern among systems analysts, users and procurers. The paper presents a classification of communication challenges that arise during the requirements elicitation process. Empirical evidence from a brief case study is used to illustrate the scope and impact of these communication challenges and to present a complementary approach to requirements elicitation. The paper introduces the Repertory Grid technique as a means to ameliorate some of the communication issues that persist, particularly in projects where information systems support specialized work. The paper is written in the form of a case tutorial, providing insight into the contribution of the Repertory Grid technique to requirements elicitation.

The Use of Focus Groups in Design Science Research

Focus groups to investigate new ideas are widely used in many research fields. The use of focus groups in design science research poses interesting opportunities and challenges. Traditional focus group methods must be adapted to meet two specific goals of design research. For the evaluation of an artifact design, exploratory focus groups (EFGs) study the artifact to propose improvements in the design. The results of the evaluation are used to refine the design and the cycle of build and evaluate using EFGs continues until the artifact is released for field test in the application environment. Then, the field test of the design artifact may employ confirmatory focus groups (CFGs) to establish the utility of the artifact in field use. Rigorous investigation of the artifact requires multiple CFGs to be run with opportunities for quantitative and qualitative data collection and analyses across the multiple CFGs. In this chapter, we discuss the adaptation of focus groups to design science research projects. We demonstrate the use of both EFGs and CFGs in a design research doctoral thesis in the health-care field.

Doing more with more information: Changing healthcare planning with OLAP tools

On-line analytical processing (OLAP) is an example of a new breed of tools for decision support that give decision makers the flexibility to customize the selection, aggregation, and presentation of data. To understand the impact of this type of tool, we study an implementation of an OLAP interface on the CATCH data warehouse used by knowledge workers at a regional health planning agency in the State of Florida. The results of a qualitative field study show that after the OLAP implementation, these workers made use of the additional capabilities of OLAP (e.g., aggregation levels and intuitive data manipulation), thereby leveraging their individual abilities to enhance and expand on the tasks they performed for their community. Consequently, they were able to perform in more of a consultative role to their clients, and improved their reputation in the community they serve. This research adds a new dimension to prior research in data warehousing by focusing on the decision support capabilities of OLAP.

Identifying fall-related injuries: Text mining the electronic medical record

Unintentional injury due to falls is a serious and expensive health problem among the elderly. This is especially true in the Veterans Health Administration (VHA) ambulatory care setting, where nearly 40% of the male patients are 65 or older and at risk for falls. Health service researchers and clinicians can utilize VHA administrative data to identify and explore the frequency and nature of fall-related injuries (FRI) to aid in the implementation of clinical and prevention programs. Here we define administrative data as structured (coded) values that are generated as a result clinical services provided to veterans and stored in databases. However, the limitations of administrative data do not always allow for conclusive decision making, especially in areas where coding may be incomplete. This study utilizes data and text mining techniques to investigate if unstructured text-based information included in the electronic medical record can validate and enhance those records in the administrative data that should have been coded as fall-related injuries. The challenges highlighted by this study include data extraction and preparation from administrative sources and the full electronic medical records, de-indentifying the data (to assure HIPAA compliance), conducting chart reviews to construct a “gold standard” dataset, and performing both supervised and unsupervised text mining techniques in comparison with traditional medical chart review.

High Volume Software Testing using Genetic Algorithms

The potential cost savings from handling software errors within a development cycle, rather than the subsequent cycles, has been estimated at nearly 40 billion dollars by the National Institute of Standards and Technology. This figure emphasizes that current testing methods are often inadequate, and that helping reduce software bugs and errors is an important area of research with a substantial payoff. This is particularly true for the increasingly complex, distributed systems used in many applications from embedded control systems to military command and control systems. These systems may exhibit intermittent or transient errors after prolonged execution that are very difficult to diagnose. This paper explores strategies that combine automated test suite generation techniques with high volume or long sequence testing. Long sequence testing repeats test cases many times, simulating extended execution intervals. These testing techniques have been found useful for uncovering errors resulting from component coordination problems, as well as system resource consumption (e.g. memory leaks) or corruption. Coupling automated test suite generation with long sequence testing could make this approach more scalable and effective in the field.

Investigating the performance of genetic algorithm-based software test case generation

Highly complex and interconnected systems may suffer from intermittent or transient failures that are particularly difficult to diagnose. This research focuses on the use of genetic algorithms for automatically generating large volumes of software test cases. In particular, the paper explores two fundamental strategies for improving the performance of genetic algorithm test case breeding for high volume testing. The first strategy seeks to avoid evaluating test cases against the real target system by using oracles or models. The second strategy involves improving the more costly components of genetic algorithms, such as fitness function calculations. Together, the various approaches offer opportunities for performance improvements that make these techniques more scalable for realistic applications.

A reengineering framework for evaluating a financial imaging system

T HIS article presents a framework for comparing and evaluating efforts in reengineering, or business process redesign. The framework is applied to the study of the reengineering of the securities processing function at the brokerage and financial services firm Merrill Lynch, comparing the firm’s old and new processes. The new process features image processing, character recognition, and extensive redesign. Reengineering, one of the latest trends in the information systems field, is defined by Hammer and Champy [4] as “the fundamental

Finding falls in ambulatory care clinical documents using statistical text mining.

OBJECTIVE To determine how well statistical text mining (STM) models can identify falls within clinical text associated with an ambulatory encounter. MATERIALS AND METHODS 2241 patients were selected with a fall-related ICD-9-CM E-code or matched injury diagnosis code while being treated as an outpatient at one of four sites within the Veterans Health Administration. All clinical documents within a 48-h window of the recorded E-code or injury diagnosis code for each patient were obtained (n=26 010; 611 distinct document titles) and annotated for falls. Logistic regression, support vector machine, and cost-sensitive support vector machine (SVM-cost) models were trained on a stratified sample of 70% of documents from one location (dataset Atrain) and then applied to the remaining unseen documents (datasets Atest-D). RESULTS All three STM models obtained area under the receiver operating characteristic curve (AUC) scores above 0.950 on the four test datasets (Atest-D). The SVM-cost model obtained the highest AUC scores, ranging from 0.953 to 0.978. The SVM-cost model also achieved F-measure values ranging from 0.745 to 0.853, sensitivity from 0.890 to 0.931, and specificity from 0.877 to 0.944. DISCUSSION The STM models performed well across a large heterogeneous collection of document titles. In addition, the models also generalized across other sites, including a traditionally bilingual site that had distinctly different grammatical patterns. CONCLUSIONS The results of this study suggest STM-based models have the potential to improve surveillance of falls. Furthermore, the encouraging evidence shown here that STM is a robust technique for mining clinical documents bodes well for other surveillance-related topics.

Breeding software test cases for complex systems

The potential cost savings from handling software errors within a development cycle, rather than subsequent cycles, has been estimated at 38.3 billion dollars. Such figures emphasize that current testing methods are inadequate, and that helping reduce software bugs and errors is an important area of research with a substantial payoff. This paper reports on research using genetic algorithms for test case generation for systems level testing, building on past work at the unit testing level. The goals of the paper are to explore the use of genetic algorithms for testing complex distributed systems, as well as to develop a framework or vocabulary of important environmental attributes that characterize complex systems failures. In addition, preliminary visualization techniques that might help software developers to understand and uncover complex systems failures are explored.