Eric Larson

Generating Evil Test Strings for Regular Expressions

Regular expressions are a powerful string processing tool. However, they are error-prone and receive little error checking from the compiler as most regular expressions are syntactically correct. This paper describes EGRET, a tool for generating evil test strings for regular expressions. EGRET focuses on common mistakes made by developers when creating regular expressions and develops test strings that expose these errors. EGRET has found errors in 284 out of 791 regular expressions. Prior approaches to test string generation have traversed all possible paths in the equivalent nondeterministic finite state automaton leading to the generation of too many strings. EGRET keeps the set of test strings to a manageable number: Fewer than 100 test strings were generated for 96% of the regular expressions, a manageable 307 test strings were generated for the most complex regular expression.

Program analysis too loopy? Set the loops aside

Among the many obstacles to efficient and sound program analysis, loops may be the most prevalent. In program analyses that traverse paths, loops introduce a variable, possibly infinite and number of paths. This study assesses the potential of a program analysis technique that analyses loops separately and replaces the loop with a summary, similar to how many analyses use summaries for interprocedural analysis. This study is conducted by comparing the path counts when loops are analysed separately to a baseline path count where loops are traversed at most once. Although the number of paths is decreased in many cases, the magnitude of the decrease is typically not sufficient for long, complex functions. In addition, loops are classified by the task they perform, analysed using the number of paths as an estimate of their complexity and further inspected for programming elements that may make loop analysis more difficult. Of the 2869 loops used in this study, 84% of the loops have fewer than ten paths and only 1.3% have more than 10 000 paths. Nearly 60% of the loops traverse arrays or strings and roughly half of the loops contain a function call.

SUDS: An Infrastructure for Creating Bug Detection Tools

SUDS is a powerful infrastructure for creating dynamic bug detection tools. It contains phases for both static analysis and dynamic instrumentation allowing users to create tools that take advantage of both paradigms. The results of static analysis phases can be used to improve the quality of dynamic bug detection tools created with SUDS and could be expanded to find defects statically. The instrumentation engine is designed in a manner that allows users to create their own correctness models quickly but is flexible to support construction of a wide range of different tools. The effectiveness of SUDS is demonstrated by showing that it is capable of finding bugs and that performance is improved when static analysis is used to eliminated unnecessary instrumentation.

Assessing work for static software bug detection

A common static software bug detection technique is to use path simulation. Each execution path is simulated using symbolic variables to determine if any software errors could occur. The scalability of these approaches is dependent on the number of paths in the program. This paper uses number of paths in a program to estimate the amount of work necessary to determine if a particular pointer or array operation is safe. A unique aspect in the study is that we explore the amount of work necessary for verifying the operations together versus verifying operations individually. Results show that the work for the worst-case operation individually generally improves but the magnitude of this improvement is dependent on the program. Often, there are one or two functions that contribute a large percentage of the paths. The results depend on how critical these functions are to verifying an individual operation.

MDAT: a multithreading debugging and testing tool

MDAT is a multithreaded testing and debugging tool designed for students learning to program with multiple threads. MDAT automatically generates random schedules to allow students to more thoroughly test their programs. The design of MDAT takes full control over the scheduling allowing a failing run to be reproduced. To assist debugging, MDAT includes an output trace that shows the status of all threads, locks, and semaphores in the program and has an interactive mode that allows students to try out their own schedules. MDAT was effective at detecting deadlock and mutual exclusion violations in student submissions of the unisex restroom problem.

SUDS: an infrastructure for creating dynamic software defect detection tools

SUDS is a powerful infrastructure for creating dynamic software defect detection tools. It contains phases for both static analysis and dynamic instrumentation allowing users to create tools that take advantage of both paradigms. The results of static analysis phases can be used to improve the quality of dynamic defect detection tools created with SUDS by focusing the instrumentation on types of defects, sources of data, or regions of code. The instrumentation engine is designed in a manner that allows users to create their own correctness models quickly but is flexible to support construction of a wide range of different tools. The effectiveness of SUDS is demonstrated by showing that it is capable of finding bugs and that performance improves when static analysis is used to eliminate unnecessary instrumentation.

A plethora of paths

A common static software bug detection technique is to use path simulation. Each execution path is simulated using symbolic variables to determine if any software errors could occur. The scalability of this and other path-based approaches is dependent on the number of paths in the program. This paper explores the number of paths in 15 different programs. Often, there are one or two functions that contribute a large percentage of the paths within a program. A unique aspect in this study is that slicing was used in different ways to determine its effect on the path count. In particular, slicing was applied to each interesting statement individually to determine if that statement could be analyzed without suffering from path explosion. Results show that slicing is only effective if it can completely slices away a function that suffers from path explosion. While most programs had several statements that resulted in short path counts, slicing was not adequate in eliminating path explosion occurrences. Looking into the tasks that suffer from path explosion, we find that functions that process input, produce stylized output, or parse strings or code often have significantly more paths than other functions.

An undergraduate course on software bug detection tools and techniques

The importance of software bug detection tools is high with the constant threat of malicious activity. Companies are increasingly relying on software bug detection tools to catch exploitable bugs before the program is released. This paper describes a course on software bug detection techniques that is aimed at undergraduates. Courses in software verification are often taught at the graduate level and too theoretical and research oriented for undergraduates. A key component of the course is the programming assignments where students gain practical experience in creating their own software bug detection tools using a source to source converter for a subset of C++.

Program Analysis Too Loopy? Set the Loops Aside

Among the many obstacles in efficient and sound program analysis, loops may be the most prevalent. In program analyses that traverse paths, loops introduce a variable, possibly infinite, number of paths. This paper looks at whether loops could be analyzed separately and replaced with a summary. First, the complexity of a loop is estimated by counting the paths through the body of the loop. 85% of the loops have fewer than ten paths and less than 1% have more than 10,000 paths. Second, the number of paths is computed by analyzing loops separately to assess the feasibility of such an analysis approach. While the number of paths is decreased in many cases, it is typically not sufficient for long, complex functions. Finally, loops are classified based on their stopping condition and further analyzed for programming elements that may make loop analysis more difficult. Nearly half of the loops are array traversals and over half of the loops contain a function call.

A simple but realistic assembly language for a course in computer organization

Computer science curriculums, constantly evolving to include new material and methodologies, have reduced the amount of time spent on low-level computer hardware and organization. Our institution recently combined a course on computer organization and a course on assembly language programming into one course covering both topics. The choice of assembly language is a critical decision that contributes to the success of the course. ANNA (a new noncomplex architecture) is a new 16-bit instruction set architecture that is similar to MIPS but has fewer instructions. The instruction set, while small, is sufficient in illustrating how high-level languages are translated into assembly, how to design a CPU datapath, and how to implement pipelining. Real-world assembly languages such as IA-32 or MIPS have many subtleties that complicate the learning experience for students and consume valuable class time. This paper describes the ANNA assembly language and the assembler and simulator tools that can be used in the classroom. In addition, the paper describes how ANNA can be used effectively in a combined course on assembly language programming and computer organization. ANNA was used in three courses with very positive results based on our observations and feedback from students.