Elizabeth N. Fong

Web Application Scanners: Definitions and Functions

There are many commercial software security assurance tools that claim to detect and prevent vulnerabilities in application software. However, a closer look at the tools often leaves one wondering which tools find what vulnerabilities. This paper identifies a taxonomy of software security assurance tools and defines one type of tool: Web application scanner, i.e., an automated program that examines Web applications for security vulnerabilities. We describe the types of functions that are generally found in a Web application scanner and how to test it

Information management directions: the integration challenge

On October 31 - November 2, 1988, the National Computer Systems Laboratory of the National Institute of Standards and Technology, formerly the National Bureau of Standards, in cooperation with the Association for Computing Machinery Special Interest Group on Management of Data, the IEEE Computer Society Technical Committee on Database Engineering, and the Federal Data Management Users Group, held the fifth in the series of Information Management Directions (formerly called Data Base Directions) workshops. The purpose of these workshops is to examine in depth key trends and strategies that affect the future of the information management profession. The focus of Workshop 5 was on issues related to integration and productivity. The complete workshop report, INFORMATION MANAGEMENT DIRECTIONS: THE INTEGRATION CHALLENGE, has been published by the National Institute of Standards and Technology as Special Publication 500-167, and can be ordered from the Superintendent of Documents, U.S. Government Printing Office, Washington, DC 20402.

Software Assurance Using Structured Assurance Case Models

Software assurance is an important part of the software development process to reduce risks and ensure that the software is dependable and trustworthy. Software defects and weaknesses can often lead to software errors and failures and to exploitation by malicious users. Testing, certification and accreditation have been traditionally used in the software assurance process to attempt to improve software trustworthiness. In this paper, we examine a methodology known as a structured assurance model, which has been widely used for assuring system safety, for its potential application to software assurance. We describe the structured assurance model and examine its application and use for software assurance. We identify strengths and weaknesses of this approach and suggest areas for further investigation and testing.

Structured Assurance Case Methodology for Assessing Software Trustworthiness

We describe ongoing NIST research into software assurance as an important requirement for evaluating software trustworthiness. We describe our efforts to apply a structured assurance case toward assuring selected software properties.

Database Management Systems in Engineering

Most engineering-related software addresses specific problems. These problems are typically computation-intensive and limited in scope. Until relatively recently this approach has been an effective use of computer and human resources. However, in the future, engineering and manufacturing processes will need more integrated product development environments. Both cultural and procedural changes are needed to support the engineering environments of the future, and these changes will require integrated software systems. Databases are essential for integrating software and for reliably sharing data among diverse groups of people and applications. Database technology will be an integral part of the emerging software environments. In this article the application of database technology to engineering problems is examined for different levels of complexity within the computing environment. This introduction provides some background on the topic and includes the description of an example that is used throughout the article. In the first section, the use of database technology for standalone applications is considered. Mechanisms for data representation to support engineering applications are particularly important for implementing engineering software. The second section discusses database techniques for managing changes within the software environment. The third section discusses considerations for supporting multiple engineers working cooperatively. The state of database technology is discussed in the concluding section. Keywords: engineering problem; database schema; physical organization; change management; schema evolution; cooperative engineering environment; tools; standard interfaces; commercial databases; state-of-the-art

Evaluating bug finders: test and measurement of static code analyzers

Software static analysis is one of many options for finding bugs in software. Like compilers, static analyzers take a program as input. This paper covers tools that examine source code - without executing it - and output bug reports. Static analysis is a complex and generally undecidable problem. Most tools resort to approximation to overcome these obstacles and it sometimes leads to incorrect results. Therefore, tool effectiveness needs to be evaluated. Several characteristics of the tools should be examined. First, what types of bugs can they find? Second, what proportion of bugs do they report? Third, what percentage of findings is correct? These questions can be answered by one or more metrics. But to calculate these, we need test cases having certain characteristics: statistical significance, ground truth, and relevance. Test cases with all three attributes are out of reach, but we can use combinations of only two to calculate the metrics. The results in this paper were collected during Static Analysis Tool Exposition (SATE) V, where participants ran 14 static analyzers on the test sets we provided and submitted their reports to us for analysis. Tools had considerably different support for most bug classes. Some tools discovered significantly more bugs than others or generated mostly accurate warnings, while others reported wrong findings more frequently. Using the metrics, an evaluator can compare candidates and select the tool that aligns best with his or her objectives. In addition, our results confirm that the bugs most commonly found by tools are among the most common and important bugs in software. We also observed that code complexity is a major hindrance for static analyzers and detailed which code constructs tools handle well and which impede their analysis.

Large Scale Generation of Complex and Faulty PHP Test Cases

Developing good test cases is an intellectually demanding and critical task, and it has a strong impact on the effectiveness and efficiency of the whole testing process. This paper presents an automated generator of test cases, which are designed to evaluate source code security analyzers. The generator produces PHP: Hypertext Preprocessor (PHP) programs with most common vulnerabilities embedded in various code complexities. It also produces programs without vulnerabilities to test for false positives. The generator is modular and extensible. We describe its internal design and how it works. The generated PHP test cases were added to the Software Assurance Reference Dataset (SARD) and will be used to assess the effectiveness of static analyzers. We conclude with the current state of the tool, its benefits and future work.

Of Massive Static Analysis Data

The Software Assurance Metrics and Tool Evaluation (SAMATE) project at the National Institute of Standards and Technology (NIST) has organized four Static Analysis Tool Expositions (SATE). SATE is designed to advance research in static analysis tools that find security-relevant defects in source code. Briefly, participating tool makers run their tools on a set of programs. Researchers led by NIST analyze the tool outputs. The results and experiences are reported at a workshop. These expositions have accumulated large amounts of data. This collection allowed for the development and validation of practical metrics in regard to static analysis tool effectiveness and independence. In this paper, we discuss the role of the data in determining which metrics can be derived. Specifically, we detail the three characteristics test data should exhibit and explain why the data we use express each combination of two out of these three properties.

Agent-based services for B2B electronic commerce

The potential of agent-based systems has not been realized yet, in part, because of the lack of understanding of how the agent technology supports industrial needs and emerging standards. The area of business-to-business electronic commerce (b2b e-commerce) is one of the most rapidly developing sectors of industry with huge impact on manufacturing practices. In this paper, we investigate the current state of agent technology and the feasibility of applying agent-based computing to b2b e-commerce in the circuit board manufacturing sector. We identify critical tasks and opportunities in the b2b e-commerce area where agent-based services can best be deployed. We describe an implemented agent-based prototype system to facilitate the bidding process for printed circuit board manufacturing and assembly. These activities are taking place within the Internet Commerce for Manufacturing (ICM) project, the NIST- sponsored project working with industry to create an environment where small manufacturers of mechanical and electronic components may participate competitively in virtual enterprises that manufacture printed circuit assemblies.

Data base directions information resource management-making it work, executive summary

On October 21-23, 1985, the Institute for Computer Sciences and Technology of the National Bureau of Standards (NBS), in cooperation with the Association for Computing Machinery Special Interest Group on Management of Data (ACM SIGMOD), the IEEE Computer Society Technical Committee on Database Engineering, and the Federal Data Management Users Group (FEDMUG), held the fourth in their series of Data Base Directions workshops. The purpose of this workshop was to assess the nature of current information resource management (IRM) practice and problems, and to explore solutions which have proven workable.