Jooyoung Seo

A profiling method by PCB hooking and its application for memory fault detection in embedded system operational test

Context: An operational test means a system test that examines whether or not all software or hardware components comply with the requirements given to a system which is deployed in an operational environment. Objective: It is a necessary lightweight-profiling method for embedded systems with severe resource restrictions to conduct operational testing. Method: We focus on the Process Control Block as the optimal location to monitor the execution of all processes. We propose a profiling method to collect the runtime execution information of the processes without interrupting the systems operational environment by hacking the Process Control Block information. Based on the proposed method applied to detect runtime memory faults, we develop the operational testing tool AMOS v1.0 which is currently being used in the automobile industry. Results: An industrial field study on 23 models of car-infotainment systems revealed a total of 519 memory faults while only slowing down the system by 0.084-0.132x. We conducted a comparative analysis on representative runtime memory fault detection tools. This analysis result shows our proposed method that has relatively low overhead meets the requirements for operational testing, while other methods failed to satisfy the operational test conditions. Conclusion: We conclude that a lightweight-profiling method for embedded system operational testing can be built around the Process Control Block.

Which Spot Should I Test for Effective Embedded Software Testing

Today, the embedded industry is changing fast - systems have become larger, more complex, and more integrated. Embedded system consists of heterogeneous layers such as hardware, HAL, device driver, OS kernel, and application. These heterogeneous layers are usually customized for special purpose hardware. Therefore, various hardware and software components of embedded system are mostly integrated together under unstable status. That is, there are more possibilities of faults in all layers unlike package software.In this paper, we propose the embedded software interface as two essential parts: interface function that represents the statement of communication between heterogeneous layers, and interface variable that represents software and/or hardware variable which are defined in different layer from integrated software and used to expected output for decision of fault. Also, we statically investigate various views of embedded software interface and demonstrate that proposed interface should be new criterion for effective embedded software testing.

Tool support for new test criteria on embedded systems: Justitia

In this paper, we introduce a testing tool to support new test criteria in interface testing technique. Justitia is an automated testing tool for embedded software that assists an integration testing on software phase during embedded software testing cycle. Justitia has been successfully applied to device driver testing cooperated with Samsung Co.

Lightweight embedded software performance analysis method by kernel hack and its industrial field study

Despite advances in software testing technologies, there are still limitations in directly applying them to embedded software. Since the operational environment of embedded software has severe resource constraints, it is necessary to develop a lightweight testing method that has little impact on the operational environment of embedded software. We propose an agent-based performance analysis method to hack kernel performance counters that manage the systems execution information. The proposed method enables us to collect data required for analyzing performance bottlenecks and identify the causes and locations of bottlenecks with little impact on the test target systems operational environment. We introduce a test automation tool called Analytic Master of System v2.0 that we developed by employing our proposed method. Presently, Analytic Master of System v2.0 is being utilized as a standard tool for performance testing of embedded systems in the automotive industry. In addition, we suggest a guideline for performance analysis and improvement by introducing an industrial field study among our best practices, which analyze the relationship between the memory fault processing of the operating system and the application processing speed.

Software generated device exception for more intensive device-related software testing: An industrial field study

It is more important to properly handle exceptions, than to prevent exceptions from occurring, because they arise from so many different causes. In embedded systems, a vast number of exceptions are caused by hardware devices. In such cases, numerous software components are involved in these hardware device-originated exceptions, ranging from the device itself to the device driver, the kernel, and applications. Therefore, it takes a lot of time to debug software that fails to handle exceptions. This paper proposes a lightweight device exception testing method, and a related automation tool, AMOS v3.0. The proposed method artificially triggers more realistic device exceptions in runtime, and monitors how software components handle exceptions in detail. AMOS v3.0 has been applied to the exception testing of car-infotainment systems in an automobile company. The results based on this industrial field study have revealed that 39.13% of the failures in exception handling were caused by applications, 36.23% of the failures were caused by device drivers, and 24.64% were derived from the kernel. We conclude that the proposed method is highly effective, in that it can allow developers to identify the root cause of failure for exception handling.