Reproducible Execution of POSIX Programs with DiOS
Petr Ročkai, Zuzana Baranová, Jan Mrázek, Katarína Kejstová, Jiří Barnat
RReproducible Execution of
POSIX
Programswith
DiOS (cid:63)
Petr Roˇckai, Zuzana Baranov´a, Jan Mr´azek,Katar´ına Kejstov´a, and Jiˇr´ı Barnat
Faculty of Informatics, Masaryk UniversityBrno, Czech Republic { xrockai,xbaranov,xmrazek7,xkejstov,barnat } @fi.muni.cz Abstract.
In this paper, we describe
DiOS , a lightweight model oper-ating system which can be used to execute programs that make use of
POSIX
APIs. Such executions are fully reproducible: running the sameprogram with the same inputs twice will result in two exactly identicalinstruction traces, even if the program uses threads for parallelism.
DiOS is implemented almost entirely in portable C and C++: althoughits primary platform is
DiVM , a verification-oriented virtual machine, itcan be configured to also run in
KLEE , a symbolic executor. Finally, itcan be compiled into machine code to serve as a user-mode kernel.Additionally,
DiOS is modular and extensible. Its various components canbe combined to match both the capabilities of the underlying platformand to provide services required by a particular program. New compo-nents can be added to cover additional system calls or APIs.The experimental evaluation has two parts.
DiOS is first evaluated asa component of a program verification platform based on
DiVM . In thesecond part, we consider its portability and modularity by combining itwith the symbolic executor
KLEE . Real-world software has a strong tendency to interact with its execution en-vironment in complex ways. To make matters worse, typical environments inwhich programs execute are often extremely unpredictable and hard to control.This is an important factor that contributes to high costs of software validationand verification. Even the most resilient verification methods (those based ontesting) see substantial adverse effect.In automated testing, one of the major criteria for a good test case is thatit gives reliable and reproducible results, without intermittent failures. This isespecially true in the process of debugging: isolating a fault is much harder whenit cannot be consistently observed. For this reason, significant part of the effortinvolved in testing is spent on controlling the influence of the environment onthe execution of test cases. (cid:63)
This work has been partially supported by the Czech Science Foundation grantNo. 18-02177S and by Red Hat, Inc. a r X i v : . [ c s . O S ] J u l he situation is even worse with more rigorous verification methods – forinstance, soundness of verification tools based on dynamic analysis strongly de-pends on the ability to fully control the execution of the system under test.In this paper, we set out to design and implement a small and sufficiently self-contained model operating system that can provide a realistic environment forexecuting POSIX -based programs. Since this environment is fully virtualised andisolated from the host system, program execution is always fully reproducible.As outlined above, such reproducibility is valuable, sometimes even essential,in testing and program analysis scenarios. Especially dynamic techniques, likesoftware model checking or symbolic execution, rely on the ability to replayinteractions of the program and obtain identical outcomes every time.
The paper describes our effort to implement a compact operating system ontop of existing verification frameworks and virtual machines (see Section 3).Despite its minimalist design, the current implementation covers a wide rangeof
POSIX
APIs in satisfactory detail (see also Section 4.3). The complete sourcecode is available online , under a permissive open-source licence. Additionally,we have identified a set of low-level interfaces (see Section 2) with two importantqualities:1. the interfaces are lightweight and easy to implement in a VM,2. they enable an efficient implementation of complex high-level constructs.Minimal interfaces are a sound design principle and lead to improved mod-ularity and re-usability of components. In our case, identification of the correctinterfaces drives both portability and compactness of implementation .Finally, the design that we propose improves robustness of verificationtools. A common implementation strategy treats high-level constructs (e.g. the pthread API) as primitives built into the execution engine. This ad-hoc ap-proach often leads to implementation bugs which then compromise the soundnessof the entire tool. Our design, on the other hand, emphasises clean separation ofconcerns and successfully reduces the amount of code which forms the trustedexecution and/or verification core.
We would like our system to have the following properties:1. Modularity: minimise the interdependence of the individual OS components.It should be as easy as possible to use individual components (for instance libc ) without the others. The kernel should likewise be modular. https://divine.fi.muni.cz/2019/dios/
2. Portability: reduce the coupling to the underlying platform (verification en-gine), making the OS useful as a pre-made component in building verificationand testing tools.3. Veracity: the system should precisely follow
POSIX and other applicablestandardised semantics. It should be possible to port realistic programs torun on the operating system with minimal effort.Since the desired properties are hard to quantify, we provide a qualitativeevaluation in Section 5. To demonstrate the viability of our approach, we showthat many UNIX programs, e.g. gzip or a number of programs from the GNU coreutils suite can be compiled for
DiOS with no changes and subsequentlyanalysed using an explicit-state model checker.
Execution reproducibility is a widely studied problem. A number of tools capture provenance , or history of the execution, by following and recording program’sinteractions with the environment, later using this information to reproduce therecorded execution. For instance, ReproZip [4] bundles the environment vari-ables, files and library dependencies it observes so that the executable can berun on a different system. Other programs exist, that instead capture the prove-nance in form of logs [7], or sometimes more complex structures – provenancegraphs in case of ES3 [5].SCARPE [7] was developed for Java programs and captures I/O, user inputsand interactions with the database and the filesystem into a simple event log.The user has to state which interactions to observe by annotating the individualclasses that make up the program, since the instrumentation introduces substan-tial overhead and recording all interactions may generate a considerable amountof data (for example, capturing a large portion of the database).Another common approach to dealing with the complexity of interactionswith the execution environment is mocking [13, 14]: essentially building smallmodels of the parts of the environment that are relevant in the given test scenario.A mock object is one step above a stub, which simply accepts and discards allrequests. A major downside of using mock objects in testing is that sufficientlymodelling the environment requires a lot of effort: either the library only providessimple objects and users have to model the system themselves, or the mocksystem is sophisticated, but the user has to learn a complex API.Most testing frameworks for mainstream programming languages offer a de-gree of support for building mock objects, including mock objects which modelinteraction with the operating system. For instance the pytest tool [11] forPython allows the user to comfortably mock a database connection. A morecomplex example of mocking would be the filesystem support in Pex [10], a sym-bolic executor for programs targeting the .NET platform.
KLEE is a symbolicexecutor based on
LLVM and targets C (and to some degree C++) programswith a different approach to environment interaction. Instead of modelling thefile system or other operating system services, it allows the program to directly3nteract with the host operating system, optionally via a simple adaptation layerwhich provides a degree of isolation based on symbolic file models.This latter approach, where system calls and even library calls are forwardedto the host operating system is also used in some runtime model checkers, mostnotably Inspect [18] and CHESS [15]. However, those approaches, only workwhen the program interacts with the operating system in a way free from sideeffects, and when external changes in the environment do not disturb verification.Finally, standard (offline) model checkers rarely support more than a handfulof interfaces. The most widely supported is the
POSIX threading API, which ismodelled by tools such as Lazy-CSeq [6] and its variants, by Impara [17] and bya few other tools.
In this section, we will describe our expectations of the execution or verificationplatform and the low-level interface between this platform and our model oper-ating system. We then break down the interface into a small number of areas,each covering particular functionality.
The underlying platform can have different characteristics. We are mainly inter-ested in platforms or tools based on dynamic analysis, where the program is atleast partially interpreted or executed, often in isolation from the environment.If the platform itself isolates the system under test, many standard facilities likefile system access become unavailable. In this case, the role of
DiOS is to providea substitute for the inaccessible host system.If, on the other hand, the platform allows the program to access the hostsystem, this easily leads to inconsistencies, where executions explored first caninterfere with the state of the system observed by executions explored later. Forinstance, files or directories might be left around, causing unexpected changes inthe behaviour of the system under test. In cases like those, DiOS can serve toinsulate such executions from each other. Under
DiOS , the program can observethe effects of its actions along a single execution path – for instance, if theprogram creates a file, it will be able to open it later. However, this file neverbecomes visible to another execution of the same program, regardless of theexploration order.Unfortunately, not all facilities that operating systems provide to programscan be modelled entirely in terms of standard C. To the contrary, certain areasof high-level functionality that the operating system is expected to implementstrongly depend on low-level aspects of the underlying platform. Some of thoseare support for thread scheduling, process isolation, control flow constructs suchas setjmp and C++ exceptions, among others. We will discuss those in moredetail in the following sections. If execution A creates a file and leaves it around, execution B might get derailed whenit tries to create the same file, or might detect its presence and behave differently. .2 Program Memory An important consideration when designing an operating system is the semanticsof the memory subsystem of its execution platform.
DiOS is no exception: it needsto provide a high-level memory management API to the application (both theC malloc interface and the C++ new / delete interface). In principle, a singleflat array of memory is sufficient to implement all the essential functionality.However, it lacks both in efficiency and in robustness. Ideally, the platform wouldprovide a memory management API that manages individual memory objectswhich in turn support an in-place resize operation. This makes operations moreefficient by avoiding the need to make copies when extra memory is required,and the operating system logic simpler by avoiding a level of indirection.If the underlying platform is memory safe and if it provides a supervisormode to protect access to certain registers or to a special memory location,the remainder of kernel isolation is implemented by DiOS itself, by withholdingaddresses of kernel objects from the user program. In this context, memory safetyentails bound checks and an inability to overflow pointers from one memoryobject into another.
Information about active procedure calls and about the local data of each pro-cedure are, on most platforms, stored in a special execution stack . While thepresence of such a stack is almost universal, the actual representation of thisstack is very platform-specific. On most platforms that we consider, it is part ofstandard program memory and can be directly accessed using standard memoryoperations. If both reading and modifications of the stack (or stacks) is possible,most of the operations that DiOS needs to perform can be implemented withoutspecial assistance from the platform itself. Those operations are: – creation of a new execution stack is needed in two scenarios: isolation of thekernel stack from the user-space stack and creation of new tasks (threads,co-routines or other similar high-level constructs), – stack unwinding, where stack frames are traversed and removed from thestack during exception propagation or due to setjmp / longjmp .Additionally, DiOS needs a single operation that must be always provided bythe underlying platform: it needs to be able to transfer control to a particularstack frame, whether within a single execution stack (to implement non-localcontrol flow) or to a different stack entirely (to implement task switching).In some sense, this part of the platform support is the most complex and mosttricky to implement. Fortunately, the features that rely on the above operations,or rather the modules which implement those features, are all optional in
DiOS . The main exception is
KLEE , where the execution stack is completely inaccessibleto the program under test and only the virtual machine can access the informationstored in it. See also Section 3.2. .4 Auxiliary Interfaces There are three other points of contact between
DiOS and the underlying plat-form. They are all optional or can be emulated using standard C features, butif available,
DiOS can use them to offer additional facilities mainly aimed atsoftware verification and testing with fault injection.
Indeterminate values.
A few components in
DiOS use, or can be configured touse, values which are not a priori determined. The values are usually subject toconstraints, but within those constraints, each possible value will correspond toa particular interaction outcome. This facility is used for simulating interactionsthat depend on random chance (e.g. thread scheduling, incidence of clock ticksrelative to the instruction stream), or where the user would prefer to not providespecific input values and instead rely on the verification or testing platform toexplore the possibilities for them (e.g. the content of a particular file).
Nondeterministic choice.
A special case of the above, where the selection isamong a small number of discrete options. In those cases, a specific interface cangive better user experience or better tool performance. If the choice operator isnot available but indeterminate values are, they can be used instead. Otherwise,the sequence of choices can be provided as an input by the user, or they can beselected randomly. The choice operation is used for scheduling choices and forfault injection (e.g. simulation of malloc failures).
Host system call execution.
Most
POSIX operating systems provide an indirectsystem call facility, usually as the C function syscall() . If the platform makesthis function accessible from within the system under test,
DiOS can use it toallow real interactions between the user program and the host operating systemto take place and to record and then replay such interactions in a reproduciblemanner.
In the previous section, we have described the target platform in generic, abstractterms. In this section, we describe 3 specific platforms which can execute
DiOS and how they fit with the above abstract requirements.
DiVM
DiVM [16] is a verification-oriented virtual machine based on
LLVM . A suite oftools based on
DiVM implement a number of software verification techniques,including explicit-state, symbolic and abstraction-based model checking.
DiVM is the best supported of all the platforms, since it has been specifically designedto delegate responsibility for features to a model operating system. All featuresavailable in
DiOS are fully supported on this platform.6n
DiVM , the functionality that is not accessible through standard C (or
LLVM ) constructs is provided via a set of hypercalls . These hypercalls form thecore of the platform interface in
DiOS and whenever possible, ports to otherplatforms are encouraged to emulate the
DiVM hypercall interface using theavailable platform-native facilities.
KLEE
KLEE [3] is a symbolic executor based on
LLVM , suitable both for automatedtest generation and for exhaustive exploration of bounded executions. Unlike
DiVM , KLEE by default allows the program under test to perform external calls(including calls to the host operating system), with no isolation between differentexecution branches. Additionally, such calls must be given concrete arguments,since they are executed as native machine code (i.e. not symbolically). However,if the program is linked to
DiOS , both these limitations are lifted:
DiOS code canbe executed symbolically like the rest of the program, and different executionbranches are isolated from each other.However, there is also a number of limitations when
KLEE is considered as aplatform for
DiOS . The two most important are as follows:1.
KLEE does not currently support in-place resizing of memory objects. This isa design limitation and lifting it requires considerable changes. A workaroundexists, but it is rather inefficient.2. There is only one execution stack in
KLEE and there is no support for non-local control flow. This prevents
DiOS from offering threads, C++ exceptionsand setjmp when executing in
KLEE .Additionally, there is no supervisor mode and hence no isolation between thekernel and the user program. However, in most cases, this is not a substantialproblem. Non-deterministic choice is available via indeterminate symbolic valuesand even though
KLEE can in principle provide access to host syscalls, we havenot evaluated this functionality in conjunction with
DiOS . Finally, there are afew minor issues that are, however, easily corrected: KLEE does not support the va arg
LLVM instruction and relies on emulatingplatform-specific mechanisms instead, which are absent from
DiOS ,2. it handles certain C functions specially, including the malloc family, theC++ new operator, the errno location and functions related to assertionsand program termination; this interferes with the equivalent functionalityprovided by
DiOS libc , and finally3. global constructors present in the program are unconditionally executed be-fore the entry function; since
DiOS invokes constructors itself, this
KLEE behaviour also causes a conflict. A version of
KLEE with fixes for those problems is available online, along with othersupplementary material, from https://divine.fi.muni.cz/2019/dios/ . .3 Native Execution The third platform that we consider is native execution, i.e. the
DiOS kernelis compiled into machine code, like a standard user-space program, to executeas a process of the host operating system. This setup is useful in testing or instateless model checking, where it can provide superior execution speed at theexpense of reduced runtime safety. The user program still uses
DiOS libc andthe program runs in isolation from the host system. The platform-specific codein
DiOS uses a few hooks provided by a shim which calls through into the hostoperating system for certain services, like the creation and switching of stacks.The design is illustrated in Figure 1. program under testDiOS libc + kernel bitcode isolated executableplatform-specific code host shim host libc
Fig. 1.
Architecture of the native execution platform.Like in
KLEE , the native port of
DiOS does not have access to in-place resizingof memory objects, but it can be emulated slightly more efficiently using the mmap host system call. The native port, however, does not suffer from the single stacklimitations that
KLEE does: new stacks can be created using mmap calls and stackswitching can be implemented using host setjmp and longjmp functions. Thehost stack unwinding code is directly used (the
DiVM platform code implementsthe same libunwind
API that most
POSIX systems also use).On the other hand, non-deterministic choice is not directly available. It canbe simulated by using the fork host system call to split execution, but this doesnot scale to frequent choices, such as those arising from scheduling decisions.In this case, a random or an externally supplied list of outcomes are the onlyoptions.
This section outlines the structure of the
DiOS kernel and userspace, their com-ponents and the interfaces between them. We also discuss how the kernel in-teracts with the underlying platform and the user-space libraries stacked aboveit. A high-level overview of the system is shown in Figure 2. The kernel andthe user-mode parts of the system under test can be combined using different The details of how this is done are quite technical, and are discussed in the onlinesupplementary material at https://divine.fi.muni.cz/2019/dios/ . DiOS , this protection isoptional, since not all platforms provide supervisor mode or sufficient memorysafety; however, it does not depend on address space separation between thekernel and the user mode. file systemschedulerother components C99 IOPOSIX IO, syscallspthread non-IO C99 libmalloc libc++libc++abiplatform-specific codeexecution platformkernel libc C++ support
Fig. 2.
The architecture of
DiOS . The decomposition of the kernel to a number of components serves multiplegoals: first is resource conservation – some components have non-negligible mem-ory overhead even when they are not actively used. This may be because theyneed to store auxiliary data along with each thread or process, and the under-lying verification tool then needs to track this data throughout the execution orthroughout the entire state space. The second is improved portability to plat-forms which do not provide sufficient support for some of the components, forinstance thread scheduling. Finally, it allows
DiOS to be reconfigured to serve innew contexts by adding a new module and combining it with existing code.The components of the kernel are organised as a stack, where upper compo-nents can use services of the components below them. While this might appear tobe a significant limitation, in practice this has not posed substantial challenges,and the stack-organised design is both efficient and simple. A number of pre-made components are available, some in multiple alternative implementations:
Task scheduling and process management.
There are 4 scheduler implementa-tions: the simplest is a null scheduler, which only allows a single task and doesnot support any form of task switching. This scheduler is used on
KLEE . Secondis a synchronous scheduler suitable for executing software models of hardwaredevices. The remaining two schedulers both implement asynchronous, thread-based parallelism. One is designed for verification of safety properties of parallel9rograms, while the other includes a fairness provision and is therefore moresuitable for verification of liveness properties.In addition to the scheduler, there is an optional process management com-ponent. It is currently only available on the
DiVM platform, since it heavily relieson operations which are not available elsewhere. It implements the fork systemcall and requires one of the two asynchronous schedulers.
POSIX
System Calls.
While a few process-related system calls are implementedin the components already mentioned, the vast majority is not. By far the largestcoherent group of system calls deals with files, directories, pipes and sockets,where the unifying concept is file descriptors. A memory-backed filesystem mod-ule implements those system calls by default.A smaller group of system calls relate to time and clocks and those are im-plemented in a separate component which simulates a system clock. The specificsimulation mode is configurable and can use either indeterminate values to shiftthe clock every time it is observed or a simpler variant where ticks of fixed lengthare performed based on the outcome of a nondeterministic choice.The system calls covered by the filesystem and clock modules can be alter-nately provided by a proxy module, which forwards the calls to the host oper-ating system or by a replay module which replays traces captured by the proxy module.
Auxiliary modules.
There is a small number of additional modules which do notdirectly expose functionality to the user program. Instead, they fill in supportroles within the system. The two notable examples are the fault handler and the system call stub component.The fault handler takes care of responding to error conditions indicated by theunderlying platform. It is optional, since not all platforms can report problemsto the system under test. If present, the component allows the user to configurewhich problems should be reported as counterexamples and which should beignored. The rest of
DiOS also uses this component to report problems detectedby the operating system itself, e.g. the libc uses it to flag assertion failures.The stub component supplies fallback implementations of all system callsknown to
DiOS . This component is always at the bottom of the kernel config-uration stack – if any other component in the active configuration implementsa particular system call, that implementation is used. Otherwise, the fallback iscalled and raises a runtime error, indicating that the system call is not supported.
One of the innovative features of
DiOS is that it implements the
POSIX thread-ing API using a very simple platform interface. Essentially, the asynchronousschedulers in
DiOS provide an illusion of thread-based parallelism to the pro-gram under test, but only use primitives associated with coroutines – creationand switching of execution stacks (cf. Section 2.3).10owever, an additional external component is required: both user and li-brary code needs to be instrumented with interrupt points , which allow threadpreemption to take place. Where to insert them can be either decided statically(which is sufficient for small programs) or dynamically, allowing the state spaceto be reduced using more sophisticated techniques. The implementation of theinterrupt point is, however, supplied by
DiOS : only the insertion of the functioncall is done externally.The scheduler itself provides a very minimal internal interface – the remain-der of thread support is implemented in user-space libraries (partly libc andpartly libpthread , as is common on standard
POSIX operating systems). Eventhough the implementation is not complete (some of the rarely-used functionsare stubbed out), all major areas are well supported: thread creation and can-cellation, mutual exclusion, condition variables, barriers, reader-writer locks, in-teraction with fork , and thread-local storage are all covered. Additionally, bothC11 and C++11 thread APIs are implemented in terms of the pthread interface.
The system call interface of
DiOS is based on the ideas used in fast system call implementations on modern processors. A major advantage of this approach isthat system calls can be performed using standard procedure calls on platformswhich do not implement supervisor mode.The list of system calls available in
DiOS is fixed: in addition to the kernel-side implementation, which may or may not be available depending on the activeconfiguration, each system call has an associated user-space C function, whichis declared in one of the public header files and implemented in libc .The available system calls cover thread management, sufficient to imple-ment the pthread interface (the system calls themselves are not standardisedby POSIX ), the fork system call, kill and other signal-related calls, various pro-cess and process group management calls ( getpid , getsid , setsid , wait andso on). Notably, exec is currently not implemented and it is not clear whetheradding it is feasible on any of the platforms. The thread- and process- relatedfunctionality was described in more detail in Section 4.2.Another large group of system calls cover files and networking, including thestandard suite of POSIX calls for opening and closing files, reading and writ-ing data, creating soft and hard links. This includes the *at family introducedin
POSIX .1 which allows thread-safe use of relative paths. The standard BSD In DIVINE [1], a model checker based on
DiVM , interrupt points are dynamicallyenabled when the executing thread performs a visible action. Thread identificationis supplied by the scheduler in
DiOS using a platform-specific (hypercall) interface. For instance, on contemporary x86-64 processors, this interface is available via the syscall and sysret instructions. The list of system calls is only fixed relative to the host operating system. To allowthe system call proxy component to function properly, the list needs to match whatis available on the host. For instance, creat , uname or fdatasync are system callson Linux but standard libc functions on OpenBSD. clock gettime , gettimeofday ) and setting clocks ( clock settime , settimeofday ). DiOS comes with a complete ISO C99 standard library and the C11 threadAPI. The functionality of the C library can be broken down into the followingcategories: – Input and output. The functionality required by ISO C is implemented interms of the
POSIX file system API. Number conversion (for formatted inputand output) is platform independent and comes from pdclib . – The string manipulation and character classification routines are completelysystem-independent. The implementations were also taken from pdclib . – Memory allocation: new memory needs to be obtained in a platform-dependent way. Optionally, memory allocation failures can be simulated us-ing a non-deterministic choice operator. The library provides the standardassortment of functions: malloc , calloc , realloc and free . – Support for errno : this variable holds the code of the most recent errorencountered in an API call. On platforms with threads (like
DiOS ), errno is thread-local. – Multibyte strings: conversions of Unicode character sequences to and fromUTF-8 is supported. – Time-related functions: time and date formatting ( asctime ) is supported,as is obtaining and manipulating wall time. Interval timers are currently notsimulated, although the relevant functions are present as simple stubs. – Non-local jumps. The setjmp and longjmp functions are supported on
DiVM and native execution, but not in
KLEE .In addition to ISO C99, there are a few extensions (not directly related tothe system call interface) mandated by
POSIX for the C library: – Regular expressions. The
DiOS libc supports the standard regcomp & regexec APIs, with implementation based on the TRE library. – Locale support: A very minimal support for
POSIX internationalisation andlocalisation APIs is present. The support is sufficient to run programs whichinitialise the subsystem. – Parsing command line options: the getopt and getopt long functions ex-ist to make it easy for programs to parse standard UNIX-style commandswitches.
DiOS contains an implementation derived from the OpenBSD codebase.Finally, C99 mandates a long list of functions for floating point math, includ-ing trigonometry, hyperbolic functions and so on. A complete set of those func-tions is provided by
DiOS via its libm implementation, based on the OpenBSDversion of this library. 12 .5 C++ Support Libraries
DiOS includes support for C++ programs, up to and including the C++17 stan-dard. This support is based on the libc++abi and libc++ open-source librariesmaintained by the
LLVM project. The versions bundled with
DiOS contain onlyvery minor modifications relative to upstream, mainly intended to reduce pro-gram size and memory use in verification scenarios.Notably, the exception support code in libc++abi is unmodified and worksboth in
DiVM and when
DiOS is executing natively as a process of the hostoperating system. This is because libc++abi uses the libunwind library toimplement exceptions. When
DiOS runs natively, the host version of libunwind is used, the same as with setjmp . When executing in
DiVM , DiOS supplies itsown implementation of the libunwind
API, as described in [19].
When dealing with verification of real-world software, the exact layout of datastructures becomes important, mainly because we would like to generate nativecode from verified bitcode files (when using either
KLEE or DiVM ). To thisend, the layouts of relevant data structures and values of relevant constants areautomatically extracted from the host operating system and then used in the DiOS libc . As a result, the native code generated from the verified bitcodecan be linked to host libraries and executed as usual. The effectiveness of thisapproach is evaluated in Section 5.3. user program native code userDiOS headers bitcode host libcDiOS libraries DiVM bitcode verification
Fig. 3.
Building verified executables with
DiOS . We have tested
DiOS in a number of scenarios, to ensure that it meets the goalsthat we describe in Section 1.2. The first goal – modularity – is hard to quantifyin isolation, but it was of considerable help in adapting
DiOS for different usecases. We have used
DiOS with success in explicit-state model checking of parallel This extraction is performed at
DiOS build time, using hostabi.pl , which is partof the
DiOS source distribution. The technical details are discussed in the onlinesupplementary material.
DiOS has also been used for recording, replaying and fuzzing systemcall traces [8].
DiVM
In this paper, we report on 3 sets of tests that we performed particularly to eval-uate
DiOS . The first is a set of approximately 2200 test programs which covervarious aspects of the entire verification platform. Each of them was executed in DiOS running on top of
DiVM and checked for a number of safety criteria: lackof memory errors, use of uninitialized variables, assertion violations, deadlocksand arithmetic errors. In the case of parallel programs (about 400 in total), allpossible schedules were explored. Additionally, approximately 700 of the testprograms depend on one or more input values (possibly subject to constraints),in which case symbolic methods or abstraction are used to cover all feasible pathsthrough the program. The tests were performed on two host operating systems:Linux 4.19 with glibc
To evaluate the remaining ports of
DiOS , we have taken a small subset (370programs, or 17%) of the entire test suite and executed the programs on theother two platforms currently supported by
DiOS . The subset was selected tofall within the constraints imposed by the limitations of our
KLEE port – inparticular, lack of support for threads and for C++ exceptions. We have focusedon filesystem and socket support (50 programs) and exercising the standardC and C++ libraries shipped with
DiOS . The test cases have all completedsuccessfully, and
KLEE has identified all the annotated safety violations in theseprograms.
Finally to evaluate our third goal, we have compiled a number of real-worldprograms against
DiOS headers and libraries and manually checked that they All test programs are available online at http://divine.fi.muni.cz/2019/dios/ ,including scripts to reproduce the results reported in this and in the following sec-tions.
DiOS running on
DiVM , fully isolated fromthe host operating system. The compilation process itself exercises source-level(API) compatibility with the host operating system.We have additionally generated native code from the bitcode that resultedfrom the compilation using
DiOS header files (see Figure 3) and which we con-firmed to work with
DiOS libraries. We then linked the resulting machine codewith the libc of the host operating system ( glibc – coreutils – diffutils diff3 was checked to work correctly, while the patch program failed to build dueto lack of exec support on DiOS , – sed – make exec support, – the wget download program failed to build due to lack of gethostbyname support, the cryptographic library nettle failed due to deficiencies in ourcompiler driver and mtools failed due to missing langinfo.h support. We have presented
DiOS , a
POSIX -compatible operating system designed to offerreproducible execution, with special focus on applications in program verifica-tion. The larger goal of verifying unmodified, real-world programs requires thecooperation of many components, and a model of the operating system is an im-portant piece of the puzzle. As the case studies show, the proposed approach is aviable way forward. Just as importantly, the design goals have been fulfilled: wehave shown that
DiOS can be successfully ported to rather dissimilar platforms,and that its various components can be disabled or replaced with ease.Implementation-wise, there are two important future directions: further ex-tending the coverage and compatibility of
DiOS with real operating systems, andimproving support for different execution and software verification platforms. Interms of design challenges, the current model of memory management for multi-process systems is suboptimal and there are currently no platforms on which the exec family of system calls could be satisfactorily implemented. We would liketo rectify both shortcomings in the future.15 ibliography [1] Zuzana Baranov´a, Jiˇr´ı Barnat, Katar´ına Kejstov´a, Tade´aˇs Kuˇcera, HenrichLauko, Jan Mr´azek, Petr Roˇckai, and Vladim´ır ˇStill. Model checking of Cand C++ with DIVINE 4. 2017.[2] Dirk Beyer. Reliable and reproducible competition results with BenchExecand witnesses report on SV-COMP 2016. In
Tools and Algorithms for theConstruction and Analysis of Systems (TACAS) , pages 887–904. Springer,2016. ISBN 978-3-662-49673-2. doi: 10.1007/978-3-662-49674-9 55.[3] Cristian Cadar, Daniel Dunbar, and Dawson R. Engler. KLEE: Unassistedand automatic generation of high-coverage tests for complex systems pro-grams. In
Operating Systems Design and Implementation , pages 209–224.USENIX Association, 2008.[4] Fernando Chirigati, Dennis Shasha, and Juliana Freire. Reprozip: Usingprovenance to support computational reproducibility. In
Proceedings of the5th USENIX Workshop on the Theory and Practice of Provenance , TaPP’13, pages 1:1–1:4, Berkeley, CA, USA, 2013. USENIX Association. URL http://dl.acm.org/citation.cfm?id=2482949.2482951 .[5] James Frew, Dominic Metzger, and Peter Slaughter. Automatic capture andreconstruction of computational provenance.
Concurr. Comput. : Pract.Exper. , 20(5):485–496, April 2008. ISSN 1532-0626. doi: 10.1002/cpe.v20:5.URL http://dx.doi.org/10.1002/cpe.v20:5 .[6] O. Inverso, T. L. Nguyen, B. Fischer, S. L. Torre, and G. Parlato.Lazy-CSeq: A context-bounded model checking tool for multi-threadedC-programs. In , pages 807–812, Nov 2015. doi:10.1109/ASE.2015.108.[7] Shrinivas Joshi and Alessandro Orso. Scarpe: A technique and tool forselective capture and replay of program executions. pages 234 – 243, 112007. ISBN 978-1-4244-1256-3. doi: 10.1109/ICSM.2007.4362636.[8] Katar´ına Kejstov´a. Model checking with system call traces. Master’s thesis,Masarykova univerzita, Fakulta informatiky, Brno, 2019. URL http://is.muni.cz/th/tukvk/ .[9] Katar´ına Kejstov´a, Petr Roˇckai, and Jiˇr´ı Barnat. From model checking toruntime verification and back. In
Runtime Verification , volume 10548 of
LNCS , pages 225–240. Springer, 2017. doi: 10.1007/978-3-319-67531-2 14.[10] Soonho Kong, Nikolai Tillmann, and Jonathan de Halleux. Automatedtesting of environment-dependent programs - a case study of modeling thefile system for pex. pages 758–762, 01 2009. doi: 10.1109/ITNG.2009.80.[11] Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe,Brianna Laugher, and Florian Bruhin. pytest 4.5, 2004. URL https://github.com/pytest-dev/pytest .[12] Henrich Lauko, Vladim´ır ˇStill, Petr Roˇckai, and Jiˇr´ı Barnat. ExtendingDIVINE with symbolic verification using SMT. In Dirk Beyer, MariekeHuisman, Fabrice Kordon, and Bernhard Steffen, editors,
Tools and Algo-rithms for the Construction and Analysis of Systems , pages 204–208, Cham,2019. Springer International Publishing. ISBN 978-3-030-17502-3.1613] Tim Mackinnon, Steve Freeman, and Philip Craig. Extreme programmingexamined. chapter Endo-testing: Unit Testing with Mock Objects, pages287–301. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA,2001. ISBN 0-201-71040-4. URL http://dl.acm.org/citation.cfm?id=377517.377534 .[14] S. Mostafa and X. Wang. An empirical study on the usage of mockingframeworks in software testing. In , pages 127–132, Oct 2014. doi: 10.1109/QSIC.2014.19.[15] Madan Musuvathi, Shaz Qadeer, Tom Ball, Gerard Basler, Pira-manayakam Arumuga Nainar, and Iulian Neamtiu. Finding and reproducingheisenbugs in concurrent programs. In
Symposium on Operating SystemsDesign and Implementation . USENIX, December 2008.[16] Petr Roˇckai, Vladim´ır ˇStill, Ivana ˇCern´a, and Jiˇr´ı Barnat. DiVM: Modelchecking with LLVM and graph memory.
Journal of Systems and Software ,143:1 – 13, 2018. ISSN 0164-1212. doi: https://doi.org/10.1016/j.jss.2018.04.026.[17] Bj¨orn Wachter, Daniel Kroening, and Joel Ouaknine. Verifying multi-threaded software with impact. In
Formal Methods in Computer-AidedDesign , pages 210–217. IEEE, 10 2013. doi: 10.1109/FMCAD.2013.6679412.[18] Yu Yang, Xiaofang Chen, and Ganesh Gopalakrishnan. Inspect: A runtimemodel checker for multithreaded c programs. Technical report, 2008.[19] Vladim´ır ˇStill, Petr Roˇckai, and Jiˇr´ı Barnat. Using off-the-shelf exceptionsupport components in C++ verification. In