Browsix: Bridging the Gap Between Unix and the Browser
BB ROWSIX : Bridging the Gap Between Unix and the Browser
Bobby Powers, John Vilk, Emery D. BergerUniversity of Massachusetts Amherst
Abstract
Applications written to run on conventional operating sys-tems typically depend on OS abstractions like processes, pipes,signals, sockets, and a shared file system. Porting these ap-plications to the web currently requires extensive rewritingor hosting significant portions of code server-side becausebrowsers present a nontraditional runtime environment thatlacks OS functionality.This paper presents B ROWSIX , a framework that bridgesthe considerable gap between conventional operating systemsand the browser, enabling unmodified programs expecting aUnix-like environment to run directly in the browser. B ROWSIX comprises two core parts: (1) a JavaScript-only system thatmakes core Unix features (including pipes, concurrent pro-cesses, signals, sockets, and a shared file system) available toweb applications; and (2) extended JavaScript runtimes for C,C++, Go, and Node.js that support running programs writtenin these languages as processes in the browser. B ROWSIX supports running a POSIX shell, making it straightforward toconnect applications together via pipes.We illustrate B ROWSIX ’s capabilities via case studies thatdemonstrate how it eases porting legacy applications to thebrowser and enables new functionality. We demonstrate a B ROWSIX -enabled L A TEX editor that operates by executing un-modified versions of pdfLaTeX and BibTeX. This browser-onlyL A TEX editor can render documents in seconds, making it fastenough to be practical. We further demonstrate how B ROWSIX lets us port a client-server application to run entirely in thebrowser for disconnected operation. Creating these applica-tions required less than 50 lines of glue code and no codemodifications, demonstrating how easily B ROWSIX can beused to build sophisticated web applications from existingparts without modification.
1. Introduction
Web browsers make it straightforward to build user interfaces,but they can be difficult to use as a platform to build sophis-ticated applications. Code must generally be written fromscratch or heavily modified; compiling existing code or li-braries to JavaScript is not sufficient because these applica-tions depend on standard OS abstractions like processes and ashared file system, which browsers do not support. Many webapplications are thus divided between a browser front-end anda server backend. The server runs on a traditional operatingsystem, where the application can take advantage of familiarOS abstractions and run a wide variety of off-the-shelf librariesand programs. As a representative example, websites like ShareLaTeX and Overleaf let users write and edit L A TEX documents in thebrowser without the need to install a TEX distribution locally.This workflow lowers the barrier for students and first-timeL A TEX authors and enables real-time collaboration, eliminatingsome of the complexity of creating multi-author documents.These applications achieve this functionality by providing abrowser-based frontend for editing; user input is sent to theserver for persistence and collaboration purposes. When theuser requests a generated PDF, the website runs pdflatex and bibtex server-side on the user’s behalf, with the resultingPDF sent to the browser when complete.These web applications generate PDFs server-side out ofnecessity because browsers lack the operating system servicesand execution environment that Unix programs expect. Creat-ing PDFs from L A TEX requires spawning multiple processes torun pdflatex and bibtex , which need to read from and writeto a shared file system. If PDF generation takes too long andthe user cancels the request, the application needs to send a
SIGTERM or SIGKILL signal to clean up any running processes.If PDF generation encounters an error, the application needs topipe the output of the relevant process back to the client overthe network. Since browsers do not support processes, signals,pipes, sockets, or a shared filesystem, they cannot perform anyof these steps without program modification.Previous attempts to cope with this impedance mismatchbetween conventional applications and the browser fall shortof providing the environment needed by many programs (seeSection 7). Emscripten and Doppio provide a POSIX-like run-time system for single processes, including a single-processfile system, limited support for threads, synchronous I/O, andproxying support for TCP/IP sockets [9, 12]. While thesesingle-process runtimes are useful for some applications, theyare severely limited because they are unable to provide therange of operating system functionality that many legacy ap-plications demand.To overcome these limitations, we introduce B
ROWSIX ,a framework that brings Unix abstractions to the browserthrough a shared kernel and common system-call conventions,bridging the gap between conventional operating systems andthe browser. B
ROWSIX consists of two core components: (1) aJavaScript-only operating system that exposes a wide array ofOS services that applications expect (including pipes, concur-rent processes, signals, sockets, and a shared file system); and(2) extended JavaScript runtimes for C, C++, Go, and Node.js a r X i v : . [ c s . O S ] A p r igure 1: A L A TEX editor built using B
ROWSIX . B
ROWSIX ’s OSservices and language runtimes make it possible to run com-plex legacy code (including pdflatex and bibtex ) directlyin the browser, without code modifications (See Section 2 fordetails.) that let unmodified programs written in these languages andcompiled to JavaScript run directly in the browser. BecauseB
ROWSIX ’s components are written entirely in JavaScript andrequire no plugins, applications using B
ROWSIX can run in awide range of modern web browsers including Google Chrome,Mozilla Firefox, and Microsoft Edge. B
ROWSIX makes it pos-sible to port a wide range of existing applications and theirlanguage runtimes to the browser by providing the core func-tionality of a full operating system: • Processes: B ROWSIX implements a range of process relatedsystem calls (including fork , spawn , exec , and wait4 ) andprovides a process primitive on top of Web Workers, lettingapplications run in parallel and spawn subprocesses. • Signals: B ROWSIX supports a substantial subset of thePOSIX signals API, including kill and signal handlers,letting processes communicate with each other asyn-chronously. • Shared Filesystem: B ROWSIX lets processes share statethrough a shared FS. • Pipes: B ROWSIX exposes the standard pipe
API, making itsimple for developers to compose processes into pipelines. • Sockets: B ROWSIX supports TCP socket servers and clients,making it possible to run server applications like databasesand HTTP servers together with their clients in the browser. • Language Agnostic: B ROWSIX includes integration withthe runtime libraries of Emscripten (C/C++), GopherJS(Go), and Node.js (JavaScript) to allow unmodified applica-tions written in these languages to run directly as processesin the browser. Through its simple system call API, develop-ers can straightforwardly integrate B
ROWSIX into additionallanguage runtimes.B
ROWSIX dramatically simplifies the process of portingcomplex applications to the browser environment. As a demon-stration, we have built a L A TEX editor that runs entirely withinthe browser. When the user requests a PDF, B
ROWSIX runs make to re-build the document with pdflatex and bibtex , and pipes their standard output and standard error to the ap-plication. These TEX programs use B
ROWSIX ’s shared filesystem to read in the user’s source files, and any packages,class files, and fonts referenced within, as in a traditional Unixenvironment. The filesystem transparently loads any neededexternal packages from the TeX Live distribution over HTTPupon first access. Subsequent accesses to the same files are in-stantaneous, as the browser caches them. While a full TeX Livedistribution is several gigabytes in size, a typical paper onlyneeds to retrieve several megabytes worth of packages beforeit can be built. If the user cancels PDF generation, B
ROWSIX sends a
SIGKILL signal to these processes. If PDF generationfails, the application can display the captured standard outand standard error. The result is serverless PDF generationcomposed from off-the-shelf parts, with minimal engineeringeffort required to glue them together.We demonstrate the utility of B
ROWSIX with two furthercase studies. Using B
ROWSIX , we build an application that dy-namically routes requests to a remote server or an in-B
ROWSIX server, both compiled from the same source code, dependingon the client’s performance and battery characteristics. We alsouse B
ROWSIX to build a UNIX terminal exposing a POSIXshell, enabling developers to launch and compose applicationsand inspect B
ROWSIX state in a familiar way.
Contributions
This paper makes the following contributions: • Bringing OS Abstractions and Services to the Browser.
We demonstrate that it is possible to provide a wide rangeof key Unix abstractions and services in the browser on topof existing web APIs. We implement these in B
ROWSIX , aJavaScript-only framework featuring a kernel and systemcalls that runs on all modern browsers (§3). • Runtime Integration for Existing Languages.
We ex-tend the JavaScript runtimes of Emscripten (a C/C++ toJavaScript toolchain), GopherJS (a Go to JavaScript com-piler), and Node.js with B
ROWSIX support, letting unmodi-fied C, C++, Go, and Node.js programs execute and inter-operate with one another within the browser as B
ROWSIX processes (§4). • Case Studies.
We demonstrate B
ROWSIX ’s utility by build-ing a L A TEX editor, a serverless client-server web application,and a Unix terminal out of off-the-shelf components withoutmodification. We characterize B
ROWSIX ’s performance un-der these case studies and with microbenchmarks (§5) andshow that its overhead is low enough for real-world usage. • Guidance for Future Browsers.
Based on our experiencewriting B
ROWSIX , we discuss current browser limitationsand propose solutions (§6).
2. B
ROWSIX
Overview
To give an overview of B
ROWSIX ’s features, this section walksthrough the process of using B
ROWSIX to build an in-browser2 A TEX editor using TeX Live utilities and GNU Make. Figure 1displays a screenshot of the editor. A TEX Editor Overview
The editor presents a split-screen view to the user, with the doc-ument’s L A TEX source on the left, and generated PDF previewon the right. The editor’s UI is a standard web application,and represents the only new code. When the user clicks on the“Build PDF” button, the editor uses B
ROWSIX to invoke GNUMake in a B
ROWSIX process, which rebuilds the PDF.The process for building the PDF is familiar to anyone whohas used L A TEX, except B
ROWSIX performs all of the neededsteps entirely in the browser instead of server-side. It runsGNU Make to read a Makefile from B
ROWSIX ’s file system,which contains rules for rebuilding L A TEX projects. Make thenruns pdflatex and bibtex , depending on whether the userhas updated the references file. pdflatex and bibtex read any required L A TEX packages,fonts, and other system files from B
ROWSIX ’s file system,which lazily pulls in files as needed from the network. Both ofthese applications write their output to B
ROWSIX ’s file system.Once all steps have completed (or an error has occurred), theMake process exits with an exit code indicating whether or notPDF generation succeeded. B
ROWSIX sends the exit code backto the web application. If GNU Make exits normally, the editorreads the PDF from B
ROWSIX ’s shared filesystem and displaysit to the user. Otherwise, it displays the standard output from pdflatex and bibtex to the user, which describes the sourceof the error.
ROWSIX
Building any web application that runs Unix programs inB
ROWSIX generally consists of the same three step process:(1) compile the programs to JavaScript (using tools withB
ROWSIX support), (2) stage files required by the applica-tion for placement in the in-browser filesystem, and (3) addsetup code to the web application to initiate B
ROWSIX andlaunch the programs.
Compiling to JavaScript:
To run pdflatex , bibtex , andGNU Make in B ROWSIX , the developer compiles each pro-gram to JavaScript using Emscripten, a C/C++ to JavaScriptcompiler [12]. We extend Emscripten’s runtime library withB
ROWSIX support, so standard Unix APIs map to B
ROWSIX primitives. We discuss this extension in more detail in Sec-tion 4.3.Before compilation, the developer needs to determine if anyof the programs use the fork command. Due to browser limi-tations explored in Section 6, B
ROWSIX can only implement fork as an asynchronous system call, which requires the useof a variant of Emscripten’s compilation procedure that gener-ates less efficient code. If this option is configured incorrectly,the program will fail at runtime when it attempts to invoke fork . For the L A TEX example, only GNU Make uses fork and requires this setting.From this point, the build process is mostly unchanged froma standard compilation. For programs that use the autotoolsbuild system, such as GNU Make and TeX Live, instead ofrunning ./configure , the developer invokes emconfigure./configure . This wrapper overrides standard tools like cc and ar with em prefixed alternatives that compile the programwith Emscripten, which produces individual JavaScript filesfor each program. Staging the Filesystem:
Next, the developer configuresB
ROWSIX ’s in-browser filesystem so that it hosts all of thefiles that the programs require. B
ROWSIX ’s file system ex-tends Doppio’s BrowserFS file system with multi-process sup-port, building on its support for files backed by cloud storage,browser-local storage, traditional HTTP servers, and more [9].For our L A TEX example, both pdflatex and bibtex requireread access to class, font, and other files from a L A TEX distribu-tion to function properly. While a complete TeX Live distribu-tion contains over 60,000 individual files, the average L A TEXdocument only references a small subset of these files.To reduce load times and minimize the amount of stor-age required on a client’s device, the developer can leverageB
ROWSIX ’s filesystem to load only the needed files. In thiscase, the developer uploads a full TeX Live distribution to anHTTP server and configures B
ROWSIX ’s filesystem to use anHTTP-backed filesystem backend. The filesystem will thenload these files on-demand from the network upon first ac-cess. The browser caches these files automatically, makingsubsequent access much faster. B ROWSIX
Setup Code:
Finally, the developer adds code tothe web application to load and initialize B
ROWSIX , and tolaunch make to build the PDF. A script tag in the HTMLloads browsix.js , and a subsequent script tag with inlineJavaScript calls B
ROWSIX ’s Boot function with the desiredfilesystem configuration.Additional application-specific initialization follows asusual. Once the filesystem is ready, the developer addscode to read the contents of main.tex and main.bib fromB
ROWSIX ’s filesystem, and display the contents in the editor;The application then registers a callback function with the“Build PDF” button to be run whenever the user clicks thebutton.
ROWSIX
When the application’s callback is executed in response to auser’s “Build PDF” click, the application invokes the system method on its kernel instance to start make . Make runs pdflatex and bibtex as described in Section 1. When theapplication receives a notification from B
ROWSIX that Makehas exited, it inspects Make’s exit code. If it is zero, the PDFwas generated successfully and is read from the filesystem.Otherwise, the captured standard output and standard error aredisplayed to the user so they can debug their markup.3 omponent Lines of Code (LoC)
Kernel 2,249BrowserFS modifications 1,231Shared syscall module 421Emscripten integration* 1,557 (C/C++ support)
GopherJS integration* 926 (Go support)
Node.js integration* 1,742
TOTAL
Figure 2: B
ROWSIX components. * indicates these componentsare written in JavaScript, while the rest of the components arewritten in TypeScript (which compiles to JavaScript).
This overview demonstrates how straightforward B
ROWSIX makes it to run existing components – designed to work in aUnix environment – and execute them seamlessly inside a webbrowser. The next two sections provide technical details onhow B
ROWSIX provides Unix-like abstractions in the browserenvironment and integrates with language runtimes.
3. B
ROWSIX
OS Support
The core of B
ROWSIX ’s OS support is a kernel that controlsaccess to shared Unix services. Unix services, including theshared file system, pipes, sockets, and task structures, live in-side the kernel, which runs in the main browser thread. Pro-cesses run separately and in parallel inside Web Workers, andaccess B
ROWSIX kernel services through a system call inter-face. B
ROWSIX and all of its runtime services are implementedin JavaScript and TypeScript, a typed variant of JavaScript thatcompiles to pure JavaScript. Figure 2 provides a breakdownof each of B
ROWSIX ’s components.
The kernel lives in the main JavaScript context alongside theweb application, and acts as the intermediary between pro-cesses and loosely coupled Unix subsystems. Processes issuesystem calls to the kernel to access shared resources, andthe kernel relays these requests to the appropriate subsystem.When the subsystem responds to the system call, it relays theresponse to the process. The kernel is also responsible fordispatching signals to processes, which we describe further inSection 3.3. Figure 3 presents a partial list of the system callsthat the kernel currently supports.In a departure from modern Unix systems, B
ROWSIX doesnot support multiple users. A traditional kernel would, forexample, use user identities to check permissions on certainsystem calls or for access to files. Instead, B
ROWSIX lever-ages and relies on the browser’s built-in sandbox and securityfeatures, such as the same origin policy. In other words, aB
ROWSIX application enjoys the same level of protection andsecurity as any other web application.
Class System calls
Process Management fork , spawn , pipe2 , wait4 , exit Process Metadata chdir , getcwd , getpid Sockets socket , bind , getsockname , listen , accept , connect Directory IO readdir , getdents , rmdir , mkdir File IO open , close , unlink , llseek , pread , pwrite File Metadata access , fstat , lstat , stat , readlink , utimes Figure 3: A representative list of the system calls implementedby the B
ROWSIX kernel. fork is only supported for C and C++programs.
The B
ROWSIX kernel supports two types of system calls: asyn-chronous and synchronous. Asynchronous system calls workin all modern browsers, but impose a high performance penaltyon C and C++ programs. Synchronous system calls enable Cand C++ programs to perform much better, but currently onlywork in Google Chrome via a mechanism we describe below;this mechanism is already standardized and is on track to besupported by other mainstream browsers.
Asynchronous System Calls: B ROWSIX implements asyn-chronous system calls in a continuation-passing style (CPS).A process initiates a system call by sending a message to thekernel with a process-specific unique ID, the system call num-ber, and arguments. B
ROWSIX copies all arguments, such asfile descriptor or a buffer to write to a file descriptor, fromthe process to the kernel - no memory is shared. When thekernel sends a response, the Web Worker process executes thecontinuation (or callback) with response values, also copiedfrom the kernel’s heap into the process’s heap.Asynchronous system calls work well for Node.js and Go,but are a poor match for many C and C++ programs. In Node.js,filesystem and other APIs accept a callback parameter to in-voke when a response is ready, which matches B
ROWSIX ’sasynchronous system call mechanism. The GopherJS runtimeprovides support for suspending and resuming the call stack inorder to implement goroutines (lightweight thread-like prim-itives); this support also meshes with B
ROWSIX ’s approach.However, when using Emscripten to compile C and C++ pro-grams, they must be compiled in an interpreted mode (calledthe Emterpreter) in order to use asynchronous system calls.This mode produces much less efficient code than the standardcompiler, which produces asm.js output by default.
Synchronous System Calls:
Synchronous system callswork by sharing a view of a process’s address space betweenthe kernel and the process, similar to a traditional operating4ystem kernel like Linux. At startup, the language runtime ina process wishing to use synchronous system calls passes tothe kernel (via an asynchronous system call) a reference to theheap (a SharedArrayBuffer object), along with two offsets intothe heap: where to put system call return values, and an offsetto use to wake the process when the syscall is complete.A process invokes a synchronous system call by sendinga message, as in the asynchronous case, but arguments arejust integers and integer offsets (representing pointers) intothe shared memory array, rather than larger objects (like Ar-rayBuffers) that would need to be copied between heaps. Forsystem calls like pread , data is copied directly from the filesys-tem, pipe or socket into the process’s heap, avoiding a poten-tially large allocation and extra copy.After sending a message to the kernel, the process per-forms a blocking wait on the address previously arrangedwith the kernel and is awakened when the system call hascompleted or a signal is received. This wait is provided by theJavaScript
Atomics.wait function, part of the ECMAScriptShared Memory and Atomics specification [4]. A side effect ofthis approach is that fork is not compatible with synchronoussystem calls, as there is no way to re-wind or jump to a partic-ular call stack in the child Web Worker.Synchronous system calls are faster in practice for a numberof reasons. First, they only require one message to be passedbetween the kernel and process, which is a relatively slowoperation. Second, system call arguments are numbers, ratherthan potentially large arrays that need to be copied betweenJavaScript contexts. Finally, synchronous system calls providea blocking primitive and do not depend on language runtimesto unwind and rewind the call stack. As such, they are suitablefor use with asm.js and WebAssembly functions on the callstack, which are faster and more amenable to optimization bythe JavaScript runtime than Emscripten’s interpreter.Synchronous system calls currently require the in-development browser features SharedArrayBuffers and Atom-ics, and currently only work in Google Chrome when launchedwith extra flags. SharedArrayBuffers and Atomics are on thestandards track, and are expected to be supported un-flaggedin mainstream browsers in the near future. B ROWSIX relies on
Web Workers as the foundation for emulat-ing Unix processes. However, Web Workers differ substantiallyfrom Unix processes, and B
ROWSIX must provide a significantamount of functionality to bridge this gap.In Unix, processes execute in isolated virtual address spaces,run in parallel with one another when the system has multipleCPU cores, and can interact with system resources and otherprocesses via system calls. However, the web browser doesnot expose a process API to web applications. Instead, webapplications can spawn a Web Worker that runs a JavaScriptfile in parallel with the application.A Web Worker has access to only a subset of browser inter- faces (notably excluding the Document Object Model (DOM)),runs in a separate execution context, and can only communi-cate with the main browser context via asynchronous messagepassing. Web Workers are not aware of one another, cannotshare memory with one another, and can only exchange mes-sages with the main browser context that created them (seeSection 6 for a discussion). Major browsers like Chrome andSafari do not support spawning sub-workers from workers,so-called nested workers , and have not added support for themsince they were first proposed in 2009. Thus, if a Web Workerneeds to perform a task in parallel, it must delegate the requestto the main browser thread, and proxy all messages to thatworker through the main browser thread. Perhaps unsurpris-ingly, the limitations and complexity of Web Workers havehindered their adoption in web applications.By contrast, B
ROWSIX implements Unix processes on top ofWeb Workers, giving developers a familiar and full-featured ab-straction for parallel processing in the browser. Each B
ROWSIX process has an associated task structure that lives in the kernelthat contains its process ID, parent’s process ID, Web Workerobject, current working directory, and map of open file descrip-tors. Processes have access to the system calls in Figure 3,and invoke them by sending a message with the system callname and arguments to the kernel. As a result, processes canshare state via the file system, send signals to one another,spawn sub-processes to perform tasks in parallel, and con-nect processes together using pipes. Below, we describe howB
ROWSIX maps familiar OS interfaces onto Web Workers. spawn : B ROWSIX supports spawn , which constructs a newprocess from a specified executable on the file system. spawn is the primary process creation primitive exposed in modernprogramming environments such as Go and Node.js, as fork is unsuitable for general use in multithreaded processes. spawn lets a process specify an executable to run, the arguments topass to that executable, the new process’s working directory,and the resources that the subprocess should inherit (such asfile descriptors). In B
ROWSIX , executables include JavaScriptfiles, file beginning with a shebang line, and WebAssemblyfiles. When a process invokes spawn , B
ROWSIX creates anew task structure with the specified resources and workingdirectory, and creates a new Web Worker that runs the targetexecutable or interpreter.There are two technical challenges to implementing spawn .First, the Web Worker constructor takes a URL to a JavaScriptfile as its first argument. Files in B
ROWSIX ’s file systemmay not correspond to files on a web server. For example,they might be dynamically produced by other B
ROWSIX pro-cesses. To work around this restriction, B
ROWSIX generates aJavaScript
Blob object that contains the data in the file, obtainsa dynamically-created
URL for the blob from the browser’swindow object, and passes that URL as a parameter to the WebWorker constructor. All modern web browsers now supportconstructing Workers from blob URLs.The second challenge is that there is no way to pass data to a5orker on startup apart from sending a message. As processessynchronously access state like the arguments vector and en-vironment map, B
ROWSIX -enabled runtimes delay executionof a process’s main() function until after the worker has re-ceived an “init” message containing the process’s argumentsand environment. fork : The fork system call creates a new process contain-ing a copy of the current address space and call stack. Forkreturns twice – first with a value of zero in the new process, andwith the PID of the new process in the original. Web Workersdo not expose a cloning API, and JavaScript lacks the reflec-tion primitives required to serialize a context’s entire state intoa snapshot. Thus, B
ROWSIX only supports fork when a lan-guage runtime is able to completely enumerate and serializeits own state. Section 4 describes how we extend Emscriptento provide fork support for C/C++ programs compiled toJavaScript. wait4 : The wait4 system call reaps child processes thathave finished executing. It returns immediately if the specifiedchild has already exited, or the
WNOHANG option is specified.Waiting requires that the kernel not immediately free task struc-tures, and required us to implement the zombie task state forchildren that have not yet been waited upon. The C library usedby Emscripten, musl, uses the wait4 system call to implementthe C library functions wait , wait3 , and waitpid . exit : Language runtimes with B
ROWSIX -support are re-quired to explicitly issue an exit system call when they aredone executing, as the containing Web Worker context hasno way to know that the process has finished. This is due tothe event-based nature of JavaScript environments – even ifthere are no pending events in the Worker’s queue, the mainJavaScript context could, from the perspective of the browser,send the Worker a message at any time. getpid , getppid , getcwd , chdir : These four system callsoperate on the data in current process’s task structure, whichlives in the B
ROWSIX kernel. getpid returns the process’s ID, getppid returns the parent process’s ID, getcwd returns theprocess’s working directory, and chdir changes the process’sworking directory. B ROWSIX pipes are implemented as in-memory buffers withread-side wait queues. If there is no data to be read when aprocess issues a read system call, B
ROWSIX enqueues thecallback encapsulating the system call response which it in-vokes when data is written to the pipe. Similarly, if there isnot enough space in a pipe’s internal buffer, B
ROWSIX onlyinvokes the callback encapsulating the system call response tothe write operation when the pipe is read from. B ROWSIX implements a subset of the BSD/POSIX socketAPI, with support for
SOCK_STREAM (TCP) sockets for com-municating between B
ROWSIX processes. These sockets en-able servers that bind , listen and then accept new connec-tions on a socket, along with clients that connect to a socketserver, with both client and server reading and writing fromthe connected file descriptor. Sockets are sequenced, reliable,bi-directional streams. B ROWSIX builds on and significantly extends BrowserFS’s filesystem, part of Doppio [9]. BrowserFS already included sup-port for multiple mounted filesystems in a single hierarchicaldirectory structure. BrowserFS provides multiple file systembackend implementations, such as in-memory, zip file, XML-HttpRequest, Dropbox, and an overlay filesystem. BrowserFSprovides a unified, encapsulated interface to all of these back-ends.B
ROWSIX extends BrowserFS in two key ways: it addsmulti-process support and incorporates improved support forloading files over HTTP. To provide multi-process support,B
ROWSIX ’s file system adds locking operations to the over-lay filesystem to prevent operations from different processesfrom interleaving. In addition, B
ROWSIX incorporates domain-specific optimizations into its file system; for example, itavoids expensive operations like recording the call stack whena path lookup fails (a common event).B
ROWSIX modifies BrowserFS’s overlay backend to lazilyload files from its read-only underlay; the original versioneagerly read all files from the read-only filesystem upon initial-ization. B
ROWSIX ’s approach drastically improves the startuptime of the kernel, minimizes the amount of data transferredover the network, and enables applications like the L A TEX edi-tor where only a small subset of files are required for a givenend user.Finally, B
ROWSIX implements system calls that operate onpaths, like open and stat , as method calls to the kernel’sBrowserFS instance. When a system call takes a file descriptoras an argument, the kernel looks up the descriptor in the tasks’sfile hashmap and invokes the appropriate method on that fileobject, calling into BrowserFS for regular files and directories.Child processes inherit file descriptor tables, and B
ROWSIX manages each object (whether it is a file, directory, pipe orsocket) with reference counting.
4. B
ROWSIX
Runtime Support
Applications access B
ROWSIX system calls indirectly throughtheir runtime systems. This section describes the runtime sup-port we added to GopherJS, Emscripten, and Node.js alongwith the APIs exposed to web applications so they can executeprograms in B
ROWSIX .6 ernel.system(’pdflatex example.tex’, function (pid, code) { if (code === 0) {displayPDF();} else {displayLatexLog();}}, logStdout, logStderr); Figure 4: Creating a B
ROWSIX process from JavaScript.
Web applications run alongside the B
ROWSIX kernel in themain browser context, and have access to B
ROWSIX featuresthrough several global APIs. B
ROWSIX exposes new APIsfor process creation, file access, and socket notifications, andan XMLHttpRequest-like interface to send HTTP requests toB
ROWSIX processes.File access acts as expected, and allows the client to ma-nipulate the filesystem, invoke a utility or pipeline of utilities,and read state from the filesystem after programs have fin-ished executing. Figure 4 shows how client applications invokeB
ROWSIX processes and react when processes exit through anAPI similar to C’s system .Socket notifications let applications register a callback tobe invoked when a process has started listening on a particularport. These notifications let web applications launch a serveras a process and appropriately delay communicating with theserver until it is listening for messages. Web applications donot need to resort to polling or ad hoc waiting.B
ROWSIX provides an XMLHttpRequest-like API for send-ing requests from the web application to in-browser HTTPservers running in B
ROWSIX . This allows JavaScript to inter-act with HTTP 1.1 servers running as B
ROWSIX processes asif they were remote HTTP servers. The API encapsulates thedetails of connecting a B
ROWSIX socket to the server, serializ-ing the HTTP request to a byte array, sending the byte array tothe B
ROWSIX process, processing the (potentially chunked)HTTP response, and generating the expected web events. B ROWSIX provides a small syscall layer as a JavaScriptmodule that runs in a Web Worker. This layer provides aconcrete, typed API for asynchronous system calls over thebrowser’s message passing primitives. Language runtimes usethis module to communicate with the shared kernel. Methodsprovided by the syscall layer take the same arguments asLinux system calls of the same name, along with an additionalargument: a callback function. This callback is executed whenthe syscall module receives a message response from the ker-nel. Unlike a traditional single-threaded process, a B
ROWSIX process can have multiple outstanding system calls, which en-ables runtimes like GopherJS to implement user-space threadson top of a single Web Worker execution context.Signals are sent over the same message passing interface as function sys_getdents64(cb, trap, fd, dirp, len) { var done = function (err, buf) { if (!err)dirp.set(buf);cb([err ? -1 : buf.byteLength, 0, err ? err : 0]);};syscall_1.syscall.getdents(fd, len, done);} Figure 5: Implementing the getdents64 system call in Go-pherJS. system calls. The common syscall module provides a wayto register signal handlers for the standard Unix signals, suchas
SIGCHLD . For many programming languages, existing language run-times targeted for the browser must bridge the impedancemismatch between synchronous APIs present on Unix-likesystems and the asynchronous world of the browser. Compile-to-JavaScript systems like Emscripten, ClojureScript [6],Scala.js [2], js_of_ocaml [10], WhaleSong (Racket) [11], andGopherJS all employ different approaches. Since B
ROWSIX supports both synchronous and asynchronous system calls,language runtimes can choose the system call convention mostappropriate for their implementation.This section describes the runtime support we added tolanguage runtimes for Go, C/C++, and Node.js. ExtendingB
ROWSIX support to additional language runtimes remains asfuture work.
Go:
Go is a systems language developed at Google designedfor readability, concurrency, and efficiency. To run Go pro-grams under B
ROWSIX , we extended the existing GopherJScompiler and runtime to support issuing and waiting for sys-tem calls under B
ROWSIX . GopherJS already provides fullsupport for Go language features like goroutines (lightweightthreads), channels (communication primitives), and delayedfunctions.We extended the GopherJS runtime with support forB
ROWSIX through modifications to the runtime. The mainintegration points are a B
ROWSIX -specific implementation ofthe syscall.RawSyscall function (which handles syscallsin Go), along with overrides of several Go runtime functions.The replacement for
RawSyscall is implemented in Go. Itallocates a Go channel object, and this function invokes theB
ROWSIX
JavaScript syscall library, passing the system callnumber, arguments, and a callback to invoke.
RawSyscall then performs a blocking read on the Go channel, which sus-pends the current goroutine until the callback is invoked. Whenthe system call response is received from the B
ROWSIX ker-nel, GopherJS’s existing runtime takes care of re-winding thestack and continuing execution. The syscall library invokesa function specific to each supported system call to marshaldata to and from the B
ROWSIX kernel. Adding support for anynew system call is a matter of writing a small handler function7 _syscall220: function (which, varargs) { if EMTERPRETIFY_ASYNC return
EmterpreterAsync.handle( function (resume) { var fd = SYSCALLS.get(), dirp = SYSCALLS.get(), count =SYSCALLS.get(); var done = function (err, buf) { if (err > 0)HEAPU8.subarray(dirp, dirp+buf.byteLength).set(buf);resume( function () { return err;});};SYSCALLS.browsix.syscall.async(done, ’getdents’, [fd,count]);}); else var fd = SYSCALLS.get(), dirp = SYSCALLS.get(), count =SYSCALLS.get(); return SYSCALLS.browsix.syscall.sync(220, fd, dirp, count);
Figure 6: Implementing the B
ROWSIX getdents64 syscall inEmscripten. and registering it; an example is shown in Figure 5B
ROWSIX replaces a number of low-level runtime functions;the most important are syscall.forkAndExecInChild and net.Listen . The former is overridden to directly invokeB
ROWSIX ’s spawn system call, and the latter to provide accessto B ROWSIX socket services. Additional integration points in-clude an explicit call to the exit system call when the mainfunction exits, and waiting until the process’s arguments andenvironment have been received before starting main() (see§3.3).
C and C++:
We also extend Emscripten, Mozilla Research’sLLVM-based C and C++ compiler that targets JavaScript, withsupport for B
ROWSIX . B
ROWSIX -enhanced Emscripten sup-ports two modes - synchronous system calls and asynchronoussystem calls (described in Section 3.2), one of which is se-lected at compile time. When asynchronous system calls areused, it requires use of Emscripten’s interpreter mode (namedthe “Emterpreter”) to save and restore the C stack. B
ROWSIX ’sasynchronous system calls require all functions that may be onthe stack under a system call to be interpreted so that the stackcan be restored when the system call completes. Emscriptencan selectively compile other parts of an application to asm.js ,which will be JIT-compiled and run as native JavaScript by thebrowser. Synchronous system calls do not have this limitation.As with GopherJS, Emscripten provides a clear integrationpoint at the level of system calls. Emscripten provides im-plementations for a number of system calls, but is restrictedto performing in-memory operations that do not block. Wereplace Emscripten system call implementations with onesthat call into the B
ROWSIX kernel, such as in Figure 6. In thecase of getdents and stat , padding was added to C struc-ture definitions to match the layout expected by the B
ROWSIX kernel.When a C process invokes fork , the runtime sends a copy ofthe global memory array, which includes the C stack and heap,along with the current program counter (PC) to the kernel. After the kernel launches a new Web Worker, it transfers thiscopy of global memory and PC to the new Worker as part ofthe initialization message. When the Emscripten runtime inthe new B
ROWSIX process receives the initialization message,if a memory array and PC are present the runtime swaps themin and invokes the Emterpreter to continue from where fork was invoked.
Node.js:
Node.js (a.k.a. “Node”) is a platform for buildingservers and command line tools with JavaScript, implementedin C, C++ and JavaScript, on top of the v8 JavaScript en-gine. Node.js APIs are JavaScript modules that can be loadedinto the current browser context by invoking the require built-in function. These high-level APIs are implemented inplatform-agnostic JavaScript, and call into lower-level C++bindings, which in turn invoke operating system interfaces likefilesystem IO, TCP sockets, and child process management.Node.js embraces the asynchronous, callback-oriented natureof JavaScript – most Node APIs that invoke system calls takea callback parameter that is invoked when results are ready.To run servers and utilities written for Node.js underB
ROWSIX , we provide a browser-node executable that pack-ages Node’s high-level APIs with pure-JavaScript replace-ments for Node’s C++ bindings that invoke B
ROWSIX systemcalls as a single file that runs in a B
ROWSIX process. B
ROWSIX also replaces several other native modules, like the modulefor parsing and generating HTTP responses and requests, withpure JavaScript implementations. Node executables can be in-voked directly, such as node server.js , or will be invokedindirectly by the kernel if node is specified as the interpreterin the shebang line of a text file marked as executable.
5. Evaluation
Our evaluation answers the following questions: (1) Doesbringing Unix abstractions into the browser enable compellinguse cases? (2) Is the performance impact of running programsunder B
ROWSIX acceptable?
We evaluate the applicability and advantages of bringing Unixabstractions into the browser with two case studies, in additionto the L A TEX editor from the overview (§2). First, we build aweb application for creating memes that can run its unmodi-fied server in B
ROWSIX . The meme generator transparentlyswitches between generating memes in-browser or server-sidedepending on network and device characteristics. Second, webuild a Unix terminal that lets application developers use dash ,a widely-used POSIX shell, to interact with B
ROWSIX in afamiliar manner.
Our meme generator lets users cre-ate memes consisting of images with (nominally) humorousoverlaid text. Figure 7 contains a screenshot. Existing ser-vices, such as
MemeGenerator.net , perform meme gener-ation server-side. Moving meme creation into the browser8 igure 7: A meme generator built using B
ROWSIX . All server-side functionality was moved into the browser without modify-ing any code. would reduce server load and reduce latency when the net-work is overloaded or unreliable, but doing so would normallypresent a significant engineering challenge. The meme gen-eration server uses sockets to communicate with the browserover HTTP and reads meme templates from the file system.Before B
ROWSIX , the client and server code would need to bere-architected and rewritten to run together in the browser.To demonstrate B
ROWSIX ’s ability to quickly port codefrom the server to the web, we implement our meme creator asa traditional client/server web application; Figure 8a containsa system diagram. The client is implemented in HTML5 andJavaScript, and the server is written in Go. The server readsbase images and font files from the filesystem, and uses off-the-shelf third-party Go libraries for image manipulation andfont rendering to produce memes [3]. The server also usesGo’s built-in http module to run its web server. Note thatthis server is stateless, following best practices [7]; porting astateful server would naturally require more care.To port the server to B
ROWSIX , we follow the process out-lined in Section 2. First, we compile the Go server to a singleJavaScript file using GopherJS, a Go to JavaScript compiler [8].Then, we stage the font and images for the BrowserFS filesys-tem. Finally, we augment the client application to load theB
ROWSIX
JavaScript module, initialize a kernel instance, andstart the meme-server.Next, we augment the web application to dynamically routememe generation requests to a server running in B
ROWSIX or to the cloud. We add a function to the application thatimplements a simple policy: if the network is inaccessible,or the browser is running on a desktop (which is a proxy fora powerful device), the application routes meme generationrequests to the server running in B
ROWSIX . Otherwise, itsends the requests to the remote server. In both cases, the webapplication uses an
XMLHttpRequest -like interface to makethe request, requiring little change to the existing code.Figure 8b displays a system diagram of the modified memegenerator. With this modification, meme generation workseven when offline. The code required to implement this pol-icy and dynamic behavior amounted to less than 30 lines ofJavaScript.
ROWSIX
Terminal:
To make it easy for devel-opers to interact with and test programs in B
ROWSIX , weimplement an in-browser Unix terminal that exposes a POSIXshell. The terminal uses the Debian Almquist shell (dash),the default shell of Debian and Ubuntu. We compile dash toJavaScript using B
ROWSIX -enhanced Emscripten, and run itin a B
ROWSIX process.Since the B
ROWSIX terminal uses a standard Linuxshell, developers can use it to run arbitrary shell com-mands in B
ROWSIX . Developers can pipe programs together(e.g. cat file.txt | grep apple > apples.txt ), exe-cute programs in a subshell in the background with & , runshell scripts, and change environment variables. Developerscan also run Go, C/C++, and Node.js programs.The terminal includes a variety of Unix utilities on theshell’s PATH that we wrote for Node.js: cat , cp , curl , echo , exec , grep , head , ls , mkdir , rm , rmdir , sh , sha1sum , sort , stat , tail , tee , touch , wc , and xargs . These programs runequivalently under Node and B ROWSIX without any modifi-cations, and were used heavily during development to debugB
ROWSIX functionality.
Summary: B ROWSIX makes it trivial to execute applica-tions designed to run in a Unix environment within the browser,enabling the rapid development of sophisticated web applica-tions. These applications can incorporate server code into thebrowser and harness the functionality of existing applications.
We evaluate the performance overhead of B
ROWSIX on ourcase studies. All experiments were performed on a ThinkpadX1 Carbon with an Intel i7-5600U CPU and 8 GB of RAM,running Linux 4.7.0. L A TEX Editor:
Running pdflatex under B
ROWSIX im-poses an order of magnitude slowdown. A native build of pdflatex under Linux takes around 100 milliseconds on asingle page document with a bibliography. When using syn-chronous calls (as supported by Google Chrome), the samedocument builds in B
ROWSIX in just under 3 seconds. Whilein relative terms this is a significant slowdown, this time isfast enough to be acceptable. Using asynchronous systemcalls and the Emterpreter, which is only necessary to enablebroader compatibility with today’s browsers, increases runtimeto around 12 seconds.
Meme Generator:
The meme generator performs two typesof HTTP requests to the server: requests for a list of availablebackground images, and requests to generate a meme. Webenchmark the performance of the meme generator server run-ning natively and running in Browsix in both Google Chromeand Firefox.On average, a request for a list of background images takes1.7 milliseconds natively, 9 ms in Google Chrome, and 6 msin Firefox. While requests to a server running natively onthe same machine as the client are faster than those served9 a) Meme creator running without B
ROWSIX (b) Meme creator running with B
ROWSIX
Figure 8: System diagram of the meme generator application with and without B
ROWSIX , demonstrating how the client and serverinteract with one another. With B
ROWSIX , the server runs in the browser without code modifications. by B
ROWSIX , B
ROWSIX is faster once a network connectionand roundtrip latencies are factored in. When comparing aninstance of the meme-server running on an EC2 instance, thein-B
ROWSIX request completed three times as fast as the re-quest to the remote machine. Times reported are the mean of100 runs following a 20-run warmup.The performance of meme generation is degraded by limi-tations in current browsers. The in-B
ROWSIX
HTTP requesttakes approximately two seconds to generate a meme in thebrowser, versus 200 ms when running server-side. This inef-ficiency is primarily due to missing 64-bit integer primitiveswhen numerical code is compiled to JavaScript with GopherJS,rather than overhead introduced by B
ROWSIX ; we expect thisto improve with both when future browsers support nativeaccess to 64-bit integers, and with independent improvementsto the GopherJS compiler. B ROWSIX
Terminal and Utilities:
Unix utilities providea mechanism to compare the performance of real-world pro-grams under Linux and B
ROWSIX . Figure 9 shows the re-sults of running the same JavaScript utility under B
ROWSIX and on Linux under Node.js, and compares this to the execu-tion time of the corresponding GNU Coreutils utility (writtenin C, running on Linux). Most of the overhead can be at-tributed to JavaScript (the basis of Node.js and B
ROWSIX );subsequently running in the B
ROWSIX environment imposesroughly a 3 × overhead over Node.js on Linux. Nonetheless,this performance (completion in under 200 milliseconds) islow enough that it should be generally acceptable to users. Command Native Node.js B
ROWSIX sha1sum ls Figure 9: Execution time of utilities under B
ROWSIX , com-pared to the same utilities run under Node.js, and the na-tive GNU/Linux utilities. sha1sum is run on usr/bin/node ,and ls is run on /usr/bin . Running in JavaScript (withNode.js and B ROWSIX ) imposes most overhead; running in theB
ROWSIX environment adds roughly another × overhead. Summary:
While B
ROWSIX ’s performance is limited by theperformance of underlying browser primitives (notably, thelack of native 64-bit longs), it provides acceptable performancefor a range of applications.
6. Discussion
The process of implementing B
ROWSIX has highlighted op-portunities for improvement in the implementation and speci-fication of browser APIs, especially Web Workers. We outlinea number of optimizations and natural extensions that are gen-erally useful, and would extend B
ROWSIX ’s reach.
Worker Priority Control:
The parent of a Web Workerhas no way to lower the priority of a created worker. Asworkers are implemented on top of OS threads, this con-cept maps cleanly onto OS-level priorities/niceness. Providingthis facility would let web applications prevent a low-priorityCPU-intensive worker from interfering with the main browserthread.10 il e s y s t e m S o c k e t c li e n t s S o c k e t s er v er s P r o ce ss e s P i p e s S i g n a l s E NVIRONMENTS B ROWSIX (cid:51) (cid:51) (cid:51) (cid:51) (cid:51) (cid:51) D OPPIO [9] † †WebAssemblyL
ANGUAGE R UNTIMES
Emscripten (C/C++) † † †GopherJS (Go)B
ROWSIX + Emscripten (cid:51) (cid:51) (cid:51) (cid:51) (cid:51) (cid:51) B ROWSIX + GopherJS (cid:51) (cid:51) (cid:51) (cid:51) (cid:51) (cid:51)
Table 1: Feature comparison of JavaScript execution environments and language runtimes for programs compiled to JavaScript. † indicates that the feature is only accessible by a single running process. B ROWSIX provides multi-process support for all of itsfeatures. postMessage()
Backpressure:
Traditional operating sys-tems attempt to prevent individual processes from affectingsystem stability in a number of ways. One of these is providingbackpressure, wherein the process attempting to write to a pipeor socket is suspended (the system call blocks) until the otherend of the pipe reads the data, or it can fit into a fixed sizebuffer. This approach prevents unbounded resource allocationin the kernel. In the browser, the postMessage() functioncan be called from a JavaScript context an unbounded numberof times and can eventually cause the browser to run out ofallocatable memory.
Message Passing Performance:
Message passing is threeorders of magnitude slower than traditional system calls in thetwo browsers we evaluate, Chrome and Firefox. A more effi-cient message passing implementation would improve the per-formance of B
ROWSIX ’s system calls and other inter-processcommunication.
Memory Mapping: B ROWSIX is currently unable to sup-port C/C++ applications like PostgreSQL that use mmap . Em-scripten uses a single typed array to represent the unmanagedC/C++ heap. While recent browser interfaces make it possi-ble to share this typed array among B
ROWSIX processes [4],browsers cannot yet map regions of the typed array into an-other typed array, which would be necessary to fully emulate mmap . Features on the WebAssembly roadmap, which aim forimplementation across browsers, would enable B
ROWSIX tosupport additional features like shared mmap and shm . Stack Management:
C provides the ability to save andchange the current thread’s context with the setcontext and getcontext functions. While rarely useful or advisable forapplications, it enables specialized low-level libraries to saveand restore the C stack. A similar JavaScript primitive coupledwith the use of SharedArrayBuffers would let B
ROWSIX sup-port fork in Emscripten applications as a synchronous systemcall. https://github.com/WebAssembly/design/blob/master/FutureFeatures.md
7. Related Work
In-Browser Execution Environments: B ROWSIX signif-icantly extends past efforts to bring traditional APIs andgeneral-purpose languages to the browser; Table 1 providesa comparison. Doppio’s focus is providing single-processPOSIX abstractions [9]. B
ROWSIX builds on and extends itsfilesystem component, BrowserFS, to support multiple pro-cesses. Emscripten compiles LLVM bytecode to JavaScript,enabling the compilation of C and C++ to JavaScript [12];as Section 4 describes, B
ROWSIX augments its runtime sys-tem so that unmodified C and C++ programs compiled withEmscripten can take full advantage of its facilities. B
ROWSIX provides similar runtime support for Go programs throughGopherJS [8].
Kernel Design and OS Interfaces: B ROWSIX resembles aLinux kernel task running on a microkernel [5], as it relieson an underlying system for messaging, scheduling and con-text switching. Barrelfish, a many-core, heterogenous OS [1]showed that asynchronous, shared-nothing system calls couldbe practical. Additionally, B
ROWSIX somewhat mirrors theper-core, shared nothing structure of a multikernel, as individ-ual B
ROWSIX processes do not use inter-domain communica-tion for tasks like memory allocation and timers.
8. Conclusion
This paper introduces B
ROWSIX , a framework that bringsthe essence of Unix to the browser. B
ROWSIX makes it al-most trivial to build complex web applications from com-ponents written in a variety of languages without modify-ing any code, and promises to significantly reduce the ef-fort required to build highly sophisticated web applications.B
ROWSIX is open source, and is freely available at https://github.com/plasma-umass/Browsix .11 eferences [1] A. Baumann, P. Barham, P.-E. Dagand, T. Harris, R. Isaacs, S. Peter,T. Roscoe, A. Schüpbach, and A. Singhania. The multikernel: a newOS architecture for scalable multicore systems. In
Proceedings of theACM SIGOPS 22nd symposium on Operating systems principles , pages29–44. ACM, 2009.[2] S. Doeraene. Scala.js: Type-Directed Interoperability with Dynami-cally Typed Languages. Technical report, École polytechnique fédéralede Lausanne, 2013.[3] M. Fogleman. fogleman/gg: Go Graphics - 2D rendering in Go with asimple API , 2016. https://github.com/fogleman/gg .[4] L. T. Hansen and J. Fairbank.
ECMAScript Shared Memory and Atom-ics , 2016. https://tc39.github.io/ecmascript_sharedmem/shmem.html .[5] H. Härtig, M. Hohmuth, J. Liedtke, J. Wolter, and S. Schönberg. Theperformance of µ -kernel-based systems. In Proceedings of the Six-teenth ACM Symposium on Operating Systems Principles , SOSP ’97,pages 66–77, New York, NY, USA, 1997. ACM.[6] R. Hickey. Clojurescript. http://clojurescript.org/about/rationale , 2016.[7] T. Mauro.
Adopting Microservices at Netflix: Lessons forArchitectural Design , 2015. .[8] R. Musiol. gopherjs/gopherjs: A compiler from Go to JavaScriptfor running Go code in a browser , 2016. https://github.com/gopherjs/gopherjs .[9] J. Vilk and E. D. Berger. D
OPPIO : Breaking the browser languagebarrier. In
Proceedings of the 2014 ACM SIGPLAN Conference onProgramming Language Design and Implementation (PLDI 2014) ,pages 508–518. ACM, 2014.[10] J. Vouillon and V. Balat. From bytecode to JavaScript: the Js_of_ocamlcompiler.
Software: Practice and Experience , 44(8):951–972, 2014.[11] D. Yoo and S. Krishnamurthi. Whalesong: running racket in thebrowser. In
DLS’13, Proceedings of the 9th Symposium on DynamicLanguages, part of SPLASH 2013, Indianapolis, IN, USA, October26-31, 2013 , pages 97–108, 2013.[12] A. Zakai. Emscripten: an LLVM-to-JavaScript compiler. In
OOPSLACompanion , pages 301–312, 2011., pages 301–312, 2011.