A Revision Control System for Image Editing in Collaborative Multimedia Design
11 A Revision Control System for Image Editing inCollaborative Multimedia Design
Fabio Calefato, Giovanna Castellano, Veronica RossanoUniversity of BariItaly{fabio.calefato, giovanna.castellano, veronica.rossano}@uniba.it
Abstract —Revision control is a vital component in collab-orative development of artifacts such as software code andmultimedia. While revision control has been widely deployed fortext files, very few attempts to control the versioning of binaryfiles can be found in the literature. This can be inconvenientfor graphics applications that use a significant amount of binarydata, such as images, videos, meshes, and animations. Existingstrategies such as storing whole files for individual revisions orsimple binary deltas, respectively consume significant storage andobscure semantic information. To overcome these limitations,in this paper we present a revision control system for digitalimages that stores revisions in form of graphs. Besides, beingintegrated with Git, our revision control system also facilitatesartistic creation processes in common image editing and digitalpainting workflows. A preliminary user study demonstrates theusability of the proposed system.
Keywords:
Collaborative multimedia design, revision con-trol, image processing, digital painting.I. I
NTRODUCTION
In multimedia design and development, there is a widerange of contents such as text, images, video and audio thatneed to be created and edited. Recently, collaborative formsof multimedia development have revealed useful for authoring,editing, collecting, and producing digital content [1],[2],[3].When the development of multimedia is carried out in col-laborative and integrated design environments, revision controlbecomes essential to calculate, represent, and store differencesbetween successive versions of the developed digital objects.Indeed, the development process can involve many authorswith potentially different skills and different authoring tools.The standard paradigm of collaborative editing lies on sharingfiles between various instances of applications. This requiresmaintaining consistency of the versions and dealing withconcurrent edits in the same part of a multimedia.So far, revision control has been widely deployed for textfiles, while binary files have received relatively less attention.Very few methods are available for the efficient storage ofmodifications on any binary content. Existing strategies suchas storing whole files for individual revisions or simple binarydeltas could consume significant storage and obscure semanticinformation. This can be inconvenient for multimedia appli-cations that use a significant amount of binary data, such asimages, videos, and animations.In this work, we present a revision control system for digitalimages that is intended to be used in a collaborative scenarioof multimedia development. The core idea is the storage of graphical operations using a directed acyclic graph (DAG),where each vertex contains the result of a performed editingoperation. This representation is suitable for nonlinear revisioncontrol and does not require storing the whole modified image.II. R
ELATED WORK
Revision control systems are widely used tools in softwaredevelopment primarily aimed on managing versions of soft-ware source code during implementation [4]. Since sourcecode is written in plain text format, these systems are generallyeasily applicable to text documents. However these systemshave traditionally lacked specific support for handling binaryfiles. Even modern revision control systems like Git still offerlimited built-in support for binary content [5]. Some extensionshave been proposed (e.g., Git LFS), but they are meant to addsupport for storing large binary files rather than for versioning.Revision control systems can be distinguished as state- or change-based depending on the adopted history model, that is,how the history of changes to artifacts is stored [8]. Change-based systems (also known as operation-based) store the oper-ations actually performed between two succeeding versions inthe revision control system. Instead, state-based systems storethe history of changes as revisions of the versioned artifacts asthey existed at various points in time. Most existing general-purpose revision systems such as Git and SVN employ astate-based model. These modern revision control tools savestorage space by computing and persisting only the difference(i.e., delta [7]) between succeeding revisions, while preservingthe full state of a few special versions – like initial or final(head) revisions. However, when dealing with image data,these systems store separate images for each revision, thuswasting storage space. This issue alone hampers the adoptionof revision control systems in managing digital images.Besides, commonly used image-editing software tools offera very limited control of image versions. For example, AdobePhotoshop provides a version history that retains the undoneoperations on a stack. Hence, the user can easily jump to anyrecent state of the image created during the current workingsession. However, high-level operations such as comparison(diff), branching, or merging of different versions are notavailable in Photoshop, nor in other current drawing tools.To overcome these limitations, Chen et al. [6] proposed asolution for the nonlinear revision control of images, imple- https://git-lfs.github.com mented as a plug-in of the image editor GIMP, which tracksuser editing actions in form of graphs to visualize revisions andsupport complex operations such as branching and merging.Along with the idea of the work proposed in [6], we havedeveloped a revision control system for digital images thatimplements a hybrid approach combining the use of graphswith both state- and change-based revision control. First, likeother change-based systems [8], we expose history as DirectedAcyclic Graph (DAG, see Section III for more) to representspatial, temporal, and semantic dependencies between succes-sive recorded image-editing operations that are stored as graphnodes. Then, like state-based revision control system, we allowusers to store important revisions as binary files, thanks to theintegration with Git.Furthermore, our system differs from the one proposed in[6] also in the following aspects: • Rather than forking the GIMP image editing software,we developed our prototype almost from scratch, us-ing Python and leveraging only stable and well-knownlibraries (e.g., Mathplotlib, Pillow). While the forkedversion of the tool developed by Chen et al. [6] is nowobsolete (i.e., no one has ported the changes to the newerversions of GIMP), our approach promises to be morefuture-proof by allowing users to freely upgrade librariesas new updates are released. • Our system is integrated with Git and GitHub, thus itenables teams to collaborate in the creation of both visualmaterial and text artifacts. Besides, our system works wellwith the Git-LFS extension, thus allowing users to checkout only the entire image files needed for the current taskat hand, and just symbolic references for the other imagesnot needed. • Finally, instead of using custom formats [6], we storemeta-data and project-related information using files instandard formats such as JSON and CSV.III. T
HE REVISION CONTROL SYSTEM
The core data structure of our system is a Direct AcyclicGraph (DAG) that is used to store the revisions as deltas.DAG nodes represent image editing operations with relevantinformation such as the type of operation and its parame-ters, the author who applied the operation, the time of theapplication and eventual notes. DAG edges represent therelationships between the operations. A (directed) sequentialpath between two nodes implies a spatial and/or semanticdependency between operations and the path direction givesinformation about their temporal order. Spatial dependencyconsiders the spatial relationships between operations. Twooperations are spatially independent if they are applied tonon-overlapping regions. For example, drawing a shape and coloring it are spatially dependent operations. Conversely, drawing a shape and coloring another existing shape areindependent operations. Semantically independent operationsare rigid transformations (translation, rotation), deformation(scale, shear, perspective), color adjustment (hue, saturation,brightness, contrast, gamma) and filter (e.g., blur, sharpen). http://gimp.org Table IO PERATIONS LEADING TO THE CREATION OF A NEW NODE IN THE
DAG.
Type Operation
Rigid Transformation
Mirror, Flip, Transpose
Deformation
Scale
Color and Filter
Histogram, Brightness, B&W,Sepia, Invert, Solarize, Posterize
Edit
Crop, Text, Reset
Brush
Brush
Load image
New, Import
Multiple parallel paths between two nodes imply indepen-dent operation sequences, namely those that apply on disjointregions. The DAG records the user editing operations anddynamically grows as more revisions are committed. Eachrevision in our system is a sub-graph of the DAG containingthe root node which represents the act of initialization, i.e.opening an empty canvas or loading an existing image. Thestate of the revision is always equivalent to the result generatedby traversing its corresponding sub-graph. It should be notedthat in our system, the DAG encodes only actions, not wholeimages.
A. Revision control commands
Based on the DAG data structure, our system providesthe primary mechanisms for automatic resolving and merg-ing multiple revisions with potential conflicts, as well as auser interface that allows manual change and interventionon automatically merged images. The implemented revisioncontrol commands include review, addition, branch, merge,and conflict resolving. We also provide an image diff toolthat can be particularly useful to track not only low-level(pixel-based) differences but also high-level differences thatmodify the semantic content conveyed by the image. All thesefunctionalities are offered through a friendly user interface.Table I lists the editing operations that lead to the creation of anew node in the DAG. In the following, we briefly describe themain revision control commands implemented in our system.
1) Diff:
While the classic line-based diff command [7] iscommonly used to extract differences between text files, thereis no such well-defined difference tool for images. Amonggeneral image comparison visualization approaches, popularones include side-by-side comparison (e.g. Adobe Bridge,Perforce), layer-based difference (e.g. PixelNovel), per-pixeldifference, image overlay (e.g. Wet Paint [10]), and flickeringdifference regions (e.g. the compare utility of ImageMagick).These approaches are designed to handle only low level bitmapdifferences, with little information about the editing semantics.In contrast, following the idea of Chen et al. [6], our systemrealizes an informative diff by recording all the relevanthigh level information in a DAG. Two visual mechanismsfor diff revision are available in our system. The first isthe visualization of the DAG itself (fig. 5), which users candirectly interact with to obtain visual clues about the involvedediting operations. The second is a standalone diff UI thatcan be triggered by right-clicking on the DAG window. Anew window is opened where the user is required to specify a source node and a target node. The diff UI provides a side-by-side comparison between revisions as well as sequentialreplay via a slider (fig. 1).
2) Merge:
Unlike text, merging two images requires com-plex procedures in order to identify the difference betweenthem. In text files changes are identified through line by linecomparison. Instead, with binary files it is difficult to definewhich parts of an image have been changed. One way toimplement the merge of two images is to represent imagesas a matrix whose elements contain information about eachpixel within the image. In particular, rows ( x ) and columns( y ) are the pixel coordinates, and each element of the matrixis an array of four elements, i.e., the three RGB (red, green,blue) values in (0 , that determine the pixel color, plusthe alpha level, which determines the opacity. Then, using thisrepresentation, applying the merge operator on two differentDAG nodes will generate a new node including all changes(fig. 2). As such, the new node will be the obtained byapplying the changes identified through the the pixel-wisediff between the two image matrices, that is, comparingthe changes between the two arrays at the same position(coordinates). Should images modify the same pixel, ratherthan generating a conflict, we assume the ‘latest’ change tosimply overwrite the other (we recall that all actions arerecorded with a time-stamp).
3) Commit:
Our prototype is capable of using Git to storeimages revisions locally as commits . Committing revisions isone of the most frequently used revision control commands.In our system, to save the current work progress as a newbinary revision, users can simply issue the command gitcommit from the revision control window. Initially, whenthe repository is empty, the user adds the initial image andcommits it as revision 0. Further commits replace individualnodes corresponding to delta changes, defining a new head
Figure 1. Diff UI. The preview window (left) shows the editing processbetween two revisions (Target and Source). Users can manually drag the sliderfor a particular state. Figure 2. An example of two-way image merge: from node 2 a new branch(node 3) and revision (node 4) are started, which are later merged into thefinal revision (node 5). revision. Although users can commit revisions whenever theylike, it is generally unnecessary for them to do so in an action-wise fine-grained fashion, since our system can record all theactions and flexibly visualize them with DAG. As a generalguideline, users proceed to commit revisions when one of thefollowing two conditions is met: 1) some milestone of thework is achieved or 2) users would like to try out differentvariations. In the latter case, the committed revision can beused as a branch point for future reference or revision rollback.
4) Push and Pull:
Our prototype is also capable of usingGit repositories shared online on GitHub. When a new projectis started, the URL of the associated GitHub repository mustbe provided, along with the path of a local folder wherethe content of the remote repository will be cloned locally.Then, it is possible to execute the git push and git pull commands, respectively, to store remotely pending commitsand download updates pushed by other collaborators.Furthermore, our system is compatible with the Git-LFS(Large-File Storage) extension that allows, when enabled, thestorage of large files to a separate repository. The originalrepository will only contain pointers to the actual large files.Thus, user can decide to actually download these files onlyif need be. As such, the use of Git-LFS is recommendedfor speeding up the access to project repositories that hosta number of very large files as in the case of images.
B. Implementation
The proposed system is entirely implemented in Python 2.7.The following libraries have been employed: • PIL and Pillow for image editing; • Networkx to create the structure of the DAG; • Mathplotlib to visualize nodes and arcs of the DAG; • Json to build the files necessary for handling the DAG; • Git-lfs and gitdb2 to implement support for Git andGitHub.
We have fully integrated our revision control system withan existing open source tool, called
Image Editor [9], whichwe forked and evolved considerably. Specifically, we havemodified the editor itself and added two main components:the logger , for recording user actions, and the replayer , forreplaying actions starting from the stored deltas. The loggersilently records user-editing actions in the background in theform of text logs that can be replayed in the image editor viathe replayer. Precisely, when a user applies an operation thatmodifies the image, the logger detects the type of operationand creates three log files: • a CSV file containing information related to a node; • a JSON file describing the structure of the DAG; • a CSV file containing the revision (delta).The last file is used by the replayer to reconstruct the image af-ter edition operation. All the user actions involving changes inthe image are annotated in the log files to enable the porting ofthe project among different platforms and operating systems.In addition, for each new project the system creates the file Project.properties containing the project information(e.g author, revision format,...). All recorded logs and otherrevision control information are stored in a local repository(i.e. the project directory) and they can eventually be storedalso in a repository shared online on GitHub. The logs areanalyzed and transformed into DAG. When a new revision iscommitted, the corresponding action logs are transferred intothe repository and recorded in the DAG.
C. User Interface
The whole User Interface (UI) has been developed usingthe
Tkinter library of Python. Below we describe the mainusage scenarios of our system through the UI. When the userlogs to the system, the starting window appears as depicted infig. 3. As in most graphic software, there is a tool bar (on theleft) with buttons to apply the different functions are available.On the top there is a menu bar where all the additionalfunctionalities can be activated, such as project creation andsaving. The canvas on the center of the window allows theuser to operate on the image. In the orange bar on the bottomthe user can add an annotation to the image. This feature isuseful in collaborative settings to annotate the applied changeswith a motivation. To start the work, it is necessary to create anew project. When the user selects the command
File>NewProject a pop-up window appears (fig. 4) that enables theuser to enter the information related to the project, such as thename and the source image (if any). Users can also specifythe author name, useful to keep track of who changed what incollaborative settings, and the file format to be used. Currentlythe system supports the following image formats: jpeg, png,tiff, and bmp.By selecting Show History from either the toolbar or themenu, the DAG related to the open project will be displayed(fig. 5). Each node will be identified using both a number anda thumbnail. The number is useful to indicate the semanticrelationship and the timeline of revisions. It is also possibleto interact with each node: the right-click shows the detailsabout the node (name, operator, time) and the left-click allows
Figure 3. Starting window of the UI.Figure 4. New Project pop-up window. choosing among different options such as apply the diff and merge operators, and refresh the DAG in order to visualizethe latest changes applied.For each node, a double-click opens the Node Info window(fig. 6), which shows the revision image and other informationsuch as the author, the image size, and its annotations. In thesame window, users can add or edit existing annotations oreven start a new revision branch from the current image (seefig. 6). IV. U
SER T EST
In order to evaluate the system, a usability test has been con-ducted. The adopted procedure is compliant to eGLU usabilityprotocol used by the Italian Public Administration [11]. Thisprotocol, which is one of the most used for simplified usabilitytests, it has been enriched by the eGLU-M for mobile systemsand by the Usability Glossary of WikiPA project [11]. Theprotocol has been defined in order to be adaptable to differentkinds of software. In this case, the following documents relatedto eGLU 2.1 protocol have been used: • data of participants; • description of the task; • questionnaire for computing the Net Promoter Score(NPS); • questionnaire for computing the System Usability Scale(SUS); • table of results. A. Sample
The participants involved in the study were 5 computerscience students. As argued by Nielsen [12], 5 users would beexpected to find 85% of the usability problems. The samplein our test was homogeneous since all the subjects had ahigh computer experience and no experience with the revisioncontrol system.
B. Assigned tasks
Each user was asked to complete the following tasks:1) create a project;2) apply three changes to the image (including resize andfiltering operators);3) add an annotation in the main window;4) visualize the revision history;5) visualize the information related to node 1 and to updatethe annotation;6) add a branch to node 1;7) apply the diff operator to visualize the changes madefrom node 0 to node 4,8) merge two nodes;9) to make a pull and a push from and to a GitHubproject.We observed that all the students took similar amounts oftime for each task and anyone encountered some difficulties inusing the system. In particular, 2 users had some problems incompleting task
Figure 5. Example of DAG related to a project. Figure 6. Node Info window. operators (and hence they used the Brightness operator in placeof a filtering operator). Moreover, 1 user encountered somedifficulty in adding the annotation (task
C. System Usability Scale
The System Usability Scale (SUS) is a quick and reliabletool for measuring the usability of an artifact (software code ormultimedia) [13] and is currently used to evaluate the usabilityof different kinds of products and services. The questionnaireconsists of 10 items using a 5-point Likert scale. For eachitem the participant’s response is converted in a new numberand then it is summed to all the scores to obtain a number in[0,100]. The obtained score represents the average satisfactionlevel of the sample used in the user test. In this particular case,the sample is not very representative of the typical users ofthe system, thus the results can not be generalized, but theyare indicative of possible usage problems. Based on literature,a SUS score above 68 can be considered above the average. Inour case an average SUS score of 73.5 was achieved (fig. 7),hence we can conclude that at its primary stage the usabilityof the system is acceptable.V. C
ONCLUSIONS
In this work, we have presented a system for the revisioncontrol of digital images. Unlike other existing systems forimage revision control, the proposed system adopts a hybridapproach that saves user editing actions as direct acyclicgraphs (to save storage space), but also allows user to saveimportant milestones revisions as binary files. Finally, thanksto the integration with the popular revision control system Git,
Figure 7. Results of the usability test on 5 participants (P1,P2,...,P5). our system is capable of supporting distributed collaborativeteam work. The usability study conducted with a few subjectsprovide initial evidence that our revision system is easy to use.However, the usability test focused on single-user tasks thatcapture the predominant usage scenarios in multimedia design.Further tests should be carried out in order to evaluate oursystem in collaborative scenarios of multimedia development.The image editor developed so far is intended as a proof ofconcept for our idea. For this reason, it offers only a minimalset of graphical operations. Nevertheless, it can be alreadyused in real-world scenarios, e.g., by graphic designers. Asa forthcoming future work, we intend to make our solutionmore appealing by integrating it within a more sophisticatedand widely used open-source image editor.A
CKNOWLEDGEMENT
This work is partially funded by the project "Creative Cul-tural Collaboration" (C3) under the Apulian INNONETWORKprogramme, Italy. R
EFERENCES[1] D.S. Slaughter and M. C. Murtaugh, “Collaborative Management of theeLearning Design and Development Process,” Leading and Managinge-Learning. Springer, Cham. pp. 253-269, 2018.[2] C.W. Chen, J.W. Peng, C.M. Kuo, M.C. Hu, and Y.C. Tseng, “Ontlus:3D Content Collaborative Creation via Virtual Reality,” InternationalConference on Multimedia Modeling (pp. 386-389). Springer, Cham.,2018.[3] T.H. Claman, S.J. Coniglio, S. Daigle, R. A. Gonsalves, and R.C.Wallace (2018). U.S. Patent No. 9,864,973. Washington, DC: U.S.Patent and Trademark Office.[4] N.B. Ruparelia, “The History of Version Control,” SIGSOFT Softw.Eng. Notes, vol. 35, no. 1, pp.5-9, 2010.[5] J. Dob˘os and A. Steed, “3D Revision Control Framework,” Proceedingsof the 17th International Conference on 3D Web Technology (Web3D’12) (pp. 121-129). New York: ACM, 2012.[6] H.T. Chen, L.Y. Wei, and C.F. Chang, “Nonlinear revision control forimages,” ACM Transactions on Graphics, vol. 30, no. 4, pp. 105, tenpages, 2011.[7] J.J. Hunt, K.-P. Vo, and W.F. Tichy, “Delta algorithms: an empiricalanalysis,” ACM Trans. Softw. Eng. Methodol., vol. 7, no. 2, pp.192–214, 1998.[8] M. Kleine, R. Hirschfeld, and G. Bracha, “An abstraction for versioncontrol systems,” (No. 54). Universitätsverlag Potsdam, 2012.[9] R. Savla, Term Project: Image Editor. https://github.com/riyasavla/Image-Editor [10] L. Bonanni, X. Xiao, M. Hockenberry, P. Subramani, H. Ishii, M.Seracini, and J. Schulze, “Wetpaint: scraping through multi-layeredimages,” In Proceedings of the SIGCHI Conference on Human Factorsin Computing Systems (pp. 571-574). ACM, 2009. [11] Sito Pubblica Amministrazione, Gruppo di lavoro per l’Usabilità - GLU.