Alexandra Bonnici

A circle-based vectorization algorithm for drawings with shadows

Vectorization algorithms described in the literature assume that the drawings being vectorized are either binary images or have a clear white background. Sketches of artistic objects however also contain shadows which help the artist to portray intent, particularly in potentially ambiguous sketches. Such sketches are difficult to binarise since the shading strokes make these sketches non bimodal. For this reason, we describe a circle-based vectorization algorithm that uses signatures obtained from sample points on the line strokes to identify and vectorize the line strokes in the sketch. We show that the proposed algorithm performs as well as other vectorization techniques described in the literature, despite the shadows present in the sketch.

Scribble Vectorization Using Concentric Sampling Circles

In this paper we introduce a path extraction algorithm for multi-stroke scribbled paths by making use of path-centred concentric sampling circles. Circle and line geometry is then exploited to efficiently obtain piece-wise linear models of the multi-stroke segments in the drawing. Parzen window estimation is used to obtain the probability distribution of the grey-level profile of the sampling circles to determine the intersecting angle of the sampling circle with the stroke segments and hence determine the line model parameters. The results obtained show that the algorithm identifies the line models accurately while reducing considerably the computational time required to obtain the line models.

Exploiting artistic cues to obtain line labels for free-hand sketches

Artistic cues help designers to communicate design intent in sketches. In this paper, we show how these artistic cues may be used to obtain a line labelling interpretation of freehand sketches, using a cue-based genetic algorithm to obtain a labelling solution that matches design intent. In the paper, we show how this can be achieved from off-line or paper based sketches, thereby allowing designers greater flexibility in the choice of sketching medium.

A constrained genetic algorithm for line labelling of line drawings with shadows and table-lines

Line drawings are well known to exhibit geometric ambiguities, resulting in, drawings that can have multiple interpretations. However, drawings are used to present design concepts to peers in fields such as engineering design, where it is imperative that the observer interprets the drawing in the same way as the designer for effective communication. Designers therefore use cues, prompting the observer to resolve the geometric ambiguities and achieve the desired interpretation. In this paper, we identify the cues introduced in drawings and focus on two cues, namely table-lines (which convey information about the position of the object in space) and shadows (which convey information about the geometry of the object). These cues can be used in a line-labelling context to allow a line-labelling algorithm to overcome the geometric ambiguities of the drawing. For this purpose, we propose a cue-constrained genetic algorithm that takes the vectorized line drawing and the identified cues attached to each edge, and uses these cues as constraints on the edge labels, thus distinguishing between different object-background interactions. We show that the proposed algorithm can be used to successfully label intentionally ambiguous line drawings according to some desired interpretation as specified by the additional cues present in the drawing. Graphical abstractDisplay Omitted Highlights? Create a cue dictionary by observing the way cues constrain an edge interpretation. ? Formalize the representation of foreground-background scenarios. ? Generalize the cue dictionary to allow for different light source locations. ? Use the cues as soft constraints in an edge labelling algorithm. ? Evaluate the algorithm with drawings with missing, inaccurate and inconsistent cues.

Genetic algorithm for line labeling of diagrams having drawing cues

Drawings are an integral part of the design process, helping designers communicate abstract concepts to others. In this paper we propose a genetic algorithm that successfully exploits cues present in drawings in a line labeling algorithm for sketches.

Preparation of Music Scores to Enable Hands-free Page Turning Based on Eye-gaze Tracking

Digital copies of musical scores may be saved on tablet devices, compressing volumes of scores into a single portable device. Tablet screens are however typically smaller than printed sheet music such that the score needs to be resized for readability. This necessitates additional page turning which is made more complex when repeat instructions are used since these give rise to forward and backward page turns of the music. In this paper, we tackle this problem by first performing image analysis of the score in order to identify repeat instructions and hence flatten the score. Thus, the music player is presented the score as it should be played. We then propose the use of eye-gaze tracking to provide a hands-free page turning mechanism. Thus, the player remains in full control of when the page turn occurs. Through a preliminary study, we found that our proposed score flattening and eye-gaze page turning reduced the time spent navigating the page turns by 47% in comparison to available music score reading tools.

Vectorisation of Sketched Drawings Using Co-occurring Sample Circles

This paper presents a drawing vectorisation algorithm which uses multiple concentric families of circles placed in a dense grid on the image space. We show that any off-centered junction within the family of circles can be located and hence show how these junction points may be linked to neighbouring junction points, thereby creating a vector representation of the drawing geometry. The proposed algorithm identified \(98\%\) of the junctions in the drawings on which it was evaluated, each within a localisation error of \(4.7 \pm 2.3\) pixels, resulting in straight line vectors which are well placed with respect to the drawn edges.

Automatic Ornament Localisation, Recognition and Expression from Music Sheets

Musical notation is a means of passing on performance instructions with fidelity to others. Composers, however, often introduced embellishments to the music they performed notating these embellishments with symbols next to the relevant notes. In time, these symbols, known as ornaments, and their interpretation became standardized such that there are acceptable ways of interpreting an ornament. Although music books may contain footnotes which express the ornament in full notation, these remain cumbersome to read. Ideally, a music student will have the possibility of selecting ornamented notes and express them as full notation. The student should also have the possibility to collapse the expressed ornament back to its symbolic representation, giving the student the possibility of also becoming familiar with playing from the ornamented score. In this paper, we propose a complete pipeline that achieves this goal. We compare the use of COSFIRE and template matching for optical music recognition to identify and extract musical content from the score. We then express the score using MusicXML and design a simple user interface which allows the user to select ornamented notes, view their expressed notation and decide whether they want to retain the expressed notation, modify it, or revert to the symbolic representation of the ornament. The performance results that we achieve indicate the effectiveness of our proposed approach.

Vectorisation of sketches with shadows and shading using COSFIRE filters

Engineering design makes use of freehand sketches to communicate ideas, allowing designers to externalise form concepts quickly and naturally. Such sketches serve as working documents which demonstrate the evolution of the design process. For the product design to progress, however, these sketches are often redrawn using computer-aided design tools to obtain virtual, interactive prototypes of the design. Although there are commercial software packages which extract the required information from freehand sketches, such packages typically do not handle the complexity of the sketched drawings, particularly when considering the visual cues that are introduced to the sketch to aid the human observer to interpret the sketch. In this paper, we tackle one such complexity, namely the use of shading and shadows which help portray spatial and depth information in the sketch. For this reason, we propose a vectorisation algorithm, based on trainable COSFIRE filters for the detection of junction points and subsequent tracing of line paths to create a topology graph as a representation of the sketched object form. The vectorisation algorithm is evaluated on 17 sketches containing different shading patterns and drawn by different sketchers specifically for this work. Using these sketches, we show that the vectorisation algorithm can handle drawings with straight or curved contours containing shadow cues, reducing the salient point error in the junction point location by 91% of that obtained by the off-the-shelf Harris-Stephens corner detector while the overall vectorial representations of the sketch achieved an average F-score of 0.92 in comparison to the ground truth. The results demonstrate the effectiveness of the proposed approach.

Expressive Piano Music Playing Using a Kalman Filter

In this paper, we present an algorithm that uses the Kalman filter to combine simple phrase structure models with observed differences in pitch within the phrase to refine the phrase model and hence adjust the loudness level and tempo of qualities of the melody line. We show how similar adjustments may be made to the accompaniment to introduce expressive attributes to a midi file representation of a score. In the paper, we show that the subjects had some difficulty in distinguishing between the resulting expressive renderings and human performances of the same score.