Jerry Liu

Recommendation system for automatic design of magazine covers

In this paper, we present a recommendation system for the automatic design of magazine covers. Our users are non-designer designers: individuals or small and medium businesses who want to design without hiring a professional designer while still wanting to create aesthetically compelling designs. Because a design should have a purpose, we suggest a number of semantic features to the user, e.g., clean and clear, dynamic and active, or formal, to describe the color mood for the purpose of his/her design. Based on these high level features and a number of low level features, such as the complexity of the visual balance in a photo, our system selects the best photos from the users album for his/her design. Our system then generates several alternative designs that can be rated by the user. Consequently, our system generates future designs based on the users style. In this fashion, our system personalizes the designs of a user based on his/her preferences.

Automatic design of magazine covers

In this paper, we propose a system for automatic design of magazine covers that quantifies a number of concepts from art and aesthetics. Our solution to automatic design of this type of media has been shaped by input from professional designers, magazine art directors and editorial boards, and journalists. Consequently, a number of principles in design and rules in designing magazine covers are delineated. Several techniques are derived and employed in order to quantify and implement these principles and rules in the format of a software framework. At this stage, our framework divides the task of design into three main modules: layout of magazine cover elements, choice of color for masthead and cover lines, and typography of cover lines. Feedback from professional designers on our designs suggests that our results are congruent with their intuition.

Cloud-based printing for mobile devices

Consumers are increasingly using their smart phones to view web pages. However, there is no native operating system support for printing these web pages. We propose to overcome two barriers to printing from mobile devices - the inability to connect and transmit to a printer and the typically poor format of printed web pages. Our system includes a client component that causes the web browser to upload the page (as a URL reference for public pages or the DOM content for private pages) to a cloud service that extracts the content and formats it for printing. We transfer the printready content to the HP CloudPrint service and leverage its ability to locate printers and transmit print jobs. We have built a working system the uses iPhones and Windows Mobile devices clients, but the system can be extended to include other clients.

Multimedia analysis and composition cloud service

Consumers increasingly desire to create and consume multimedia seamlessly from any of their devices wherever they happen to be. To meet this demand, multimedia services such as analysis and composition are increasingly moving to the cloud. This trend is enabling multimedia capabilities that can be accessed through mobile and social network applications as well as web browsers. In this paper, we discuss the architecture of a multimedia analysis and composition cloud service. We show examples of how this architecture can be used to provide browser-based multimedia applications, mobile applications, and social network applications, while comparing the advantages of each access method.

Automatic design of colors for magazine covers

In the design of a magazine cover, making a set of decisions regarding the color distribution of the cover image and the colors of other graphical and textual elements is considered to be the concept of color design. This concept addresses a number of subjective challenges, specifically how to determine a set of colors that is aesthetically pleasing yet also contributes to the functionality of the design, the legibility of textual elements, and the stylistic consistency of the class of magazine. Our solution to automatic color design includes the quantification of these challenges by deploying a number of well-known color theories. These color theories span both color harmony and color semantics. The former includes a set of geometric structures that suggest which colors are in harmony together. The latter suggests a higher level of abstraction. Color semantics means to bridge sets of color combinations with color mood descriptors. For automatic design, we aim to deploy these two viewpoints by applying geometric structures for the design of text color and color semantics for the selection of cover images.

An investigation of document aesthetics for web-to-print repurposing of small-medium business marketing collateral

Businesses have traditionally relied on different types of media to communicate with existing and potential customers. With the emergence of the Web, the relation between the use of print and electronic media has continually evolved. In this paper, we investigate one possible scenario that combines the use of the Web and print. Specifically, we consider the scenario where a small- or medium-sized business (SMB) has an existing web site from which they wish to pull content to create a print piece. Our assumption is that the web site was developed by a professional designer, working in conjunction with the business owner or marketing team, and that it contains a rich assembly of content that is presented in an aesthetically pleasing manner. Our goal is to understand the process that a designer would follow to create an effective and aesthetically pleasing print piece. We are particularly interested to understand the choices made by the designer with respect to placement and size of the text and graphic elements on the page. Toward this end, we conducted an experiment in which professional designers worked with SMBs to create print pieces from their respective web pages. In this paper, we report our findings from this experiment, and examine the underlying conclusions regarding the resulting document aesthetics in the context of the existing design, and engineering and computer science literatures that address this topic

Mobile document scanning and copying

In this paper, we show a multimedia system for processing mobile camera captured documents. Using a client application on a mobile phone, a user can capture a document image, and send the image to a processing server so that the document image can be restored using automatic perspective and illumination corrections. The restored document can then be sent to a web-connected printer to complete the copying task.

Learning from graphic designers - using grids as a scaffolding for automatic print layout

We describe an approach for automatically laying out content for high quality printed formats such as magazines or brochures, producing an aesthetically pleasing layout that correctly conveys the semantic structure of the content and elicits the desired experiential affect in the reader. The semantic structure of the content includes the reading order graph, the association of illustrations with referring paragraphs, and the preservation of perceived text hierarchies. We appropriate a popular conceptual tool used by graphic designers called the grid. A well-designed grid will cause a pleasing uniformity through all the pages of a publication while still allowing flexibility in the layout of each page. In the space of different automatic layout systems, our approach is somewhere between template-based techniques and generative techniques, with the aesthetics determined by the combination of the grid and a generative algorithm One consequence of using the grid is that it greatly reduces the space of possible layouts from a high dimensional continuous space to a discrete space. Using a simple greedy algorithm, our first results are promising.

Snapfish Lab: An open online imaging community

With the increasing amount of digital media being stored, shared and consumed on the Internet, more imaging applications are deployed online. However, the usability and use model of those applications are often less studied. In this paper, we present Snapfish Lab, an open online imaging community web site, to enable both imaging technologists and enthusiasts participate actively in exploring the usability and use model of the exploratory imaging applications. The main topic addressed in this paper is the challenges and solutions of building a production quality Web site that supports open and high performance imaging application hosting, rich user feedback collection and satisfactory end user experience. The key value offered by Snapfish Lab is to bridge imaging research community and end users so that userpsilas demand becomes part of innovation cycle to drive Web based imaging research and development.

An end-to-end system for content-based video retrieval using behavior, actions, and appearance with interactive query refinement

We describe a system for content-based retrieval from large surveillance video archives, using behavior, action and appearance of objects. Objects are detected, tracked, and classified into broad categories. Their behavior and appearance are characterized by action detectors and descriptors, which are indexed in an archive. Queries can be posed as video exemplars, and the results can be refined through relevance feedback. The contributions of our system include the fusion of behavior and action detectors with appearance for matching; the improvement of query results through interactive query refinement (IQR), which learns a discriminative classifier online based on user feedback; and reasonable performance on low resolution, poor quality video. The system operates on video from ground cameras and aerial platforms, both RGB and IR. Performance is evaluated on publicly-available surveillance datasets, showing that subtle actions can be detected under difficult conditions, with reasonable improvement from IQR.