Brian E. Mittelstaedt

Automatic image assessment from facial attributes

Personal consumer photography collections often contain photos captured by numerous devices stored both locally and via online services. The task of gathering, organizing, and assembling still and video assets in preparation for sharing with others can be quite challenging. Current commercial photobook applications are mostly manual-based requiring significant user interactions. To assist the consumer in organizing these assets, we propose an automatic method to assign a fitness score to each asset, whereby the top scoring assets are used for product creation. Our method uses cues extracted from analyzing pixel data, metadata embedded in the file, as well as ancillary tags or online comments. When a face occurs in an image, its features have a dominating influence on both aesthetic and compositional properties of the displayed image. As such, this paper will emphasize the contributions faces have on affecting the overall fitness score of an image. To understand consumer preference, we conducted a psychophysical study that spanned 27 judges, 5,598 faces, and 2,550 images. Preferences on a per-face and per-image basis were independently gathered to train our classifiers. We describe how to use machine learning techniques to merge differing facial attributes into a single classifier. Our novel methods of facial weighting, fusion of facial attributes, and dimensionality reduction produce stateof- the-art results suitable for commercial applications.

Auto zoom crop from face detection and facial features

The automatic recomposition of a digital photograph to a more pleasing composition or alternate aspect ratio is a very powerful concept. The human face is arguably one of the most frequently photographed and important subjects. Although evidence suggests only a minority of photos contain faces, the vast majority of images used in consumer photobooks contain faces. Face detection and facial understanding algorithms are becoming ubiquitous to the computational photography community and facial features have a dominating influence on both aesthetic and compositional properties of the displayed image. We introduce a fully automatic recomposition algorithm, capable of zooming in to a more pleasing composition, re-trimming to alternate aspect ratios, or a combination thereof. We use facial bounding boxes, input and output aspect ratios, along with derived composition rules to introduce a facecrop algorithm with superior performance to more complex saliency or region of interest detection algorithms. We further introduce sophisticated facial understanding rules to improve user satisfaction further. We demonstrate through psychophysical studies the improved subjective quality of our method compared to state-of-the-art techniques.