Robert Ulichney

Dithering with blue noise

Digital halftoning, also referred to as spatial dithering is a method of rendering the illusion of continuous-tone pictures on displays that are capable of only producing binary picture elements. The concept of blue noise-high-frequency white noise-is introduced and found to have desirable properties for halftoning. Efficient algorithms for dithering with blue noise, based on perturbed error diffusion, are developed. The nature of dither patterns produced is extensively examined in the frequency domain. Metrics for analyzing the frequency content of aperiodic patterns for both rectangular and hexagonal grids are developed; blue-noise dithering is found to be ideally suited for rectangular grids. Several carefully selected digitally produced examples are included. >

Blue and green noise halftoning models

In this article, we review the spatial and spectral characteristics of blue- and green-noise halftoning models. In the case of blue noise, dispersed-dot dither patterns are constructed by isolating minority pixels as homogeneously as possible, and by doing so, a pattern composed exclusively of high-frequency spectral components is produced. Blue-noise halftoning is preferred for display devices that can accommodate isolated dots such as various video displays and some print technologies such as ink-jet. For print marking engines that cannot support isolated pixels, dispersed-dot halftoning is inappropriate. For such cases, clustered-dot halftoning is used to avoid dot-gain instability. Green-noise halftones are clustered-dot, blue-noise patterns. Such patterns enjoy the blue-noise properties of homogeneity and lack low-frequency texture but have clusters of minority pixels on blue-noise centers. Green noise is composed exclusively of midfrequency spectral components. In addition to the basic spatial and spectral characteristics of the halftoning models, this article also reviews some of the earlier work done to improve error diffusion as a noise generator. We also discuss processes to generate threshold arrays to achieve blue and green noise with the computationally efficient process of ordered dither.

Blue-noise halftoning for hexagonal grids

In this paper, we closely scrutinize the spatial and spectral properties of aperiodic halftoning schemes on rectangular and hexagonal sampling grids. Traditionally, hexagonal sampling grids have been shunned due to their inability to preserve the high-frequency components of blue-noise dither patterns at gray-levels near one-half, but as will be shown, only through the introduction of diagonal correlations between dots can even rectangular sampling grids preserve these frequencies. And by allowing the sampling grid to constrain the placement of dots, a particular algorithm may introduce visual artifacts just as disturbing as excess energy below the principal frequency. If, instead, the algorithm maintains radial symmetry by introducing a minimum degree of clustering, then that algorithm can maintain its grid defiance illusion fundamental to the spirit of the blue-noise model. As such, this paper shows that hexagonal grids are preferrable because they can support gray-levels near one-half with less required clustering of minority pixels and a higher principal frequency. Along with a thorough Fourier analysis of blue-noise dither patterns on both rectangular and hexagonal sampling grids, this paper also demonstrates the construction of a blue-noise dither array for hexagonal grids. EDICS: 4-QUAN Quantization and Halftoning.

Review of halftoning techniques

Digital halftoning remains an active area of research with a plethora of new and enhanced methods. While several fine overviews exist, this purpose of this paper is to review retrospectively the basic classes of techniques. Halftoning algorithms are presented by the nature of the appearance of resulting patterns, including white noise, recursive tessellation, the classical screen, and blue noise. The metric of radially averaged power spectra is reviewed, and special attention is paid to frequency domain characteristics. The paper concludes with a look at the components that comprise a complete image rendering system. In particular when the number of output levels is not restricted to be a power of 2. A very efficient means of multilevel dithering is presented based on scaling order- dither arrays. The case of real-time video rendering is considered where the YUV-to-RGB conversion is incorporated in the dithering system. Example illustrations are included for each of the techniques described.

One-dimensional dithering

A real-time inverse dithering system for video display can be implemented very efficiently if operations are needed on only the current scan line. To optimize overall display quality, corresponding one-dimensional ordered dither array is sought. This paper describes a one-dimensional recursive tessellation algorithm. A serendipitous implementation involves a simple bit-reversal of the horizontal pixel address. To optimize two- dimensional homogeneity, the 1-D array is phase adjusted in the vertical direction. A scheme for selecting candidate phase vectors is also presented. The recursive tessellation algorithm i generalized to identify equivalence class arrays that share the same homogeneity property but have different ordering.

Frequency Analysis of Ordered Dither

Ordered dither, the class of digital halftoning techniques which uses a periodic array of thresholds, is generalized for both rectangular and hexagonal grids, by means of the spatial method of recursive tessellation, a sub-tiling algorithm. The nature of the texture patterns so produced is illustrated and examined in the frequency domain, revealing several insights within and between the classes of rectangular and hexagonal grids. A simple explicit expression is derived which allows the use of the rectangular DFT to compute a hexagonal Fourier transform, maintaining continuous-space dimensions. Digitally produced examples are included.

Effects on Fourier Peaks Used for Periodic Pattern Detection

The detection of quasi-periodic patterns, such as those found in clustered-dot halftones, can be efficiently achieved by searching for strong peaks in the frequency domain. In this paper, we quantify four factors that contribute to the attenuation of those characteristic peaks related to mobile hand-held image capture. These include MTF, halftone cluster size, blur, and contrast. We derive the expected theoretical attenuation for each of these factors, and then compare these with experimentally measured results from mobile captured images of test prints.

Low-memory low-complexity inverse dithering

Dithering an image decreases the pixel bit depth and reduces the storage space required in the image buffer while largely preserving the perceptual quality. In some application it is desirable to reconstruct the original image; that is, restore the dithered image to its original bit-depth, for further processing or display. In this paper, we present a new color inverse dithering system designed for low-cost implementation. Our algorithms is based on edge-sensitive adaptive low-pass filtering. In order to prevent excessive blurring from low pass filtering, the system uses edge detection methods so that the filters are applied only to regions of constant color or gray level in the original image. One such method explores the fact that pixel values are only one level away from each other in a constant color region of a dithered image. Another method exploits a priori knowledge of the dither masks. By limiting the number of possible filters, and by restricting the region of support of the filters in to single image line, tremendous implementation advantages can be gained. Our prototype system uses a set of five filters, including a pair that are asymmetric about the origin specifically for application to object edges. In our implementation, the need for multipliers is eliminated by using bit replication for up- multiplication, and by using lookup tables with relatively small numbers of entries for filtering. We have found that our inverse dithering system can restore to a significantly degree a dithered image to its original form. It is especially effective for graphics and synthetic images.

Using a Data-bearing Frame to Capture an Extended Target

A system for the automatic detection of the entirety of an arbitrary image target is described. This is achieved by surrounding the target image with a data-bearing halftone or “StegaFrame” and by a camera-based detection system that can efficiently find the StegaFrame by means of judiciously placed detection windows. The frequency-domain-based detection scheme is fast and provides the added advantage of estimating scale and orientation of the desired content. An intuitive user interface guides the user until a successful detection is achieved. This approach is demonstrated in an application to locate and analyze a color chart for the purpose of calibrating a mobile device to be able to measure skin tone. An identifier to access print-run specific colorimetry data, along with other data for client analytics is embedded in the data-bearing border. The design of the detection system is optimized through exhaustive simulation. Experimental results confirm that the automated scheme, implemented on a mobile phone, performs as well as costly dedicated colorimetric hardware. Introduction There are numerous techniques which can be leveraged to flag importance of, embed data in, or associate information with a sample of graphical content. The specific problem we are concerned within this paper is the ability to identify the presence, in entirety, of an arbitrary selection of graphic content. Many analysis routines are initiated by the capture of such a desired target. Targeted inspection, forms processing and product identification are but three examples. While this result can be achieved with a process where a user makes a judgment of acceptability and manually invokes subsequent processing, our goal is to automate and therefore simplify the user-interface required to enable such an image capture operation. Barcoding techniques and watermarking schemes offer two types of solutions with very different properties. Barcodes can be interpreted by a plethora of devices at the expense of yielding graphic design flexibility. Watermarking techniques [1][2], on the other hand, can be used, for example, to associate information with graphical with, in most cases, minor visual artifacts. For some detection and analysis tasks, however, neither approach is suitable; barcodes can be intrusive, if not disallowed from use, and some watermarking schemes (such as those based on modifying wavelet or frequency coefficients) do not provide much localization information. In addition, neither of these options may provide suitable data density given the amount of space that can be used for this purpose. Other alternatives include image matching algorithms such as the Viola-Jones object detection framework [3], SIFT-based feature detectors [4], and SURF-based feature detectors [5]; however these solutions must be designed for each specific target, they are computationally expensive, and they cannot provide accurate feedback about orientation and scale corrections. This work discusses a different style of solution, whereby the signature of a halftone structure designed to carry data, is imposed upon a rendering of a border around the content of interest. The border serves multiple purposes. First, it draws attention to the fact that the content is of/should be of interest to a user. Second, it provides a signature of its presence strong enough to be detected automatically. Third, it can be used to represent any data critical to a given application. In this manner, it becomes possible to construct a completely generic object detection, or in this case, corralling scheme. The approach is demonstrated in an application involving a color characterization chart. When successfully captured, the image data of the color chart combined with the corresponding colorimetric data can be used to measure the color of a sample, such as skin tone. Fortified with this feature, we developed an accurate mobile color measurement scheme that can replace more costly hardware used for the same purpose. Detecting and Corralling an Object We have developed a system for corralling an object with a “StegaFrame”, a steganographic halftone or stegatone [6] of a fixed gray frame that surrounds a target. Data is embedded by means of single-pixel shifts of halftone clusters. We exploit the close-focus ability of smartphone cameras that deliver the resolution necessary to interpret the high capacity encodings. Measurement of the effective resolution of most current smartphones as a function of distance is plotted in Figure 1. It is important to point out that this data is for video capture; full-sensor photo capture offers much higher resolutions but using it does not often result in a fluid user interface. Figure 1. Smartphone video capture resolution. An example StegaFrame, in this case used to corral color characterization patches, is shown in Figure 2, along with a depiction of how it is used for measuring the skin tone with our ©2016 Society for Imaging Science and Technology IS&T International Symposium on Electronic Imaging 2016 Color Imaging XXI: Displaying, Processing, Hardcopy, and Applications COLOR-324.1 mobile app. 72 color patches surround a hole through which the skin sample is measured. The color chart is surrounded by a StegaFrame used for automatic detection. The 2.2 by 3.6 inch frame appears as a gray border but is comprised of 4428 halftone clusters each carrying 2 bits of data. With a redundancy factor of 4 this results in 2214 data payload bits. To account for occlusions even higher redundancy can be used. Figure 2. The StegaFrame enclosed color calibration target and its use for capturing a skin tone sample. Figure 3. Smartphone video display and UI. An example of our StegaFrame detection system UI is illustrated in Figure 3. Video is captured on an imaging device and four detection windows are positioned within the capture image to allow a range of distances at which the StegaFrame and inner chart can be captured. The detectors can, quite effectively and efficiently, identify the clustered-dot halftone pattern and determine the associated scale and orientation of the pattern within the detection window [7]. Detection windows are colored red in the overlaid display when the pattern is not detected and colored green when it is. Figure 3(a) illustrates a case where the camera is too close to the target. Two of the detection windows are red because they do not cover any part of the StegaFrame. The green windows properly detect the StegaFrame but also detect that the scale is too large (i.e., that the imaging device is too close to the target) and could report that back to the user. In Figure 3(b), the frame is detected in all four windows, the corresponding video frame is stored for processing, and the user is informed. In practice, only a subset of the correctly scaled positive detections are needed, which enables the approach to address partial occlusions. Real-time detection is accomplished by analyzing the Discrete Fourier Transform (DFT) amplitude and searching for peaks that are characteristic of the quasi-periodic stegatone pattern. To illustrate a typical example of the frequency domain signature associated with the border, consider the right-most detection window in Figure 3(b) as detailed in Figure 4. Figure 4. Detail of the pattern captured in the right-most detection window in Figure 3(b). The DFT magnitude of that detection window is shown in Figure 5. Even though only part of the detection window includes StegaFrame samples, the DFT is dominated by the energy from this halftone pattern as seen by the four peaks. To accentuate the presence of these peaks we mask the DC area that can be seen as the central disc. Also, the effect of windowing imparts strong harmonics on both axes, so we mask those areas as well. Figure 6 illustrates how detected peak positions to measure the distances H and V from which the horizontal and vertical scale are found, along with the rotation angle . The peaks in Figure 5 reveal that there are 3.43 horizontal samples/cluster, 3.41 vertical samples/cluster, and that the target is rotated by 8.49 degrees. Thus, the DFT of the contents of each window can be used to quickly estimated if the StegaFrame is present, along with an accurate determination of both scale and orientation angle. The combined results from all detection windows is used to establish if the target is successfully captured. (a) (b) ©2016 Society for Imaging Science and Technology IS&T International Symposium on Electronic Imaging 2016 Color Imaging XXI: Displaying, Processing, Hardcopy, and Applications COLOR-324.2 Horizontal cycles/pixel Ve rt ic al c yc le s/ pi xe l Figure 5. DFT of the rightmost detection window in Figure 3(b). Figure 6. DFT peak locations. Exhaustive Simulation Unlike lab measurement devices that can be carefully positioned and fixed in a location relative to a target to be captured, portable imaging systems may be hand-held and positioned by users in non-advantageous lighting conditions. The variability of capturing conditions must be accounted for in order to ensure both a fluid user experience and high quality results. Detection performance is measured in terms of correct or full detections (true positives), incorrect or partial detections (false positives), missed detections (false negatives) and cases of no detection when the object is not fully present (true negatives). Examples of each of these four states are illustrated in Figure 7. A true positive results when the whole of our corralled target is captured in the video frame and detection is indicated; a false positive occurs when the system indicates a detection and the whole of the target is not captured. False negatives events occur when the whole of the target is captured but detection is not indicated. True negatives are simply a no detection event when all or any part of the target is outside the video frame. The goal herein is to produce the largest number of true