Tomás Denemark

Universal distortion function for steganography in an arbitrary domain

Currently, the most successful approach to steganography in empirical objects, such as digital media, is to embed the payload while minimizing a suitably defined distortion function. The design of the distortion is essentially the only task left to the steganographer since efficient practical codes exist that embed near the payload-distortion bound. The practitioner’s goal is to design the distortion to obtain a scheme with a high empirical statistical detectability. In this paper, we propose a universal distortion design called universal wavelet relative distortion (UNIWARD) that can be applied for embedding in an arbitrary domain. The embedding distortion is computed as a sum of relative changes of coefficients in a directional filter bank decomposition of the cover image. The directionality forces the embedding changes to such parts of the cover object that are difficult to model in multiple directions, such as textures or noisy regions, while avoiding smooth regions or clean edges. We demonstrate experimentally using rich models as well as targeted attacks that steganographic methods built using UNIWARD match or outperform the current state of the art in the spatial domain, JPEG domain, and side-informed JPEG domain.

Selection-channel-aware rich model for Steganalysis of digital images

From the perspective of signal detection theory, it seems obvious that knowing the probabilities with which the individual cover elements are modified during message embedding (the so-called probabilistic selection channel) should improve steganalysis. It is, however, not clear how to incorporate this information into steganalysis features when the detector is built as a classifier. In this paper, we propose a variant of the popular spatial rich model (SRM) that makes use of the selection channel. We demonstrate on three state-of-the-art content-adaptive steganographic schemes that even an imprecise knowledge of the embedding probabilities can substantially increase the detection accuracy in comparison with feature sets that do not consider the selection channel. Overly adaptive embedding schemes seem to be more vulnerable than schemes that spread the embedding changes more evenly throughout the cover.

Steganalysis Features for Content-Adaptive JPEG Steganography

All the modern steganographic algorithms for digital images are content adaptive in the sense that they restrict the embedding modifications to complex regions of the cover, which are difficult to model for the steganalyst. The probabilities with which the individual cover elements are modified (the selection channel) are jointly determined by the size of the embedded payload and the content complexity. The most accurate detection of content-adaptive steganography is currently achieved with the detectors built as classifiers trained on cover and stego features that incorporate the knowledge of the selection channel. While the selection-channel-aware features have been proposed for detection of spatial domain steganography, an equivalent for the JPEG domain does not exist. Since modern steganographic algorithms for JPEG images are currently best detected with the features formed by the histograms of the noise residuals split by their JPEG phase, we use such feature sets as a starting point in this paper and extend their design to incorporate the knowledge of the selection channel. This is achieved by accumulating in the histograms a quantity that bounds the expected absolute distortion of the residual. The proposed features can be efficiently computed and provide a substantial detection gain across all the tested algorithms especially for small payloads.

Further study on the security of S-UNIWARD

Recently, a new steganographic method was introduced that utilizes a universal distortion function called UNIWARD. The distortion between the cover and stego image is computed as a sum of relative changes of wavelet coefficients representing both images. As already pointed out in the original publication, the selection channel of the spatial version of UNIWARD (the version that hides messages in pixel values called S-UNIWARD) exhibits unusual properties – in highly textured and noisy regions the embedding probabilities form interleaved streaks of low and high embedding probability. While the authors of UNIWARD themselves hypothesized that such an artifact in the embedding probabilities may jeopardize its security, experiments with state-of-the-art rich models did not reveal any weaknesses. Using the fact that the cover embedding probabilities can be approximately estimated from the stego image, we introduce the novel concept of content-selective residuals and successfully attack S-UNIWARD. We also show that this attack, which is made possible by a faulty probabilistic selection channel, can be prevented by properly adjusting the stabilizing constant in the UNIWARD distortion function.

Improving Steganographic Security by Synchronizing the Selection Channel

This paper describes a general method for increasing the security of additive steganographic schemes for digital images represented in the spatial domain. Additive embedding schemes first assign costs to individual pixels and then embed the desired payload by minimizing the sum of costs of all changed pixels. The proposed framework can be applied to any such scheme -- it starts with the cost assignment and forms a non-additive distortion function that forces adjacent embedding changes to synchronize. Since the distortion function is purposely designed as a sum of locally supported potentials, one can use the Gibbs construction to realize the embedding in practice. The beneficial impact of synchronizing the embedding changes is linked to the fact that modern steganalysis detectors use higher-order statistics of noise residuals obtained by filters with sign-changing kernels and to the fundamental difficulty of accurately estimating the selection channel of a non-additive embedding scheme implemented with several Gibbs sweeps. Both decrease the accuracy of detectors built using rich media models, including their selection-channel-aware versions.

Random projections of residuals as an alternative to co-occurrences in steganalysis

Today, the most reliable detectors of steganography in empirical cover sources, such as digital images coming from a known source, are built using machine-learning by representing images with joint distributions (co-occurrences) of neighboring noise residual samples computed using local pixel predictors. In this paper, we propose an alternative statistical description of residuals by binning their random projections on local neighborhoods. The size and shape of the neighborhoods allow the steganalyst to further diversify the statistical description and thus improve detection accuracy, especially for highly adaptive steganography. Other key advantages of this approach include the possibility to model long-range dependencies among pixels and making use of information that was previously underutilized in the marginals of co-occurrences. Moreover, the proposed approach is much more flexible than the previously proposed spatial rich model, allowing the steganalyst to obtain a significantly better trade off between detection accuracy and feature dimensionality. We call the new image representation the Projection Spatial Rich Model (PSRM) and demonstrate its effectiveness on HUGO and WOW – two current state-of-the-art spatial-domain embedding schemes.

Improving Selection-Channel-Aware Steganalysis Features.

Currently, the best detectors of content-adaptive steganography are built as classifiers trained on examples of cover and stego images represented with rich media models (features) formed by histograms (or co-occurrences) of quantized noise residuals. Recently, it has been shown that adaptive steganography can be more accurately detected by incorporating content adaptivity within the features by accumulating the embedding change probabilities (change rates) in the histograms. However, because each noise residual depends on an entire pixel neighborhood, one should accumulate the embedding impact on the residual rather than the pixel to which the residual is formally attributed. Following this observation, in this paper we propose the expected value of the residual L1 distortion as the quantity that should be accumulated in the selectionchannel-aware version of rich models to improve the detection accuracy. This claim is substantiated experimentally on four modern content-adaptive steganographic algorithms that embed in the spatial domain. Motivation Modern content-adaptive steganography dates back to 2010 when HUGO (Highly Undetectable steGO) was introduced [22]. It incorporated syndrome-trellis codes [6] as the most innovative element that is currently used in all modern steganographic schemes operating in any domain. Such advanced coding techniques gave the steganographer control over where the embedding changes are to be executed by specifying the costs of modifying each pixel. The costs, together with the payload size, determine the probability with which a given pixel is to be modified during embedding. These probabilities, also called change rates, are recognized as the so-called selection channel. Since the costs of virtually all content-adaptive embedding techniques are not very sensitive to the embedding changes themselves [25], they are also available to the steganalyst. For simpler embedding paradigms, such as the Least Significant Bit (LSB) replacement combined with naive adaptive embedding, researchers have shown how a publicly known selection channel can be used to improve the WS detector [23]. Modern adaptive steganographic schemes for digital images [13, 19, 26, 16, 24], however, do not use LSB replacement or naive adaptive embedding, and their detection requires detectors built with machine learning. The prevailing trend is to represent images using rich media models, such as the Spatial Rich Model (SRM) [7], Projection Rich Model (PSRM) [14], and their numerous variants designed for the spatial domain [2], JPEG domain [17, 12, 15, 27], and for color images [8, 9]. Such rich models are concatenations of histograms (for projection type rich models [14] and phase-aware models [15, 12, 27]) or co-occurrences of quantized noise residuals obtained with a variety of linear and non-linear pixel predictors. In [28], the authors proposed to compute the co-occurrences in the SRM only from a fraction of pixels with the highest embedding change probability. Even though this decreased the amount of data available for steganalysis, the authors showed that the embedding algorithm WOW could be detected with a markedly better accuracy. A generalization of this approach was later proposed that utilized the statistics of all pixels by accumulating the maximum of the four pixel change rates in the co-occurrences of four neighboring residuals. This version of the SRM called maxSRM [5] improved the detection of all content-adaptive algorithms to a varying degree. The idea was, however, not extensible to spatial-domain rich features for detection of JPEG steganography [15, 12, 27] or to projection type features because the residuals depend on numerous pixels and one can no longer associate a pixel (or a DCT coefficient) change rate with a given residual sample. This paper resolves this issue by replacing the change rate with the expected value of the residual distortion as the quantity that should be accumulated in the histograms (for JPEG phase-aware features and projection type features) and in co-occurrences (for SRM). This extension is relatively straightforward for linear residuals since the relationship tying the embedding domain and the residual domain is linear. If the embedding changes are executed independently,1 one can easily compute the expected value of the embedding distortion in the residual domain analytically. A major complication, however, occurs for non-linear residuals due to the necessity to compute marginals of high-dimensional probability mass functions. This is why the emphasis of this paper is on rich representations formed from linear residuals. An extension of the idea presented in this paper to phase-aware JPEG features appears in [3]. In the next section, we include a brief overview of the SRM, PSRM, and maxSRM to prepare the ground for the third section, where we describe the quantity that 1This is true for all current steganographic schemes with the notable exception of steganography that synchronizes the selection channel [4, 20]. will be accumulated in the histograms (PSRM) and cooccurrences of quantized noise residuals (SRM) in the selection-channel-aware version of such features. Since the PSRM is extremely computationally demanding, we only work with a subset of its features that come from linear (’spam’ type) residuals of dimension 1,980. In the fourth section, we show that making this relatively compact feature space properly aware of the selection channel achieves state-of-the-art performance with the ensemble classifier. The paper is concluded in the fifth section, where we summarize the contribution and outline how the proposed idea can be executed for phase-aware JPEG features. Preliminaries: SRM, PSRM, and maxSRM In this section, we review the basics of the SRM, its projection version, the PSRM, and the selection-channelaware maxSRM. This is done in order to make the paper self-contained and easier to read. The symbols X,Y∈ {0, . . . ,255}n1×n2 will be used exclusively for two-dimensional arrays of pixel values in an n1×n2 grayscale cover and stego image, respectively. Elements of a matrix will be denoted with the corresponding lower case letter. The pair of subscripts i, j will always be used to index elements in an n1×n2 matrix. The cardinality of a finite set S will be denoted |S|. SRM Both the SRM and the PSRM extract the same set of noise residuals from the image under investigation. They differ in how they represent their statistical properties. The SRM uses four dimensional co-occurrences while the PSRM uses histograms of residual projections. A noise residual is an estimate of the image noise component obtained by subtracting from each pixel its estimate (expectation) obtained using a pixel predictor from the pixel’s immediate neighborhood. Both rich models use 45 different pixel predictors of two different types – linear and non-linear. Each linear predictor is a shift-invariant finite-impulse response filter described by a kernel matrix K(pred). The noise residual Z = (zkl) is a matrix of the same dimension as X: Z = K(pred) ?X−X , K?X. (1) In (1), the symbol ′?′ denotes the convolution with X mirror-padded so that K?X has the same dimension as X. This corresponds to the ’conv2’ Matlab command with the parameter ’same’. An example of a simple linear residual is zij = xi,j+1− xij , which is the difference between a pair of horizontally neighboring pixels. In this case, the residual kernel is K = ( −1 1 ), which means that the predictor estimates the pixel value as its horizontally adjacent pixel. This predictor is used in submodel ’spam14h’ in the SRM. All non-linear predictors in the SRM are obtained by taking the minimum or maximum of up to five residuals obtained using linear predictors. For example, one can predict pixel xij from its horizontal or vertical neighbors, obtaining thus one horizontal and one vertical residual Z(h) = (z ij ), Z (v) = (z ij ): z (h) ij = xi,j+1−xij , (2) z (v) ij = xi+1,j −xij . (3) Using these two residuals, one can compute two nonlinear ’minmax’ residuals as: z (min) ij = min{z (h) ij ,z (v) ij }, (4) z (max) ij = max{z (h) ij ,z (v) ij }. (5) The next step in forming the SRM involves quantizing Z with a quantizer Q−T,T with centroids Q−T,T = {−Tq,(−T +1)q, . . . ,T q}, where T > 0 is an integer threshold and q > 0 is a quantization step: rij ,Q−T,T (zij), ∀i, j. (6) The next step in forming the SRM feature vector involves computing a co-occurrence matrix of fourth order, C(SRM) ∈ Q−T,T , from four (horizontally and vertically) neighboring values of the quantized residual rij (6) from the entire image:2 c (SRM) d0d1d2d3 = n1,n2−3 ∑ i,j=1 [ri,j+k = dk,∀k = 0, . . . ,3], (7)

Side-informed steganography with additive distortion

Side-informed steganography is a term used for embedding secret messages while utilizing a higher quality form of the cover object called the precover. The embedding algorithm typically makes use of the quantization errors available when converting the precover to a lower quality cover object. Virtually all previously proposed side-informed steganographic schemes were limited to the case when the side-information is in the form of an uncompressed image and the embedding uses the unquantized DCT coefficients to improve the security when JPEG compressing the precover. Inspired by the side-informed (SI) UNIWARD embedding scheme, in this paper we describe a general principle for incorporating the side-information in any steganographic scheme designed to minimize embedding distortion. Further improvement in security is obtained by allowing a ternary embedding operation instead of binary and computing the costs from the unquantized cover. The usefulness of the proposed embedding paradigm is demonstrated on a wide spectrum of various information-reducing image processing operations, including image downsampling, color depth reduction, and filtering. Side-information appears to improve empirical security of existing embedding schemes by a rather large margin.

Theoretical model of the FLD ensemble classifier based on hypothesis testing theory

The FLD ensemble classifier is a widely used machine learning tool for steganalysis of digital media due to its efficiency when working with high dimensional feature sets. This paper explains how this classifier can be formulated within the framework of optimal detection by using an accurate statistical model of base learners projections and the hypothesis testing theory. A substantial advantage of this formulation is the ability to theoretically establish the test properties, including the probability of false alarm and the test power, and the flexibility to use other criteria of optimality than the conventional total probability of error. Numerical results on real images show the sharpness of the theoretically established results and the relevance of the proposed methodology.

Model Based Steganography with Precover.

It is widely recognized that steganography with sideinformation in the form of a precover at the sender enjoys significantly higher empirical security than other embedding schemes. Despite the success of side-informed steganography, current designs are purely heuristic and little has been done to develop the embedding rule from first principles. Building upon the recently proposed MiPOD steganography, in this paper we impose multivariate Gaussian model on acquisition noise and estimate its parameters from the available precover. The embedding is then designed to minimize the KL divergence between cover and stego distributions. In contrast to existing heuristic algorithms that modulate the embedding costs by 1–2|e|, where e is the rounding error, in our model-based approach the sender should modulate the steganographic Fisher information, which is a loose equivalent of embedding costs, by (1–2|e|)^2. Experiments with uncompressed and JPEG images show promise of this theoretically well-founded approach. Introduction Steganography is a privacy tool in which messages are embedded in inconspicuous cover objects to hide the very presence of the communicated secret. Digital media, such as images, video, and audio are particularly suitable cover sources because of their ubiquity and the fact that they contain random components, the acquisition noise. On the other hand, digital media files are extremely complex objects that are notoriously hard to describe with sufficiently accurate and estimable statistical models. This is the main reason for why current steganography in such empirical sources [3] lacks perfect security and heavily relies on heuristics, such as embedding “costs” and intuitive modulation factors. Similarly, practical steganalysis resorts to increasingly more complex high-dimensional descriptors (rich models) and advanced machine learning paradigms, including ensemble classifiers and deep learning. Often, a digital media object is subjected to processing and/or format conversion prior to embedding the secret. The last step in the processing pipeline is typically quantization. In side-informed steganography with precover [21], the sender makes use of the unquantized cover values during embedding to hide data in a more secure manner. The first embedding scheme of this type described in the literature is the embedding-while-dithering [14] in which the secret message was embedded by perturbing the process of color quantization and dithering when converting a true-color image to a palette format. Perturbed quantization [15] started another direction in which rounding errors of DCT coefficients during JPEG compression were used to modify the embedding algorithm. This method has been advanced through a series of papers [23, 24, 29, 20], culminating with approaches based on advanced coding techniques with a high level of empirical security [19, 18, 6]. Side-information can have many other forms. Instead of one precover, the sender may have access to the acquisition oracle (a camera) and take multiple images of the same scene. These multiple exposures can be used to estimate the acquisition noise and also incorporated during embedding. This research direction has been developed to a lesser degree compared to steganography with precover most likely due to the difficulty of acquiring the required imagery and modeling the differences between acquisitions. In a series of papers [10, 12, 11], Franz et al. proposed a method in which multiple scans of the same printed image on a flat-bed scanner were used to estimate the model of the acquisition noise at every pixel. This requires acquiring a potentially large number of scans, which makes this approach rather labor intensive. Moreover, differences in the movement of the scanner head between individual scans lead to slight spatial misalignment that complicates using this type of side-information properly. Recently, the authors of [7] showed how multiple JPEG images of the same scene can be used to infer the preferred direction of embedding changes. By working with quantized DCT coefficients instead of pixels, the embedding is less sensitive to small differences between multiple acquisitions. Despite the success of side-informed schemes, there appears to be an alarming lack of theoretical analysis that would either justify the heuristics or suggest a well-founded (and hopefully more powerful) approach. In [13], the author has shown that the precover compensates for the lack of the cover model. In particular, for a Gaussian model of acquisition noise, precover-informed rounding is more secure than embedding designed to preserve the cover model estimated from the precover image assuming the cover is “sufficiently non-stationary.” Another direction worth mentioning in this context is the bottom-up model-based approach recently proposed by Bas [2]. The author showed that a high-capacity steganographic scheme with a rather low empirical detectability can be constructed when the process of digitally developing a RAW sensor capture is sufficiently simplified. The impact of embedding is masked as an increased level of photonic noise, e.g., due to a higher ISO setting. It will likely be rather difficult, however, to extend this approach to realistic processing pipelines. Inspired by the success of the multivariate Gaussian model in steganography for digital images [25, 17, 26], in this paper we adopt the same model for the precover and then derive the embedding rule to minimize the KL divergence between cover and stego distributions. The sideinformation is used to estimate the parameters of the acquisition noise and the noise-free scene. In the next section, we review current state of the art in heuristic side-informed steganography with precover. In the following section, we introduce a formal model of image acquisition. In Section “Side-informed steganography with MVG acquisition noise”, we describe the proposed model-based embedding method, which is related to heuristic approaches in Section “Connection to heuristic schemes.” The main bulk of results from experiments on images represented in the spatial and JPEG domain appear in Section “Experiments.” In the subsequent section, we investigate whether the public part of the selection channel, the content adaptivity, can be incorporated in selection-channel-aware variants of steganalysis features to improve detection of side-informed schemes. The paper is then closed with Conclusions. The following notation is adopted for technical arguments. Matrices and vectors will be typeset in boldface, while capital letters are reserved for random variables with the corresponding lower case symbols used for their realizations. In this paper, we only work with grayscale cover images. Precover values will be denoted with xij ∈ R, while cover and stego values will be integer arrays cij and sij , 1 ≤ i ≤ n1, 1 ≤ j ≤ n2, respectively. The symbols [x], dxe, and bxc are used for rounding and rounding up and down the value of x. By N (μ,σ2), we understand Gaussian distribution with mean μ and variance σ2. The complementary cumulative distribution function of a standard normal variable (the tail probability) will be denoted Q(x) = ∫∞ x (2π)−1/2 exp ( −z2/2 ) dz. Finally, we say that f(x)≈ g(x) when limx→∞ f(x)/g(x) = 1. Prior art in side-informed steganography with precover All modern steganographic schemes, including those that use side-information, are implemented within the paradigm of distortion minimization. First, each cover element cij is assigned a “cost” ρij that measures the impact on detectability should that element be modified during embedding. The payload is then embedded while minimizing the sum of costs of all changed cover elements, ∑ cij 6=sij ρij . A steganographic scheme that embeds with the minimal expected cost changes each cover element with probability βij = exp(−λρij) 1 +exp(−λρij) , (1) if the embedding operation is constrained to be binary, and βij = exp(−λρij) 1 +2exp(−λρij) , (2) for a ternary scheme with equal costs of changing cij to cij ± 1. Syndrome-trellis codes [8] can be used to build practical embedding schemes that operate near the rate–distortion bound. For steganography designed to minimize costs (embedding distortion), a popular heuristic to incorporate a precover value xij during embedding is to modulate the costs based on the rounding error eij = cij − xij , −1/2≤ eij ≤ 1/2 [23, 29, 20, 18, 19, 6, 24]. A binary embedding scheme modulates the cost of changing cij = [xij ] to [xij ] + sign(eij) by 1−2|eij |, while prohibiting the change to [xij ]− sign(eij): ρij(sign(eij)) = (1−2|eij |)ρij (3) ρij(−sign(eij)) = Ω, (4) where ρij(u) is the cost of modifying the cover value by u∈ {−1,1}, ρij are costs of some additive embedding scheme, and Ω is a large constant. This modulation can be justified heuristically because when |eij | ≈ 1/2, a small perturbation of xij could cause cij to be rounded to the other side. Such coefficients are thus assigned a proportionally smaller cost because 1− 2|eij | ≈ 0. On the other hand, the costs are unchanged when eij ≈ 0, as it takes a larger perturbation of the precover to change the rounded value. A ternary version of this embedding strategy [6] allows modifications both ways with costs: ρij(sign(eij)) = (1−2|eij |)ρij (5) ρij(−sign(eij)) = ρij . (6) Some embedding schemes do not use costs and, instead, minimize statistical detectability. In MiPOD [25], the embedding probabilities βij are derived from their impact on the cover multivariate Gaussian model by solving the following equation for each pixel ij: βijIij = λ ln 1−2βij βij , (7) where Iij = 2/σ̂4 ij is the Fisher information with σ̂ 2 ij an estimated variance of the acquisition noise at pixel ij, and λ is a Lagrange multiplier determined by the payload size. To incorporate the side-information, the sender first converts the embedding probabilities into costs and then modulates them as in (3) or (5). This can be done b