Towards Versatility: Lossless Data Hiding in JPEG Bitstream via Table-irrelevant Code Mapping
IIEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 1
A Universal Framework to Construct aHuffman-Code-Mapping-based Reversible DataHiding Scheme for JPEG Images
Abstract —Huffman code mapping (HCM) is a recent techniquefor reversible data hiding (RDH) in JPEG images. The existingHCM-based RDH schemes cause neither file-size increment norvisual distortion for the marked JPEG image, which is thesuperiority compared to the RDH schemes that use other tech-niques, such as histogram shifting (HS). However, the embeddingcapacity achieved by the HCM-based RDH schemes is much lowerthan the HS-based RDH schemes. Moreover, the existing HCM-based schemes are only applicable to the JPEG images codedwith the default Huffman table. In this paper, we propose auniversal framework to design the HCM-based RDH scheme.Under this framework, the key issue of HCM-based schemes, i.e.,construct the optimal code mapping relationship, is converted tosolve a combinatorial optimization problem. The high embeddingcapacity can be achieved with a slight increase in the file-size ofthe marked JPEG image. In addition, the problem of applicabilityis also solved by customizing the Huffman table. As a realization,we construct a new HCM-based scheme by employing the geneticalgorithm to search the nearly optimal solution. Experimentsshow that the performance on the file-size preservation, visualquality, and computational complexity is superior to recent HS-based RDH schemes under the identical payload.
Index Terms —Reversible data hiding (RDH), JPEG, Huffmancoding mapping (HCM), genetic algorithm (GA).
I. Introduction R EVERSIBLE data hiding (RDH) is a branch of datahiding technology, which can embed additional data intocover media imperceptibly, and the cover media can be restoredlosslessly. The reversibility is necessary for many applications,e.g., medical image processing and multimedia management.In the past decade, many RDH schemes for uncompressedimages are designed. The existing RDH schemes are mainlybased on three categories techniques: 1) lossless compression[1] [2], 2) difference expansion (DE) [3] [4], and 3) histogramshifting (HS) [5]–[8]. However, the compressed images aremore widely used on the Internet, benefit from the low costfor storage or transmission. As JPEG is the most commonlyused compression image format, developing the RDH schemesfor JPEG images is required.Recently, quite some RDH schemes for JPEG images havebeen proposed. According to the modification objects, theexisting RDH schemes for JPEG images can be divided intotwo categories. The first category of schemes [9]–[20] isbased on modifying the Discrete Cosine Transform (DCT)coefficients. The second category of schemes [21]–[24] isbased on modifying the Huffman codes.Since the DCT coefficients and the pixels are all integers,most of the techniques used in the first category of RDH inJPEG images are derived from the techniques designed foruncompressed images. For instance, some early works [9] [10] are proposed to embed data by modifying the quantizationtable while modifying the DCT coefficients. In essence, thetechniques adopted in [9] [10] is the DE technique. Thelossless compression for DCT coefficients is introduced in[11], which is proposed by Fridrich et al . . In [11], additionaldata is embedded into a vacated room created by compressingthe LSBs of non-zero alternating current (AC) coefficients.The HS technique can also be extended from uncompressedimages to JPEG images. In addition, HS technique is the mostpopular technique for RDH in JPEG images since this typeof schemes can keep good visual quality and also achieve asmaller file size increments. In early HS-based RDH schemes[12] [13] for JPEG images, a problem is modifying the ACcoefficients that values of 0, which leads to a significantincrease in the file-size inevitably. To address this problem,Huang et al . [14] proposed a new HS-based RDH schemethat keeps the zero AC coefficient unchanged and sets the ACcoefficients that values of 1 and -1 as the peak points of thehistogram. The variation of all non-zero AC coefficients is atmost 1 so the visual quality is well preserved. Later, someimproved works [15]–[17] based on Huang et al . ’s schemeare proposed, which aim to further improve the capabilitiesof visual quality and file-size preservation. In 2020, He et al . [18] established a negative influence model to determine thefrequency selection and defined a weighting factor to adjust theoptimization objective of the provided scheme. Yin et al . [19]improved the prior HS-based methods by designing a multiple-objective optimization algorithm. Li et al . [20] proposed ascheme that constructs a new two-dimensional histogram forJPEG images and achieved well performance.For the second category of RDH schemes, the used tech-nique is a new reversible embedding technique based onthe characteristic of the JPEG bitstream, which is referredto as Huffman code mapping (HCM). Qian and Zhang [21]proposed a file-size preserving HCM-based scheme. In [21],the used code is assigned with multiple unused codes withthe same code lengths to construct a code mapping set. TheJPEG file header is also modified to keep the visual qualityunchanged. After that, some works [22]–[24] are proposed toimprove the embedding capacity. In Hu et al . ’s scheme [22],the capacity is improved by searching the optimal combinationaccording to the frequency of codes. Then, Qiu et al . [23]proposed an alternative embedding method by reordering thecodes to improve the capacity further. Later, Zhang et al . [24]improved the scheme further by enlarging the search space.Compared to the HS-based RDH schemes, the explicit ad-vantages of HCM-based RDH schemes are no visual distortionand no increase in the file-size of the marked JPEG image. a r X i v : . [ c s . MM ] J un EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 2
However, two problems existing in the HCM-based RDHschemes cannot be ignored. One is the quite lower embeddingcapacity than HS-based RDH schemes. The other one is theprevious HCM-based schemes can only be applied to theJPEG images coded with the default Huffman table. Becausethe unused codes contained in the default Huffman table arenecessary to construct the mapping relationship for the HCMtechnique. When there are no unused codes in a Huffman table,the previous HCM technique is unavailable.Is it possible to construct an HCM-based RDH scheme thatcan be applied to the JPEG images coded with any Huffmantable and achieve high embedding capacity while keepingvisual quality with no distortion? In this paper, we positivelyanswer this question by designing a universal framework toconstruct the HCM-based RDH scheme and proposing a newHCM-based RDH scheme as an example. The high embeddingcapacity can be achieved by mapping multiple codes to thehigh-frequency symbols in a Huffman table. Surely, the benefitof high embedding capacity cannot come for free. The cost isa slight file-size increment. In Section VI, we compare the file-size increment at identical payload with the recent HS-basedschemes and find that the file-size increment caused by ourproposed scheme is lower than HS-based schemes. The maincontributions of this paper are summarized as follows:1) A new code mapping strategy is adopted to convert theconstruction of the optimal code mapping relationshipinto a combinatorial optimization problem.2) A universal framework to construct an HCM-basedscheme is formally proposed. Under this framework, thekey issue is to solve the optimization problem withdifferent given constraints.3) As a realization of the proposed framework, a new HCM-based RDH scheme for JPEG images is provided by em-ploying the genetic algorithm (GA) with some problem-oriented designs to solve the optimization problem.The remainder of this paper is organized as follows. Firstly,the encoding of AC coefficients in JPEG bitstream and theHCM-based RDH technique is introduced in Section II. Sec-tion III provided the motivation of this paper. Then, the newcode mapping strategy and the universal framework based onit are presented in Section IV. Later, the proposed scheme isdescribed in detail in Section V. In Section VI, experimentalresults and analysis are included. Finally, the conclusions aredrawn in Section VII. II. BackgroundIn this section, the encoding of AC coefficients in the JPEGbitstream is first introduced briefly. Then, the current HCM-based RDH schemes are reviewed.
A. Encoding of AC coefficients in JPEG Bitstream
In the entropy coding phase of JPEG standard, the quantizedAC coefficients are first compressed by run length encoding(RLE) into the form of ( run / size , v alue ) . v alue is the am-plitude of the next nonzero AC coefficient, which is coded byvariable length integer (VLI) encoding and the generated codesare referred to as appended bits. run ( ≤ run ≤ ) represents the number of zero AC coefficients before the next nonzeroAC coefficient. size ( ≤ size ≤ ) denotes the code lengthneeded to represent v alue . The symbol run / size , which isreferred to as to Run/Size Value (RSV) throughout the paper, isencoded with Huffman coding for further compression. Finally,the entropy coded data and other information used to encodeand decode are stored in the form of bitstream. FF C4 00 B5 10 00 02 01 03 03 02 04
03 05 05 04 04 00 00 01 7D 01 02 03
00 04 11 05 12 21 31 41 06 13 51 61 07 22 71 14 32 81 91 A1 08 23 42 B1
C1 15 52 D1 F0 24 33 62 72 82 09 0A
16 17 18 19 1A 25 26 27 28 29 2A 34
35 36 37 38 39 3A 43 44 45 46 47 48
49 4A 53 54 55 56 57 58 59 5A 63 64 65 66 67 68 69 6A 73 74 75 76 77 78
79 7A 83 84 85 86 87 88 89 8A 92 93
94 95 96 97 98 99 9A A2 A3 A4 A5 A6 A7 A8 A9 AA B2 B3 B4 B5 B6 B7 B8 B9
BA C2 C3 C4 C5 C6 C7 C8 C9 CA D2 D3
D4 D5 D6 D7 D8 D9 DA E1 E2 E3 E4 E5
E6 E7 E8 E9 EA F1 F2 F3 F4 F5 F6 F7
F8 F9 FA
Marker Length Type Code Lengths (List BITS)Symbols/Values (List HUFFVAL)
Fig. 1. The Structure of the DHT Segment for AC coefficients.
In JPEG bitstream, only the RSVs and the lengths ofthe codes assigned to each RSV are recorded in the defineHuffman table (DHT) segment of the JPEG header, insteadof the Huffman codes. The DHT segment includes two lists,BITS and HUFFVAL, which is provided in Fig. 1. As shown inFig. 1, is the marker represents the DHT segment and is the segment length except for the marker .The list BITS contains the number of codes of differentlengths, which occupies 16 bytes totally, i.e., the numberof codes of length i occupies one byte. The list HUFFVALcontains the symbols/RSVs, which is the hexadecimal form ofthe RSVs associated with each code of length i . According tothe two lists, the Huffman table can be constructed. B. Overview of HCM-based RDH Schemes
The previous HCM-based schemes is based on the exploita-tion of redundancy, which exists in the JPEG bitstream codedwith the default Huffman table. The default Huffman tablefor AC coefficients is provided in the JPEG standard [25],which includes 162 RSVs and corresponding codes. However,for a specific JPEG image, not all the codes appear in theentropy coded data. The symbols/RSVs corresponding to theunused codes have no need to be recorded in HUFFVALbecause their frequencies are equal to zero. These symbols areredundant and can be exploited to embed data. The main ideais to replace several unused symbols with one used symbol inHUFFVAL to construct a mapping set between a used codeand several unused codes. Each code in a mapping set carriesdifferent data. Since the corresponding symbols are the same
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 3
The code length T he nu m be r o f c ode s T he f r equen cy o f c ode s The used codesThe unused codesFrequency (a) Lena (QF = 70)
The code length T he nu m be r o f c ode s T he f r equen cy o f c ode s The used codesThe unused codesFrequency (b) Baboon (QF = 70)Fig. 2. The statistic results of the Huffman codes on the image "Lena" and "Baboon" with QF = 70. for all of the codes in a mapping set, the decoded result keepsunchanged after replacing the original code with one of thecodes in the mapping set. In addition, the appended bits andcodes for the DC coefficients are all unchanged during dataembedding. Thus, the HCM-based scheme will not impact theimage visual quality.To keep the file-size unchanged after data embedding, Qianand Zhang [21] proposed to group all the codes according tothe code length. Then, the codes in a mapping set are all withthe same code lengths. For ease of description throughout thepaper, this specific mapping strategy is referred to as grouped-based code mapping (GCM) strategy. The follow-up works[22]–[24] are mainly proposed to improve the embeddingcapacity by exploiting the statistic information of codes. Theadopted mapping strategy is still the GCM strategy proposedby Qian and Zhang [21].III. MotivationAlthough previous HCM-based RDH schemes have satisfy-ing performances on file-size preservation and visual quality,two existing problems cannot be ignored, which are analyzedas follows.
1) Low Embedding Capacity:
The embedding capacityachieved by the file-size preserving RDH schemes is ratherlimited compared to HS-based RDH schemes. This is becausethe limitation of GCM strategy. To demonstrate the statement,we present the calculation of embedding capacity C . Let { f i , , . . . , f i , u i } denote the frequency of used RSVs in the i -thgroup, then the embedding capacity EC can be calculated by EC = (cid:213) i = (cid:213) u i j = f i , j × log k i , j , (1)where u i is the number of used RSVs in i -th group and k i , j is the number of codes assigned for the j -th used RSVin i -th group. The k i , j codes includes k i , j − unused codesand one used code. According to Eq. (1), the embeddingcapacity depends on the frequency of used RSVs and thenumber of unused codes. However, for a JPEG image codedwith the default Huffman table, the frequencies of used theRSVs assigned with short coeds are much larger than the onesassigned with the long codes. Take the typical images "Lena"and "Baboon" as examples, the statistic results are shown inFig. 2. As depicted in Fig. 2, the frequency of RSV decreases with the code length increasing and most of the unused codesare with longer lengths. The numbers of unused codes in thefirst few groups are often near or equal to zero. That meansthe embedding capacity is mainly contributed by the latter fewgroups. Thus, the embedding capacity achieved with the GCMstrategy is low.
2) Weak Applicability:
The previous HCM-based schemescan only be applied to the JPEG bitstream coded with thedefault Huffman table, which exists unused codes. Becausethe GCM strategy is unavailable when no unused codes existin the Huffman table. This Huffman table is called as the optimized Huffman table , which only includes the used RSVsand corresponding codes.To address above two existing problems, we propose anadaptive code mapping (ACM) strategy to guide the construc-tion of the mapping relationship. The idea of ACM strategyis that all of the used RSVs are possible to be assigned withmultiple unused codes. With the ACM strategy, the RSVs areno need to be grouped according to the code length, which isdifferent from the GCM strategy. The number of codes mappedto each used RSV is unfixed and can be adjusted according tothe given constraint. Therefore, the high embedding capacitycan be achieved by assigning the high-frequency RSV withmultiple codes to construct a mapping set.In addition, when adopting the ACM strategy to constructthe mapping relationship, whether the JPEG encoder usesthe default Huffman table or the optimized Huffman table,the Huffman table in the marked JPEG bitstream is onlycustomized according to the frequency of the used RSVs. Theunused codes are added according to the constructed codemapping relationship instead of the existing unused codes inthe default Huffman table. Therefore, the HCM-based schemedesigned based on the ACM strategy is applicable to theJPEG image coded with any Huffman table. That means theapplicability is improved compared to using the GCM strategy.IV. Universal Framework to Construct the HCM-basedRDH SchemesIn this section, based on the proposed ACM strategy, a uni-versal framework to construct the HCM-based RDH schemesis proposed. According to the universal framework, the taskis to design an optimization algorithm to obtain the optimal
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 4 code mapping relationship. Thus, the optimization problemsand corresponding evaluation method will be introduced.
A. The Framework
The framework consists of three key modules, which isdescribed as follows:
1) Bitstream Parsing:
In first, the original Huffman tableis constructed by parsing the two lists BITS amd HUFFVALin the DHT segment. With the original Huffman table, all theHuffman codes and appended bits in the entropy coded dataare extracted. Then the frequency of used RSVs is counted.
2) Optimal Code Mapping:
According to a given con-straint, the used RSVs are assigned with multiple codesadaptively to construct the optimal code mapping relationship.The optimal code mapping relationship is the one with thebest performance under the given constraint. We use a N -dimensional vector to represent the number of codes mappedto different used RSV, such as m = ( m , m , . . . , m N ) . m is called as the mapping vector , where N is the number ofused RSVs in the cover bitstream and m i is the number ofcodes mapped to the i -th used RSV. The mapping vector m can be used to represent the mapping relationship. Then,the construction of optimal code mapping can be seen as a combinatorial optimization problem, i.e., search the optimalmapping vector under the given constraint. A simple idea isto exhaustive search from the entire solution space, i.e., allthe possible mapping vectors and find the one with the bestperformance. However, it is impractical to evaluate all of themapping vectors because the solution space is massive. Thus,design a fast and effective optimization algorithm is vital tosearch for the optimal solution.
3) Embedding:
With the optimal mapping relationship, thecustomized Huffman table is constructed by generating thenew lists, i.e., BITS and HUFFVAL. The additional datais then embedded by replacing the original code with theone of the codes in a mapping set according to the datato embedded. Other codes are all replaced according to thecustomized Huffman table. By merging the new file header andnew entropy coded data, the marked JPEG bitstream is finallyobtained. The marked JPEG bitstream can still be decodeddirectly by the popular decoders.
Cover BitstreamMarked Bitstream Additional Data Optimal Code MappingBitstream ParsingEmbedding
Optimization Algorithm
Fig. 3. The universal framework for HCM-based RDH.
The framework is shown in Fig. 3. To construct an HCM-based RDH scheme according to the proposed framework, thekey issue is to solve the optimization problem. After designingan effective optimization algorithm to find the optimal solutionfrom the solution space, one can get a new HCM-based RDHscheme for JPEG images.
B. Optimization Problem under Different Constraints
For the marked JPEG image generated by the HCM-basedRDH scheme, since the decoded image content is lossless, theoptimization of visual distortion is not required. Therefore,the given constraints are generally divided into two categories:payload and file-size increment. Then, the optimization prob-lems can be divided into two types: file-size-limited optimiza-tion (FLO) problem and payload-limited optimization (PLO)problem. The FLO problem is to maximize the embeddingcapacity in the condition that the actual file-size incrementis less than or equal to a given increment. Conversely, thePLO problem is to minimize the file-size increment whenembedding the data with a given payload.In fact, optimizing the PLO problem is more valuable thanoptimizing the FLO problem. In most practical applicationscenarios, the additional data is first given to perform theembedding process so the given payload is determined. Thus,it is a PLO problem. For the HS-based RDH schemes, the PLOproblem is one of the problems to be optimized. Limited tothe embedding capacity, for the previous HCM-based schemes,only the FLO problem can be optimized and the optimizationis only limited to when the file-size increment is 0. This canexplain why the HCM-based RDH schemes are not practical.Based on the proposed universal framework to HCM-basedschemes, both of the two problems can be formulated andoptimized. Except for the given constraint, two constraintsaccording to the rule of Huffman coding in JPEG standard[25] also need to be considered to define the optimizationproblems:1) For each used RSV, at least one code should be mappedto it to ensure the correct encoding, i.e., each m i in themapping vector m = ( m , m , . . . , m N ) must be greaterthan 1.2) The max number of codes in a customized Huffman tablecan up to 256 according to the JPEG standard [25]. Thus,the sum of m i is from N to 256. No mapping sets existin the customized Huffman table when the sum of m i isequal to N .Then, the two types of problems are defined as follows:
1) FLO problem:
For this problem, the given constraint is aspecified file-size increment G I and the optimization objectiveis to maximize the embedding capacity. Therefore, the FLOproblem is formulated as:max EC ( m ) s . t . F I ( m ) ≤ G I (cid:205) Ni = m i ≤ m i ≥ . (2)where F I ( m ) and EC ( m ) are the file-size increment andembedding capacity for the mapping vector m respectively. m i is the i -th element in m . According to the definition of FLOproblem, we can regard the previous HCM-based schemes asthe optimization of an FLO problem, i.e., to maximize theembedding capacity under the given file-size increment is 0.
2) PLO problem:
Opposite to the FLO problem, for PLOproblem„ the given constraint is a specified payload G P EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 5 and the optimization objective is to minimize the file-sizeincrement. Therefore, the PLO problem is formulated as:min
F I ( m ) s . t . EC ( m ) ≥ G P (cid:205) Ni = m i ≤ m i ≥ , (3) C. Evaluation of the Mapping Vector
To optimize the two problems, in this section, we introducethe calculation of the embedding capacity and file-size incre-ment to evaluate the mapping vector m .
1) Embedding Capacity:
Let F used = { f ui | ≤ i ≤ N } denote the frequency set of used RSVs in the cover bitstream,the embedding capacity can be calculated directly by EC ( m ) = (cid:213) Ni = f ui × (cid:4) log m i (cid:5) , (4)where (cid:98)·(cid:99) is the f loor function.
2) File-size Increment:
The increment (in bits) in the file-size of the marked bitstream mainly consists of two parts: oneis from the DHT segment and another one is from the entropycoded data. Noted that during the bitstream generating, the filesize increment caused by byte alignment and zero-byte paddingis not included since the increment cannot be predicted inadvance and it is so small that it can be negligible.The increment from the DHT segment
F I dht is caused bythe extra symbols in the list HUFFVAL to construct the codemapping relationship. One symbol occupies one byte (8 bits)of space in bitstream. We denote the number of symbols inthe original Huffman table and the customized Huffman tableby N ori and N cus respectively. N cus can be expressed by N cus = (cid:213) Ni = m i . (5)Then, the increment from DHT segment F I dht can be repre-sented by
F I dht = × ( N cus − N ori ) . (6)The increment from the entropy coded data F I ecs is causedby code replacing when using the customized Huffman tableto generate the marked bitstream. Therefore,
F I ecs is equal tothe difference of the file-sizes of the entropy coded data inthe cover bitstream and the entropy coded data in the markedbitstream. For the entropy coded data, only the Huffman codesfor AC coefficients will be modified before and after codereplacing. We denote the file-sizes of all the Huffman codesfor AC coefficients in cover bitstream and marked bitstreamby H cover and H marked . To calculate H cover , the code lengthfor each code is required, which can be parsed from the listBITS in the cover bitstream. Let L cover = { l ∗ i | ≤ i ≤ } denote the number set of codes in different lengths in thecover bitstream and F cover = { f ∗ , f ∗ , . . . , f ∗ N ori } denote thefrequency of all RSVs. l ∗ i is the number of codes of length i and the sum of l ∗ i is equal to N ori . For example, the numberof codes of length 3 and 4 are 1 and 3 respectively in Fig. 1. The code length for each code in the cover bitstream S cover = { s ∗ i | ≤ i ≤ N ori } can be represented as S cover = , . . . , (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) l ∗ , , . . . , (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) l ∗ , . . . , , . . . , (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) l ∗ . (7)Then, the file-size of the Huffman codes in the cover bitstream H cover can be calculated by H cover = (cid:213) N ori i = f ∗ i × s ∗ i . (8)However, the file-size of the Huffman codes in the markedbitstream H marked cannot be calculated directly. Because inthe marked bitstream, the number of the codes mapped to allthe used RSVs is undetermined and it is possible to exceed162. That means the list BITS in the marked bitstream needsto be customized. The customization of the list BITS shouldbe based on the frequency of code/symbol instead of thefrequency of used RSV. Because the frequency of symbol/codeis not equivalent to the frequency of the RSV when themapping sets exist. In the customized Huffman table, the i -thused RSV is assigned with m i codes to construct a mappingset. The sum of the frequencies of the m i codes is equal tothe frequency of the i -th used RSV. However, unless the dataembedding process is completed, the frequencies of the m i codes cannot be determined, thereby further generating thelist BITS. Paradoxically, the data embedding process can onlybe performed by first customizing the list BITS. To deal withthis, we propose to use the estimated frequency instead of theactual frequency after embedding to customize the list BITS.With the assumption that the "0" and "1" in the additional dataare distributed equally, the estimated frequencies of the codesmapped to the i -th used RSV f (cid:48) i are the same, which can berepresented by f (cid:48) i = f ui m i . (9)It is noted that the estimated frequencies are only involvedin the customization of the list BITS so f (cid:48) i does not have tobe an integer. Then the estimated frequency set F est can berepresented by F est = f (cid:48) , . . . , f (cid:48) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) m , f (cid:48) , . . . , f (cid:48) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) m , . . . , f (cid:48) N , . . . , f (cid:48) N (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) m N . (10)Take the estimated frequency set F est as input, the list BITSis customized according to the Section K.2 in JPEG standard[25]. To maximize the compression, the high-frequency RSVsshould be assigned short codes, which is in accord with therule of Huffman coding. Therefore, we sort the estimatedfrequency set F est in descending order and denote f (cid:48)(cid:48) i asthe i -th estimated frequency after sorting. In accordance withthe customized list BITS, the number set of codes in differ-ent lengths in the marked bitstream can be represented by L marked = { l (cid:48) i | ≤ i ≤ } . Then, the code length for eachcode in the marked bitstream S marked = { s (cid:48) i | ≤ i ≤ N cus } EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 6 can be expressed as S marked = , . . . , (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) l (cid:48) , , . . . , (cid:124) (cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32) (cid:125) l (cid:48) , . . . , , . . . , (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) l (cid:48) . (11)Therefore, the file-size of the Huffman codes in the markedbitstream H marked can be estimated as following H marked = (cid:213) N cus i = f (cid:48)(cid:48) i × s (cid:48) i . (12)Then, the increment from the entropy coded data F I ecs canbe represented by
F I ecs = H marked − H cover . (13)Finally, the total file size for the mapping vector m can becalculated by F I ( m ) = F I dht + F I ecs = F I dht + H marked − H cover = × ( N cus − N ori ) + (cid:213) N cus i = f (cid:48)(cid:48) i × s (cid:48) i − (cid:213) N ori i = f ∗ i × s ∗ i = (cid:213) N cus i = ( f (cid:48)(cid:48) i × s (cid:48) i + ) − (cid:213) N ori i = ( f ∗ i × s ∗ i + ) . (14)V. ÂăExample:ÂăAÂăNewÂăHCM-basedRDHÂăSchemeÂăConstructedÂăUnderÂăTheÂăPro-posedÂăFrameworkIn this section, a new HCM-based RDH scheme is presentedas an example of the construction under the proposed frame-work. In this scheme, an optimization algorithm to automati-cally search for the optimal solution is provided by adoptingthe GA, with some problem-oriented designs to improve searchaccuracy and convergence speed. Then, the details on dataembedding and extraction are presented. A. The Framework of the Proposed GA
Algorithm 1 presents the framework of proposed GA algo-rithm. In this algorithm, K individuals are random generatedto initialize the parent population P . Then, the fitness of eachindividual in P is computed. To complete the reproduction ofnew population, the GA operations, i.e., selection, crossoverand mutation, are performed to generate the new individuals.The reproduction of new population terminates when themax generation number G max is reached. In the followingsubsections, some problem-oriented designs in the proposedGA algorithm are introduced.
1) Encoding for Individual:
To solve the optimization prob-lems using GA, all the tunable parameters in a individual,i.e., the mapping vector m = ( m , m , . . . , m N ) should berepresented with binary encoding. We limit the value of each m i to be the one in { , , , } to reduce the solution space.Consider the practical applications, that is enough. This limitalso can be adjusted. Then each element in { , , , } can berepresented by two bits. Such as, the codes "00","01","10",and "11" are used to represent 1, 2, 4, 8 respectively. Thus, an Algorithm 1:
Framework of the Proposed GA
Input: population size K max generation number G max crossover rate r c mutation rate r m Output: best individual p elite Generate initial parent population of K individuals P = { p , p , . . . , p K } ; for i = to G max do Compute the fitness of each p i ∈ P ; Find the best individual and define as p elite ; Insert p elite in the new population P (cid:48) ; Select K − P for reproduction; Perform the crossover operation based on r c ; Perform the mutation operation based on r m ; Insert the new K − P (cid:48) ; P ← P (cid:48) ; end return p elite ;individual is composed of a sequence with 2 N bits. Such as,the t -th individual p t in the population can be represented by p t = { a i a i | a i , a i ∈ { , } and i ∈ [ , N ]} . (15)where the pair a i a i in an individual represents the value of m i . For example, a a =
10 means the value of m is 4.
2) GA Operations:
In the reproduction process of thepopulation, we adopt the elite preservation strategy. That is, forthe best individual, we insert it in the new population directlyinstead of modifying it. For the remaining individuals, weuse the roulette wheel selection to choose better individuals.After crossover and mutation steps, the generated K −
3) Fitness Computation:
To evaluate the population, thecomputation of fitness value should be adjusted according todifferent problems. For the PLO problem, the best individualis the one with the minimal file-size increment, which iscalculated according to Eq. (14). However, the roulette wheelselection tends to choose the individual with higher fitnessvalue. Thus, we adjust the fitness value for the t -th individualas follows: f itness ( t ) = F I max − F I t . (16)where F I max is the maximal file-size increment in one gen-eration and
F I t is the file-size increment caused by the t -thindividual calculated by Eq. (14). For the individual that doesnot satisfy the payload constraint, the fitness value is assignedto 0 as a penalty.To the opposite, for the FLO problem, the optimizationobjective is the embedding capacity. Therefore, the fitnessis equal to the capacity calculated by Eq. (4). The bestindividual is the one with the maximal embedding capacity.For the individual that does not satisfy the file-size incrementconstraint, the fitness value is assigned to 0 as a penalty. EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 7
FF C4 00 46 10 00 02 01 03 02 04
04 03 06 04 05 04 01 03 02 07 01 02 03 00 04 11 12 21 05 13 31 41
22 51 61 71 14 81 91 06 23 32 A1 B1 C1 15 42 D1 F0 24 33 52 62 E1 43 72 82 F1 92 53 64 A2 16 25 34
44 35 C2 54 63 D2
FF C4 00 49 10 00 02 01 03 01 05
05 05 06 04 05 04 01 03 02 07 01 02 03 11 04 00 21 31 12 12 41 13
51 05 05 22 71 81 61 61 91 14 A1 32 B1 23 C1 06 D1 42 F0 15 52 E1 33 62 24 F1 72 82 43 92 A2 53 64
16 25 34 44 C2 35 54 63 D2
Increment: (49 H – H ) = 3 Bytes = 24 bits.Mapping Set {"110111","111000"}Mapping Set {"1110111","1111000"}Mapping Set {"11110111","11111000"} DHT in the Cover Bitstream DHT in the Marked Bitstream
Constructed Mapping Relationship
Fig. 4. An example of the modification of the DHT Segment. Compared to the prior HCM-based schemes, the modification on the DHT segment not onlyincludes the symbols in list HUFFVAL, but also the length and the list BITS.
According to the proposed GA, a new HCM-based schemeis constructed under the universal framework. The imple-mentation detail on data embedding and extraction will beintroduced in the next section.
B. Implementation Details of Data Embedding & Extraction
After the optimal mapping vector determined, the optimalcode mapping relationship can be constructed by modifyingthe DHT segment. An example of DHT modification is shownin Fig. 4. The DHT segment in the upper left in Fig. 4 isderived from the image "Baboon" at the quality factor (QF) of70. The DHT segment in the upper right is modified accordingto the optimal mapping vector produced by our proposedGA under the given payload G P is 5000. Compared to theDHT segment in the cover bitstream, the file-size of the DHTsegment in the marked bitstream increased 3 bytes (24 bits),which caused by the construction of the mapping relationship.The mapping relationship consists of three mapping sets. Foreach mapping set in Fig. 4, two codes are mapped to oneRSV. Therefore three extra symbols have to added in thelist HUFFVAL to construct the mapping relationship. Thefrequencies of RSV "1/2", "0/5", and "6/1" are 2759, 1501,and 748 respectively. Therefore, the embedding capacity is thesum of the three frequencies, i.e., 5008, which is larger thanthe given payload 5000.The detailed procedures of data embedding is describedbelow.1) Construct the original Huffman table for AC coefficientsby parsing the DHT segment from the cover JPEGbitstream.2) Count all the used RSVs by parsing the entropy codeddata according to the Huffman table.3) Find the near optimal mapping vector m ∗ accordingto given objective using the proposed GA described inSection V-A.4) Construct the mapping relationship by modifying theDHT segment in accordance with the mapping vector m ∗ . 5) Embed the addition data in the entropy coded data byreplacing the original codes with one of the codes in amapping set.6) Merge the modified JPEG header and the modified en-tropy coded data and then the marked JPEG bitstream isgenerated.In the data extraction process, the restored Huffman tablehas need to be the same with the original Huffman table. Weuse the optimized Huffman table to synthesize the restoredJPEG bitstream. Since the RSV distribution is unchanged afterrestoring, the restored JPEG bitstream is equivalent to thecover JPEG bitstream. The detailed data extraction steps isdescribed below.1) Construct the customized Huffman table for AC coef-ficients by parsing the DHT segment from the markedJPEG bitstream.2) Parse the mapping relationship according to the cus-tomized Huffman table.3) Extract the embedded data from the entropy coded dataaccording the mapping relationship.4) Synthesize a new JPEG bitstream using the optimizedHuffman table.VI. Experimental Results and Analysis A. Datasets
The images tested in our experiments are selected fromtwo commonly used image databases, which are depicted asfollows:
1) USC-SIPI [26]:
The USC-SIPI image database is acollection of digitized images. In our follow-up experiments,four classical 512 ×
512 images from the USC-SIPI imagedatabase are selected, namely, "Lena", "Elaine", "Baboon",and "Boat". The four selected images can be classified intotwo categories. For the first two images, "Lena" and "Elaine",more low-frequency details and smooth areas are contained.On the other hand, more high-frequency details and texture
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 8 areas are contained in the second two images, "Baboon" and"Boat".
2) BOSSbase 1.01 [27]:
To further investigate the suitabil-ity of the HCM-based scheme constructed by our proposedframework, the popular image database, BOSSbase 1.01, isemployed, which consists of 10000 512 ×
512 grayscaleimages in the PGM format. In our follow-up experiments,200 images from the BOSSbase image database are randomlyselected.In addition, we convert all the selected images into grayscaleJPEG images, since the images in the two databases areuncompressed images. To convert the image from the un-compressed format, TIFF or PGM, into the JPEG format, the imwrite function is used in the software MATLAB. All theJPEG images generated by the imwrite function is codedwith the default Huffman table. To test the suitability of ourproposed scheme, the JPEG images coded with the optimizedHuffman table are also regarded as the cover images in ourexperiments. The JPEG images coded with the optimizedHuffman table are generated using the jpeg_read functionand jpeg_write function proposed in the JPEG toolbox [28].
B. Evaluation Metrics
For different types of schemes, the evaluation metrics aredifferent, which are as follows:
1) For HCM-based schemes:
The evaluation metric is theembedding capacity under the zero file-size increment sincethe prior HCM-based schemes are all the file-size preservingschemes.
2) For HS-based schemes:
The evaluation metrics includefile-size preservation, visual quality, and computational com-plexity. The file-size preservation and the computational com-plexity are evaluated using the file-size increment and therunning time. For easy comparison, we use the mean squareerror (MSE) between the cover JPEG image and the markedJPEG image instead of the peak signal-to-noise ratio (PSNR)to evaluate the visual quality. The MSE is represented as
MSE = h × w (cid:213) hi = (cid:213) wj = (cid:16) X i , j − X (cid:48) i , j (cid:17) (17)where h and w are the height and width of cover JPEG image. X i , j and X (cid:48) i , j are the ( i , j ) -th pixel in cover JPEG image andmarked JPEG image respectively. According to Eq. (17), thesmaller MSE stands for the better visual quality. When MSEis equal to zero means that the visual quality of the markedJPEG image is no distortion.In general, the MSE, the file-size increment, and the runningtime are evaluated under different payloads. C. Baselines and Experimental Setup
For comparison, two types of recent schemes, i.e., HS-basedschemes and HCM-based schemes, are selected as baselines.The selected HCM-based schemes are proposed by including:Qian and Zhang [21], Hu et al . [22], and Qiu et al . [23]. Theselected HS-based schemes are proposed by including: Hou et al . [17], He et al . [18], Li et al . [20], and Yin et al . [19].We have performed the experiments for all the baselinesand the proposed scheme on a PC with a 16 GB RAM and an AMD Ryzen 7 2700 processor with 8 cores. The PC isrunning with the windows 10 system. The program codes forthe schemes proposed by Qian and Zhang [21], Hu et al . [22],Qiu et al . [23], He et al . [18], Li et al . [20], Yin et al . [19],and the proposed scheme are implemented by ourselves usingthe software MATLAB and the MATLAB version is R2019b.The program code for the scheme proposed by Hou et al . [17]is downloaded from http://home.ustc.edu.cn/ ∼ houdd.It is noted that, for prior HCM-based schemes [21]–[23], thecover JPEG images must be coded by the default Huffman ta-ble. Because the prior HCM-based schemes cannot be appliedto the JPEG images coded by the optimized Huffman table.For a fair comparison, the tested images are all coded by thedefault Huffman table when comparing with the HCM-basedschemes.For the scheme proposed by He et al . [18], the weightingfactor α of the negative influence model are set to 0 and 1. α = α = et al . [19], the coverJPEG images can only be coded by the default Huffman table.Therefore, when comparing with the HS-based schemes, onlyYin et al . ’s scheme [19] is not involved in the evaluation forthe performance on the JPEG images coded by the optimizedHuffman table.In this paper, the parameters of the GA in the proposedscheme are determined experimentally, e.g., population size,max generation number, crossover rate, and mutation rate,which is shown in Table. I. TABLE IParameters’ Setting for proposed GA.
Parameters ValuesPopulation size K G max r c r m D. Comparison with HCM-based Schemes
The comparison of embedding capacity under zero file-sizeincrement for the four test images from USC-SIPI database isshown in Table. II. The QFs of test JPEG images are set to10 from 90. It can be observed from Table. II, the embed-ding capacities achieved by our proposed scheme are almostthe highest under different QFs. Compared to [21]–[23], theaverage capacity of the proposed scheme increases 329.91%-1967.24%, 221.59%-1225.81%, and 184.37%-626.06%, re-spectively. The reason for the significant improvement com-pared with the prior HCM-based schemes is the adoption ofthe customized Huffman table, which is generated accordingto the frequency of used RSVs of a specific JPEG image. Theredundancy is better exploited by our proposed scheme thanthe previous schemes.
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 9
TABLE IIComparison of embedding capacity (bits) under the constraint of zero file-size increment across QFs from 10 to 90 (for the four test images fromthe USC-SIPI database)
Image Scheme Quality Factor Average Increase(%)
10 20 30 40 50 60 70 80 90Elaine Qian and Zhang [21] 116 142 177 328 402 218 272 576 1002 359 1967.24Hu et al . [22] 341 264 200 414 553 326 409 788 1746 560 1225.81Qiu et al . [23] 1036 942 842 933 974 763 781 997 1937 1023 626.06Proposed Lena Qian and Zhang [21] 261 210 253 292 365 183 208 249 298 258 1463.91Hu et al . [22] 564 326 262 297 370 198 289 352 593 361 1015.56Qiu et al . [23] 1253 1004 916 939 996 766 759 695 742 897 349.41Proposed Baboon Qian and Zhang [21] 4966 2269 1276 1587 1083 1006 1087 615 342 1581 329.91Hu et al . [22] 6596 3475 1760 1792 1274 1312 1363 760 692 2114 221.59Qiu et al . [23] 7168 3767 1966 2079 1483 1478 1627 1034 912 2390 184.37Proposed Boat Qian and Zhang [21] 484 388 522 481 522 714 370 510 978 552 748.66Hu et al . [22] 950 551 691 572 689 811 687 1046 1989 887 428.05Qiu et al . [23] 1633 1185 1242 1020 1079 1139 958 1161 2212 1292 262.63Proposed To demonstrate the suitability of the proposed scheme, the200 images from the BOSSbase image database are also betested. The average embedding capacities with QF from 10 to90 are illustrated in Fig. 5. It can be observed from Fig. 5that the average embedding capacity of proposed scheme issuperior than previous HCM-based schemes with any QFs.The capacity decreases with the QF increasing because ofthe redundancy caused by using the default Huffman tabledecreases.
10 20 30 40 50 60 70 80 90
Quality factor E m bedd i ng c apa c i t y ( b i t s ) Qian and ZhangHu et al.Qiu et al.Proposed
Fig. 5. Average embedding capacity under zero file-size increment of 200images from the BOSSbase 1.01 image database. The cover JPEG images arecoded with the default Huffman table.
E. Comparison with HS-based Schemes
Here, the file-size increment, the MSE and, running timeare compared between the HS-based RDH schemes and theproposed scheme. In the first, different payloads are set withdifferent QFs for the four test images from the USC-SIPIdatabase. 2000, 3000, 4000, and 5000 bits of additional dataare embedded into the test JPEG images with QF = 30. 3000,5000, 7000, and 9000 bits of additional data are embedded intothe test images with QF = 50. 4000, 7000, 10000, and 13000 bits of additional data are embedded into the test images withQF = 70. 5000, 9000, 13000, and 17000 bits of additional dataare embedded into the test images with QF = 90.
1) File-size Increment:
The file size increments betweenthe cover JPEG images coded with the default Huffman tableand the optimized Huffman table are different so they are allevaluated in this experiment.For the cover JPEG images coded with the default Huffmantable, the comparison of the file size increments of the pro-posed scheme and the previous HS-based schemes is providedin Table. III. The lowest file size increments for different testimages with different QFs are displayed in bold in Table. III.It can be observed from Table. III that the file-size incrementsof all the marked JPEG images generated by our schemeare the lowest. Some file-size increments are even negative,which means that the file-size of the marked JPEG bitstreamis smaller than the file-size of the cover JPEG bitstream.For the cover JPEG images coded with the optimizedHuffman table, the comparison of the file size incrementsof the proposed scheme and the previous HS-based schemesis provided in Table. IV. The lowest file size increments fordifferent test images with different QFs are displayed in bold inTable. IV. It can be observed from Table. IV that most of thefile-size increments of the marked JPEG images generated byour scheme are lower than the prior HS-based schemes. Mostof the file-size increments of the proposed scheme are a littlehigher than the given payload while some file-size incrementsare lower than the given payload. The difference is related tothe zero-byte padding and byte-alignment operations in theJPEG standard [25].Further, we test the performance of file-size preservation onBOSSbase databases. 200 images from the BOSSbase imagedatabase are embedded the data with different payloads anddifferent QFs. The average file size increments are shownin Fig. 6 and Fig. 7. The cover JPEG images in Fig. 6are coded with the default Huffman table and the ones inFig. 7 are coded with the optimized Huffman table. It isobserved from Fig. 6 and Fig. 7 that the average file-size
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 10
TABLE IIIThe file size increments (bits) of the marked JPEG images under different QFs and different payloads using the proposed scheme and theprevious HS-based schemes. The input JPEG images are coded with the default Huffman table.
Image Scheme Payload (bits) with QF=30 Payload (bits) with QF=50 Payload (bits) with QF=70 Payload (bits) with QF=90 et al . [17] 3440 4800 6256 7568 4616 7600 10872 13624 6464 10816 14672 18960 7208 13704 20280 26304He et al . ( α =0) [18] 3304 5168 6264 7400 4496 7456 9776 11992 5176 8992 12008 15512 6792 12072 17312 23160He et al . ( α =1) [18] 1752 2888 4056 5520 3008 5664 8120 10816 3480 7072 10464 14520 4144 7760 11976 16536Li et al . [20] 1960 2976 4096 5304 3616 5792 8120 10280 5184 8456 11416 14568 6392 11272 16784 22096Yin et al . [19] 2640 4048 5360 6848 3776 6440 9632 12656 5160 9120 13440 17432 6872 12776 18632 24944Proposed -5080 -4120 -2976 -1976 -4208 -2064 -120 1960 -3304 -152 2472 5656 -9344 -5312 -1216 2736 Lena Hou et al . [17] 2728 3784 5136 6664 4296 7008 9552 12424 5808 10080 14032 18176 6568 12208 17640 23648He et al . ( α =0) [18] 2824 4152 5184 6568 4392 6704 9440 12240 5384 9392 13184 17944 6344 10744 16064 21104He et al . ( α =1) [18] 1896 2864 4088 5528 3008 5368 8352 11240 3792 7856 12496 17352 4736 9840 14800 20432Li et al . [20] 2048 3400 4736 5928 3288 5616 8256 10896 4336 7736 11384 15688 5360 9408 14304 19072Yin et al . [19] 2248 3016 4312 5624 3600 6344 8720 11640 5048 9232 12960 17744 6024 11336 16712 22848Proposed -3936 -2936 -1960 -1040 -608 1288 3320 5264 752 3864 6936 9936 -16 4248 8280 12216 Baboon Hou et al . [17] 2896 3832 5168 6888 4016 6592 9184 11648 5616 10144 14424 19216 8160 14456 22160 29192He et al . ( α =0) [18] 2888 4040 5384 6520 3696 5856 8240 11840 5688 9632 13904 18672 7784 14272 21912 27920He et al . ( α =1) [18] 1536 2448 3688 4624 2616 4488 7000 9168 4008 6960 11152 15360 5016 9512 14224 19128Li et al . [20] 2112 2888 4192 5328 3192 5200 7472 9984 4936 8432 12384 16608 7200 12704 19120 24040Yin et al . [19] 2296 3472 4664 5760 3568 5888 8576 11032 5216 9536 13880 18496 7856 13856 21072 27680Proposed -7224 -6088 -5144 -4064 -3176 -1232 824 2656 32 3344 6368 9152 -5072 -912 3152 6896 Boat Hou et al . [17] 3120 4048 5832 6648 4104 7080 9800 12336 5720 10432 14360 18720 7360 14304 21480 28408He et al . ( α =0) [18] 2928 4184 5272 6408 4232 6544 9480 12232 5784 10272 14328 19264 6384 11952 18272 24760He et al . ( α =1) [18] 1896 3024 4144 5176 2920 5584 8248 11560 4888 8856 13248 17992 5408 10488 15784 21560Li et al . [20] 2296 3272 4360 5360 3472 5664 8032 10664 4344 8336 12320 16384 5024 10248 16064 21928Yin et al . [19] 2464 3128 4920 5416 3296 5736 8264 11064 4744 9208 13536 17880 6928 12568 19184 25728Proposed -3672 -2616 -1600 -704 -72 1864 3640 5800 1112 3920 7312 10104 -10088 -5784 -1784 2272 Payload (bits) -50510 F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Yin et al.Proposed (a) BOSSbase (QF=30)
Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Yin et al.Proposed (b) BOSSbase (QF=50)
Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Yin et al.Proposed (c) BOSSbase (QF=70)
Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Yin et al.Proposed (d) BOSSbase (QF=90)Fig. 6. Average file size increment of 200 images from the BOSSbase image database. The cover JPEG images are compressed by the default Huffman table. increments obtained by the proposed scheme are lower thanthe previous schemes at different payloads and QFs obviously.Whether the cover JPEG images are coded with the defaultHuffman table or the optimized Huffman table, the proposedscheme is effective. Especially for the JPEG images codedwith the default Huffman table, the improvement is significant.Except the proposed scheme, the minimal file-size incrementis achieved by He et al . ’s scheme under the weighting factor α =
1, which is to minimize the file-size increments.
2) Visual Quality:
The comparison of PSNR values atdifferent payloads are listed in Table. V. As shown in Table.V, the MSEs of the proposed scheme are always equal tozero. That means the marked JPEG images generated by theproposed scheme keep the visual quality unchanged comparedto the cover JPEG images. Also we test the MSEs of the 200images from the BOSSbase image database. The results areillustrated in Fig. 8. As shown in Fig. 8, the MSEs of theproposed scheme are equal to zero. Among the recent HS-based RDH schemes, Yin et al . ’s scheme can preserve thebest visual quality at any QFs and payloads. For He et al . ’s scheme, when the weighting factor α =
0, the visual qualityis close to Yin et al . ’s scheme. However, when the weightingfactor α =
1, the visual quality is poorer than other HS-basedschemes. Since only file-size preservation is considered in thiscase.
3) Running Time:
For the HS-based RDH schemes, theadditional data is embedded by modifying the DCT coeffi-cients, whereas the data is embedded by replacing the codes inJPEG bitstream for our proposed scheme. Since the embeddingmechanism is different, the computational complexity cannotbe compared and we only compare the running time. Theaverage running time of 50 images from the UCID imagesdatabase is illustrated in Fig. 9 and Fig. 10. The cover JPEGimages in Fig. 9 are coded with the default Huffman table andthe ones in Fig. 10 are coded with the optimized Huffmantable. As shown in Fig. 9 and Fig. 10, the running timeof the proposed scheme is quite lower than the previousschemes. Noted that the running time of Yin et al . ’s schemefluctuates obviously, this is because of the computation ofthe optimization toolbox proposed by MATLAB. It can be EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 11
TABLE IVThe file size increments (bits) of the marked JPEG images under different QFs and different payloads using the proposed scheme and theprevious HS-based schemes. The input JPEG images are coded with the optimized Huffman table.
Image Scheme Payload (bits) with QF=30 Payload (bits) with QF=50 Payload (bits) with QF=70 Payload (bits) with QF=90 et al . [17] 3368 4584 5504 6632 4896 7624 9832 11952 6560 10720 14216 16936 8864 14672 21176 26192He et al . ( α =0) [18] 3336 4640 5472 6576 4848 7440 9504 11248 5896 10240 13384 16040 8680 14560 20000 25888He et al . ( α =1) [18] 2976 4352 5808 et al . [20] 2832 3848 4984 6400 4776 6672 9504 11704 5624 9144 12744 15968 7448 13144 18864 24648Proposed Lena Hou et al . [17] 2632 3952 5440 6760 3728 6080 8688 10520 5480 9128 12672 15480 7472 12776 17216 21584He et al . ( α =0) [18] 2576 3832 5328 6760 3616 5808 8768 10656 5424 8944 12192 15648 7680 11776 16840 21168He et al . ( α =1) [18] 2560 4024 5312 6560 et al . [20] 2272 3768 5008 6400 3144 5552 8088 10616 4712 7808 11232 15336 6136 10416 15024 19504Proposed Baboon Hou et al . [17] 2816 3928 5224 6384 3264 5136 7584 9400 4664 8640 12344 16168 7072 12168 18480 23664He et al . ( α =0) [18] 2888 4072 5288 6360 3080 4960 7320 9832 4736 8344 11968 16352 7776 13512 19864 24624He et al . ( α =1) [18] Li et al . [20] 2240 3528 4528 5736 2600 4536 6720 9208 4168 7464 11152 15280 7168 12160 18008 21744Proposed 2032 3176 4240 et al . [17] 2552 3528 4512 5584 3512 5768 7768 9896 4720 8384 11672 14928 7840 14272 19432 24112He et al . ( α =0) [18] 2344 3464 4456 5448 3552 5824 7968 10040 4800 8392 11776 16112 7728 13320 18376 24272He et al . ( α =1) [18] 2104 3304 4376 5504 et al . [20] 2168 3192 4368 5264 3192 5360 7712 9840 4136 7328 11016 14968 6320 11112 17200 21504Proposed Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Proposed (a) BOSSbase (QF=30)
Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Proposed (b) BOSSbase (QF=50)
Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Proposed (c) BOSSbase (QF=70)
Payload (bits) F il e S i z e I n c r e m en t ( b i t s ) Hou et al.He et al.( = 0)He et al.( = 1) Li et al.Proposed (d) BOSSbase (QF=90)Fig. 7. Average file size increment of 200 images from the BOSSbase image database. The cover JPEG images are compressed by the optimized Huffmantable.
Payload (bits) M SE Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (a) BOSSbase (QF=30)
Payload (bits) M SE Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (b) BOSSbase (QF=50)
Payload (bits) M SE Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (c) BOSSbase (QF=70)
Payload (bits) M SE Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (d) BOSSbase (QF=90)Fig. 8. Average MSE of 200 images from the BOSSbase image database.
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 12
TABLE VThe MSE of the marked JPEG images under different QFs and different payloads using the proposed scheme and the previous HS-based schemes.
Image Scheme Payload (bits) with QF=30 Payload (bits) with QF=50 Payload (bits) with QF=70 Payload (bits) with QF=90 et al . [17] 1.81 3.01 4.41 5.99 1.23 2.29 3.54 4.94 0.88 1.69 2.57 3.59 0.40 0.94 1.67 2.54He et al . ( α =0) [18] 1.73 2.97 4.29 5.96 1.15 2.11 3.19 4.45 0.74 1.39 2.11 3.02 0.29 0.60 0.96 1.37He et al . ( α =1) [18] 2.87 4.89 7.79 9.93 1.73 2.85 4.33 5.15 1.29 2.06 2.76 3.56 0.88 1.54 2.02 2.53Li et al . [20] 2.34 3.58 5.14 6.87 1.56 2.61 3.85 5.31 0.89 1.61 2.42 3.36 0.30 0.61 0.98 1.38Yin et al . [19] 1.52 2.56 3.76 5.29 1.02 1.96 3.11 4.48 0.72 1.46 2.30 3.30 0.37 0.86 1.54 2.35Proposed Lena Hou et al . [17] 2.13 3.69 5.43 7.48 1.31 2.61 4.33 6.41 0.70 1.53 2.60 4.11 0.21 0.42 0.70 1.04He et al . ( α =0) [18] 2.08 3.61 5.38 7.30 1.26 2.54 4.20 6.27 0.66 1.44 2.46 3.95 0.19 0.37 0.59 0.88He et al . ( α =1) [18] 2.88 6.77 8.98 10.17 2.81 5.13 7.90 9.52 1.36 4.11 4.88 6.47 0.79 0.63 1.64 1.12Li et al . [20] 2.45 4.31 6.55 9.19 1.50 2.97 5.20 8.38 0.78 1.67 2.99 5.24 0.19 0.37 0.60 0.94Yin et al . [19] 1.60 2.60 4.07 5.88 1.02 2.18 3.69 5.76 0.60 1.34 2.34 3.85 0.19 0.38 0.64 0.98Proposed Baboon Hou et al . [17] 2.31 3.90 5.74 7.77 1.79 3.44 5.47 8.04 1.31 2.83 4.86 7.85 0.73 1.62 2.80 4.25He et al . ( α =0) [18] 2.34 4.00 5.81 7.79 1.76 3.44 5.60 7.90 1.27 2.62 4.41 6.80 0.58 1.31 2.34 3.65He et al . ( α =1) [18] 10.96 15.07 17.41 20.77 7.07 14.94 18.98 26.60 8.72 15.47 23.84 31.43 2.21 5.78 9.74 12.91Li et al . [20] 2.29 3.85 5.80 7.96 1.73 3.44 5.53 8.10 1.23 2.69 4.74 7.46 0.55 1.24 2.26 3.57Yin et al . [19] 2.01 3.37 4.92 6.74 1.57 3.01 4.95 7.50 1.16 2.56 4.62 7.35 0.68 1.51 2.61 3.98Proposed Boat Hou et al . [17] 2.33 3.90 5.66 7.47 1.59 3.16 4.90 6.96 0.89 2.04 3.42 5.14 0.42 0.96 1.60 2.28He et al . ( α =0) [18] 2.28 3.74 5.24 6.92 1.54 3.03 4.73 6.72 0.85 1.97 3.25 4.88 0.24 0.53 0.91 1.39He et al . ( α =1) [18] 4.03 7.56 8.32 10.27 3.88 5.94 10.53 12.57 2.39 5.14 7.44 10.81 1.41 2.66 4.14 4.18Li et al . [20] 2.69 4.30 6.04 7.86 1.81 3.45 5.43 7.72 0.97 2.18 3.60 5.54 0.25 0.56 0.97 1.52Yin et al . [19] 1.75 2.92 4.54 5.92 1.21 2.51 4.08 6.10 0.75 1.72 3.05 4.81 0.36 0.82 1.39 2.04Proposed Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (a) BOSSbase (QF=30)
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (b) BOSSbase (QF=50)
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (c) BOSSbase (QF=70)
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Yin et al.Proposed (d) BOSSbase (QF=90)Fig. 9. Average running time of 200 images from the BOSSbase image database. The cover JPEG images are compressed by the default Huffman table. observed that the runtime of the proposed scheme increaseswith the QFs increases. That is because the number of nonzeroAC coefficients increases and the corresponding codes to bereplaced during the bitstream generation are increasing. Thus,more time is required to process the JPEG image. For theHS-based schemes, the number of whole AC coefficients isunchanged under any QF so the runtime is stable. Even ifthere is an increasing trend in the running time of the proposedscheme, the performance is still better than the other HS-basedschemes. VII. ConclusionAs one of the techniques used in RDH for JPEG images,the HCM technique existing two drawbacks, i.e., the lowembedding capacity and weak applicability. However, we areattracted by the capability of no visual distortion and thuswant to address the two problems. In this paper, we adopt theACM strategy that each used RSV is possible to be assignedwith more than one code. Based on the ACM strategy, we propose a universal framework to construct an HCM-basedRDH scheme for JPEG images. Under the framework, onecan get a new HCM-based RDH scheme by designing aspecific optimization algorithm. As an example, we proposea new HCM-based RDH scheme using the GA to obtain theoptimal code mapping relationship. The proposed scheme canbe applied to the JPEG images coded with any Huffman tableand obtain the high embedding capacity while keeping thevisual quality unchanged. The experiment results demonstratesthe performances on both of the file-size preservation andcomputational complexity of the HCM-based scheme surpassthe recent RDH schemes using the most popular technique,HS.In the future, we will continue to explore the followingdirections:1) More efficient optimization algorithms can be developedto construct an HCM-based scheme. Under the proposedframework, the design of the optimization algorithm is theonly problem to be deal with.2) The performance using the HCM-based scheme on the
EEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY 13
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Proposed (a) BOSSbase (QF=30)
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Proposed (b) BOSSbase (QF=50)
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Proposed (c) BOSSbase (QF=70)
Payload (bits) R unn i ng T i m e ( s ) Hou et al.He et al.( = 0) He et al.( = 1)Li et al. Proposed (d) BOSSbase (QF=90)Fig. 10. Average running time of 200 images from the BOSSbase image database. The cover JPEG images are compressed by the optimized Huffman table. encrypted JPEG bitstream should be investigated. Recently,RDH in encrypted images attracts more and more attentionfrom the researchers, and we are no exception.AcknowledgementReferences [1] J. Fridrich, M. Goljan, and R. Du, “Lossless data embeddingâĂŤnewparadigm in digital watermarking,”
EURASIP Journal on Advances inSignal Processing , vol. 2002, no. 2, p. 986842, 2002.[2] M. U. Celik, G. Sharma, A. M. Tekalp, and E. Saber, “Losslessgeneralized-lsb data embedding,”
IEEE transactions on image process-ing , vol. 14, no. 2, pp. 253–266, 2005.[3] J. Tian, “Reversible data embedding using a difference expansion,”
IEEEtransactions on circuits and systems for video technology , vol. 13, no. 8,pp. 890–896, 2003.[4] I.-C. Dragoi and D. Coltuc, “Local-prediction-based difference expan-sion reversible watermarking,”
IEEE Transactions on image processing ,vol. 23, no. 4, pp. 1779–1790, 2014.[5] Z. Ni, Y.-Q. Shi, N. Ansari, and W. Su, “Reversible data hiding,”
IEEETransactions on Circuits and Systems for Video Technology , vol. 16,no. 3, pp. 354–362, 2006.[6] X. Zhang, “Reversible data hiding with optimal value transfer,”
IEEETransactions on Multimedia , vol. 15, no. 2, pp. 316–325, 2012.[7] W. Zhang, X. Hu, X. Li, and N. Yu, “Recursive histogram modification:establishing equivalency between reversible data hiding and lossless datacompression,”
IEEE transactions on image processing , vol. 22, no. 7,pp. 2775–2785, 2013.[8] Y. Jia, Z. Yin, X. Zhang, and Y. Luo, “Reversible data hiding basedon reducing invalid shifting of pixels in histogram shifting,”
SignalProcessing , vol. 163, pp. 238–246, 2019.[9] J. Fridrich, M. Goljan, and R. Du, “Lossless data embedding for allimage formats,” in
Security and Watermarking of Multimedia ContentsIV , vol. 4675. International Society for Optics and Photonics, 2002,pp. 572–584.[10] K. Wang, Z.-M. Lu, and Y.-J. Hu, “A high capacity lossless data hidingscheme for jpeg images,”
Journal of systems and software , vol. 86, no. 7,pp. 1965–1975, 2013.[11] J. Fridrich, M. Goljan, Q. Chen, and V. Pathak, “Lossless data em-bedding with file size preservation,” in
Security, Steganography, andWatermarking of Multimedia Contents VI , vol. 5306. InternationalSociety for Optics and Photonics, 2004, pp. 354–366.[12] G. Xuan, Y. Q. Shi, Z. Ni, P. Chai, X. Cui, and X. Tong, “Reversibledata hiding for jpeg images based on histogram pairs,” in
InternationalConference Image Analysis and Recognition . Springer, 2007, pp. 715–727.[13] Q. Li, Y. Wu, and F. Bao, “A reversible data hiding scheme for jpegimages,” in
Pacific-Rim Conference on Multimedia . Springer, 2010, pp.653–664.[14] F. Huang, X. Qu, H. J. Kim, and J. Huang, “Reversible data hidingin jpeg images,”
IEEE Transactions on Circuits and Systems for VideoTechnology , vol. 26, no. 9, pp. 1610–1621, 2016.[15] Z. Qian, S. Dai, and B. Chen, “Reversible data hiding in jpeg images us-ing ordered embedding,”
KSII Transactions on Internet and InformationSystems (TIIS) , vol. 11, no. 2, pp. 945–958, 2017.[16] F. T. Wedaj, S. Kim, H. J. Kim, and F. Huang, “Improved reversibledata hiding in jpeg images based on new coefficient selection strategy,”
EURASIP Journal on Image and Video Processing , vol. 2017, no. 1,p. 63, 2017. [17] D. Hou, H. Wang, W. Zhang, and N. Yu, “Reversible data hiding in jpegimage based on dct frequency and block selection,”
Signal Processing ,vol. 148, pp. 41–47, 2018.[18] J. He, J. Chen, and S. Tang, “Reversible data hiding in jpeg imagesbased on negative influence models,”
IEEE Transactions on InformationForensics and Security , vol. 15, pp. 2121–2133, 2020.[19] Z. Yin, Y. Ji, and B. Luo, “Reversible data hiding in jpeg imageswith multi-objective optimization,”
IEEE Transactions on Circuits andSystems for Video Technology , 2020.[20] N. Li and F. Huang, “Reversible data hiding for jpeg images based onpairwise nonzero ac coefficient expansion,”
Signal Processing , vol. 171,p. 107476, 2020.[21] Z. Qian and X. Zhang, “Lossless data hiding in jpeg bitstream,”
Journalof Systems and Software , vol. 85, no. 2, pp. 309–313, 2012.[22] Y. Hu, K. Wang, and Z.-M. Lu, “An improved vlc-based lossless datahiding scheme for jpeg images,”
Journal of Systems and Software ,vol. 86, no. 8, pp. 2166–2173, 2013.[23] Y. Qiu, H. He, Z. Qian, S. Li, and X. Zhang, “Lossless data hidingin jpeg bitstream using alternative embedding,”
Journal of VisualCommunication and Image Representation , vol. 52, pp. 86–91, 2018.[24] C. Zhang, B. Ou, and D. Tang, “An improved vlc mapping methodwith parameter optimization for reversible data hiding in jpeg bitstream,”
Multimedia Tools and Applications , pp. 1–18, 2020.[25] J. Standard, “Information technology-digital compression and coding ofcontinuous-tone still images-requirements and guidelines,”