Extension of JPEG XS for Two-Layer Lossless Coding
EExtension of JPEG XSfor Two-Layer Lossless Coding
Hiroyuki KOBAYASHI
Tokyo Metropolitan College of Industrial Technology,Email: [email protected]
Hitoshi KIYA
Tokyo Metropolitan UniversityEmail: [email protected]
BSTRACT
A two-layer lossless image coding method compatible withJPEG XS is proposed. JPEG XS is a new international standardfor still image coding that has the characteristics of very lowlatency and very low complexity. However, it does not supportlossless coding, although it can achieve visual lossless coding.The proposed method has a two-layer structure similar toJPEG XT, which consists of JPEG XS coding and a losslesscoding method. As a result, it enables us to losslessly restoreoriginal images, while maintaining compatibility with JPEGXS.
Index Terms —JPEG XS, lossless coding, two-layer coding
I. I
NTRODUCTION
JPEG XS was standardized as a new still image codingmethod [1]. This standard is also expected to be applied tovideos, and enables us to compress images with low latencyand low complexity. The coding aims to realize visual losslessquality, so it is not guaranteed to achieve lossless coding.There are many applications that require lossless coding,such as medical images, and master data of the cinemaand TV programs. In addition, lossless coding allows us tocombine coding with other technologies such as data hidingand encryption [2], [3]. Although numerous encodings suchas JPEG-LS [4], JPEG 2000 [5], and JPEG XR [6] havebeen standardized for supporting lossless coding, conventionalencoding methods have not considered the features of lowlatency and low cost that JPEG XS has.Two-layer codings have been researched as a method ofcombining the characteristics of several codings [7]–[16] JPEGXT is a two-layer coding that uses JPEG as the first baselayer in consideration of the compatibility with past JPEGdecoders. The second extension layer holds the residual imagebetween the original image and the base layer decoded image.In addition, JPEG XT Part 8 [17], which is the the extensionof JPEG XT, encodes the difference information losslessly. Asa result, losslessness of the bitstream can be realized.Because of such a situation, we propose extending JPEG XSfor supporting lossless coding, while maintaining the featuresof JPEG XS. The extended coding has a two-layer structure,where the first layer, called base layer corresponds to theJPEG XS coding, and the second one, called extension layer isused for compressing residual data between an original imageand the decoded image from the base layer. This two-layer
TABLE IB
ITRATES OF LOSSLESS CODING
Image JPEG XS JPEG 2000 JPEG LS JPEG XRlena(24[bits]) 22.0 13.62 13.57 14.10Moss(30[bits]) Not realized 19.19 20.86 23.74Moss(36[bits]) Not realized 26.30 25.55 26.67 structure has been inspired by JPEG XT Part 8 [17] and itsextension [13]. The proposed coding is compatible with JPEGXS. In an experiment, the proposed coding is demonstrated notonly to have compatibility with JPEG XS, but also to achievelossless coding. II. JPEG XSJPEG XS is a new standard for still image coding [1].This standard is intended for low latency and low complexityencoding, and is expected to be applied to moving picturecoding in which each frame is regarded as an independentstill image. JPEG XS aims at encoding at a compression ratioof about 1/2 to 1/10 while maintaining visual lossless imagequality, not improving the compression ratio for low bitrates.The JPEG XS encoding uses the wavelet transform thatis also used in JPEG 2000. However, the processing in thevertical direction is suppressed to a few lines, thereby realizinglow latency and low complexity in encoding and decoding.Furthermore, since there is no frame buffer for the entireimage, it can be implemented at low cost.JPEG XS supports visual lossless coding, but does notsupport lossless one. Table I shows examples of lossless codingfor JPEG2000, JPEG LS, JPEG XR and JPEG XS. In the table,image ‘lena’ was losslessly compressed by JPEG XS, but theimage ‘Moss’ was not done. In contrast, other compressionmethods losslessly compressed all images.III. P
ROPOSED M ETHOD
As mentioned above, JPEG XS can not encode images loss-lessly. Therefore, we consider two-layer coding that consistsof a base layer and an extension layer. Figure 1(a) shows theencoder structure of the proposed lossless two-layer codingfor N -bit-images. The coding-path for generating the baselayer is backward compatible with JPEG XS. For the extensionlayer, the residual image R ( x, y ) is generated by calculatingthe difference between decoded base layer image P (cid:48) ( x, y ) andthe original image P ( x, y ) as R ( x, y ) = P ( x, y ) − P (cid:48) ( x, y ) . (1) a r X i v : . [ c s . MM ] A ug PEG XSencoder (N[bits]) image (N[bits]) — MUX
Extension layerBase layerResidual image (N+1[bits]) losslessencoder (N+1[bits])
XS bpp
JPEG XSdecoder (N[bits])
DC shift P ( x, y ) P (cid:1) ( x, y ) R ( x, y ) R (cid:1) ( x, y ) (a) encoder Splitter JPEG XSdecoder (N[bits]) losslessdecoder (N+1[bits]) D C un s h i f t R (cid:1) ( x, y ) + R ( x, y ) P (cid:1) ( x , y ) image (N[bits]) Extension layerBase layer P ( x, y ) (b) decoder Fig. 1. Block diagram of proposed method
The residual data R ( x, y ) include negative values, but loss-less image compression methods do not support image withnegative pixel values in general. Therefore, all pixel values in R ( x, y ) are shifted by the DC shift operation as R (cid:48) ( x, y ) = R ( x, y ) + 2 N − , (2)where R (cid:48) ( x, y ) is expressed by using N + 1 bits. After theDC shifting operation, R (cid:48) ( x, y ) is encoded by using a losslessencoder such as JPEG-LS, JPEG 2000, and JPEG XR.Figure 1(b) shows the decoder structure of the proposedmethod. Bitstreams of the base layer are decoded by JPEGXS and ones of the extension layer are decoded by a losslessdecoder. A residual image is reconstructed from the decodedimage by using the DC inverse-shift operation, and is addedto an image from the base layer. Since the residual imageis decoded losslessly, the final output image is also perfectlyreconstructed. IV. E XPERIMENTAL RESULTS
The compression performance of the proposed methodwas compared with two one-layer lossless codings: JPEG-LSand JPEG 2000. In the experiment, the reference softwaresprovided by the JPEG committee were used. Six 2K imageswith a depth of 30 bits and six 4K images with a depth of36 bits provided from the Institute of Image Information andTelevision Engineers (ITE) [18] were used in this experiment.Figure 2 shows the six thumbnail images of 2K and 4K imagesand Fig.3 shows the file formats for 2K and 4K images. The 6MSB bits in the 2K images and 4 MSB bits in the 4K imagesare filled with zero bits. In this paper, the six images areclassified into two sets for convinience. Set 1 has ‘MusicBox’,‘StainedGlass’ and ‘Sea’, and Set 2 has ‘Books’, ‘Moss’, and‘ChromaKey’. (a) MusicBox (b) StainedGlass(c) Sea (d) Books(e) Moss (f) ChromaKey
Fig. 2. Test images
RRRRRRRRRR000000 R GGGGGGGGGG000000 G BBBBBBBBBB000000 B RRRRRRRRRRRR0000 R GGGGGGGGGGGG0000 G BBBBBBBBBBBB0000 B
2K 4K1920px 3840px1080px 2160px
Fig. 3. File formats
A. JPEG XS image quality
At first, the quality of base-layer images, which are pro-duced from base layer bitstreams by using the JPEG XSdecoder, is addressed. Figure 4 shows rate distortion curvesof reconstructed base-layer images.For all images, PSNR values saturated at a certain PSNRvalue. In other words, all images were not compressed loss-lessly, even when bitrate values increased. In particular, image‘Books’ saturated at 4[bpp] in 2K images and at 6[bpp] in 4Kimages.
B. Total bitrates of two-layer coding with JPEG 2000
Figure 5 shows total bitrates of the proposed two-layercoding under various bitrates of JPEG XS, where zero valuein the horizontal axis corresponds to lossless coding withoutthe base layer. From the results, the proposed coding wasdemonstrated to achieve lossless coding under all conditions.Besides, the total bitrate values increase, compared with thoseof using only JPEG 2000 without the two-layers structure.
20 25 30 35 40 45 0 2 4 6 8 10 12 14 PS NR o f ba s e i m age [ d B ] JPEG XS bitrate [bpp]MusicBoxStainedGlassSea
20 25 30 35 40 45 0 2 4 6 8 10 12 14 PS NR o f ba s e i m age [ d B ] JPEG XS bitrate [bpp]BooksMossChromaKey (a) 2K-images (Set 1) (b) 2K-images (Set 2)
20 25 30 35 40 45 50 55 60 0 2 4 6 8 10 12 14 16 18 PS NR o f ba s e i m age [ d B ] JPEG XS bitrate [bpp]MusicBoxStainedGlassSea
20 25 30 35 40 45 50 55 60 0 2 4 6 8 10 12 14 16 18 PS NR o f ba s e i m age [ d B ] JPEG XS bitrate [bpp]BooksMossChromaKey (c) 4K-images (Set 1) (d) 4K-images (Set 2)
Fig. 4. Rate distortion curves of JPEG XS
C. Comparison of lossless coders
The proposed two-layer coding allows us to use an arbitrarylossless encoder. Compared with JPEG 2000, JPEG-LS haslow-delay and low-complexity. The two-layer coding withJPEG-LS was compared with that with JPEG 2000 in termsof compression performance.Figure 6 shows the difference between total bitrate values ofthe two-layer coding with JPEG 2000 and those of with JPEG-LS. From the figure, JPEG 2000 provided higher compressionperformance than with JPEG-LS. In particular, when thebitrate value of JPEG XS was larger, the difference betweenthem was larger. V. C
ONCLUSIONS
We proposed a novel two-layer lossless coding method withbackward compatibility to JPEG XS. In the proposed coding,JPEG XS is used as the base layer, and the difference betweenan original image and the base layer image is encoded in theextension layer by using an arbitrary lossless encoder. In theexperiment, the proposed method was confirmed to achievelossless coding under all conditions. The difference betweenwith JPEG 2000 and with JPEG-XS was also compared.R
EFERENCES[1] “Information technology JPEG XS low-latency lightweight imagecoding system Part 1,” ISO/IEC 21122-1:2019, 2019.[2] M. Fujiyoshi, S. Sato, H. L. Jin, and H. Kiya, “A location-mapfree reversible data hiding method using block-based single parameter,”
Proceedings - International Conference on Image Processing, ICIP , vol.3, pp. 257–260, 2006.
10 15 20 25 30 0 2 4 6 8 10 12 14 T o t a l b i t r a t e [ bpp ] JPEG XS bitrate[bpp]MusicBoxStainedGlassSea
10 15 20 25 30 0 2 4 6 8 10 12 14 T o t a l b i t r a t e [ bpp ] JPEG XS bitrate[bpp]BooksMossChromaKey (a) 2K-images (Set 1) (b) 2K-images (Set 2)
15 20 25 30 35 0 2 4 6 8 10 12 14 16 18 T o t a l b i t r a t e [ bpp ] JPEG XS bitrate[bpp]MusicBoxStainedGlassSea
15 20 25 30 35 0 2 4 6 8 10 12 14 16 18 T o t a l b i t r a t e [ bpp ] JPEG XS bitrate[bpp]BooksMossChromaKey (c) 4K-images (Set 1) (d) 4K-images (Set 2)
Fig. 5. Total bitrates of proposed coding with JPEG 2000[3] K. Kurihara, S. Imaizumi, S. Shiota, and H. Kiya, “An Encryption-then-Compression System for Lossless Image Compression Standards,”
IEICE Transactions on Information and Systems , vol. E100.D, no. 1,pp. 52–56, 2017.[4] “Information technology Lossless and near-lossless compression ofcontinuous-tone still images: Baseline Part 1:,” ISO/IEC 14495-1:1999,1999.[5] “Information technology — JPEG 2000 image coding system – Part 1:Core coding system,” International Standard ISO/IEC IS-15444-1, dec2000.[6] “Information technology — JPEG XR image coding system – Part 1:System architecture,” ISO/IEC TR-29199-1, jul 2011.[7] M. Iwahashi and H. Kiya, “Efficient lossless bit depth scalable codingfor HDR images,” in
Proceedings of The 2012 Asia Pacific Signal andInformation Processing Association Annual Summit and Conference , dec2012, pp. 1–4.[8] M. Iwahashi, H. Kobayashi, and H. Kiya, “Lossy compression of sparsehistogram image,” in
Proc. IEEE Int. Conf. on Acoustics, Speech andSignal Processing . mar 2012, pp. 1361–1364, IEEE.[9] Sehoon Yea and W. Pearlman, “A wavelet-based two-stage near-losslesscoder,” in
Proc. IEEE Int. Conf. on Image Processing . 2004, vol. 4, pp.2503–2506, IEEE.[10] M. Okuda and N. Adami, “Two-layer coding algorithm for high dynamicrange images based on luminance compensation,”
Journal of VisualCommunication and Image Representation , vol. 18, no. 5, pp. 377–386,oct 2007.[11] M. Iwahashi and H. Kiya, “Two layer lossless coding of HDR images,”in
Proc. IEEE Int. Conf. on Acoustics, Speech and Signal Processing ,may 2013, pp. 1340–1344.[12] “Information technology — Scalable compression and coding ofcontinuous-tone still images – Part 1: Scalable compression and codingof continuous-tone still images,” ISO/IEC 18477-1, 2015.[13] O. Watanabe, H. Kobayashi, and H. Kiya, “Two-layer Lossless HDRCoding considering Histogram Sparseness with Backward Compatibilityto JPEG,” in
Picture Coding Symposium (PCS) , jun 2018, pp. 11–15.[14] H. Kobayashi, O. Watanabe, and H. Kiya, “Two-Layer Near-LosslessHDR Coding Using Zero-Skip Quantization with Backward Compati-
10 12 14 16 18 20 22 24 26 0 1 2 3 4 5 6 7 8 9 10 T o t a l b i t r a t e [ bpp ] JPEG XS bitrate [bpp]w JPEG 2000w JPEG-LS (a) 2K-images
14 16 18 20 22 24 26 28 30 0 1 2 3 4 5 6 7 8 9 10 T o t a l b i t r a t e [ bpp ] JPEG XS bitrate [bpp]w JPEG 2000w JPEG-LS (b) 4K-images
Fig. 6. Boxplot of total bitrates of the proposed coding with JPEG 2000 /JPEG-LSbility to JPEG,”
IEICE Transactions on Fundamentals of Electronics,Communications and Computer Sciences , vol. E102.A, no. 12, pp. 1842–1848, 2019.[15] H. Kobayashi, O. Watanabe, and H. Kiya, “Two-Layer Near-LosslessHDR Coding with Backward Compatibility to JPEG,”
Proceedings -International Conference on Image Processing, ICIP , pp. 3547–3551,2019.[16] H. Kobayashi and H. Kiya, “Performance evaluation of two-layer loss-less HDR coding using histogram packing technique under various tone-mapping operators,”2019 IEEE 8th Global Conference on ConsumerElectronics, GCCE 2019