Mobile-end Tone Mapping based on Integral Image and Integral Histogram
Jie Yang, Mengchen Lin, Ziyi Liu, Ulian Shahnovich, Orly Yadid-Pecht
MMobile-end Tone Mapping based on Integral Imageand Integral Histogram
Jie Yang , Mengchen Lin , Ziyi Liu , Ulian Shahnovich , and Orly Yadid-Pecht I2Sense lab, University of Calgary, Calgary T2N 1N4, Canada Westlake University, Hangzhou 310024, China
Abstract —Wide dynamic range (WDR) image tone mappingis in high demand in many applications like film production,security monitoring, and photography. It is especially crucialfor mobile devices because most of the images taken today arefrom mobile phones, hence such technology is highly demandedin the consumer market of mobile devices and is essential forgood customer experience. However, high quality and high-performance WDR image tone mapping implementations arerarely found in the mobile-end. In this paper, we introduce a highperformance, mobile-end WDR image tone mapping implemen-tation. It leverages the tone mapping results of multiple receptivefields and calculates a suitable value for each pixel. The utilizationof integral image and integral histogram significantly reduce therequired computation. Moreover, GPU parallel computation isused to increase the processing speed. The experimental resultsindicate that our implementation can process a high-resolutionWDR image within a second on mobile devices and produceappealing image quality.
Index Terms —Wide dynamic range, compression, mobilegraphic processing unit (GPU), tone mapping, parallel comput-ing.
I. I
NTRODUCTION W IDE dynamic range is defined as the ratio of theintensity of the brightest point to that of the darkestpoint in a scene or image. Traditional display devices suchas LCD, CRT, and LED are usually limited to 8 bits Thusthey can only represent the dynamic range of 255:1. How-ever, the fast developing image sensor technology and imageprocessing algorithms enable the capture of WDR images witha much wider dynamic range than that of the display devices.Therefore it is impossible to properly reproduce the WDRimage on the display directly. Tone mapping algorithms areoften called tone mapping operators (TMO), they serve thegoal of compressing the WDR image to match the dynamicrange of the display devices. The development of tone mappingalgorithms started to emerge in the early 1990s. Tumblin andRushmeier [1] and Ward [2] did the earliest attempts. Tumblinand Rushmeier aimed to match the perceived brightness of thedisplayed image with that of the scene. Ward used a linearscaling, focusing on preserving the image contrasts. Thesealgorithms are classified as global tone mapping functionsbecause they use the same tone-curve for all the pixels ofthe WDR image. In general, global tone mapping algorithmsare computationally easy and mostly artifact-free. Hence, they
Corresponding author e-mail: [email protected] have unique advantages in implementations. However, thereare only a few global TMO implementations found in theliterature because they are prone to loose details and contrastin either bright or dark regions due to the global compressionfeature. Drago et al. [3] proposed a function that could changethe base of the logarithmic function according to the pixelbrightness. It is one of the most commonly used examples oftone mapping in various publications. The sigmoid functionis similar to the response curve of the human visual system(HVS), it is thus used in many bio-inspired TMOs. Works like[4–7] try to simulate the procedure of dynamic range com-pression of the HVS by mimicking the response curve of ourphoto-receptors. Although these approaches may be effectivein reducing the dynamic range, they have an inherent flaw thatthe tone mapped image represents the internal representationrather than the luminance which is more expected by our eyes.In many research works, tone mapping is regarded as aconstrained optimization problem. The objective is to achievea tone-mapping that is most preferred in terms of objectivequality. Mantiuk et al. tried to minimize visible distortionduring the tone mapping [8]. Ma et al. proposed a tonemapping method that can optimize the tone mapped imagequality index [9]. Recently, tone mapping with edge preservingfilter has become the most popular way to tone map WDRimages [10–14]. It first applies an edge-preserving filter tothe log domain of the WDR image’s luminance channel. Theedge-preserving filter separates the image into two layers,namely a base layer that contains global brightness informationand a detailed layer that mainly consists of local texture.The base layer is processed with the compressive tone-curvewhile the detail layer is untouched. Thus, local details couldbe preserved while the overall dynamic range is reduced.Finally, the base-layer and the detail-layer are combined andtransformed back to the original linear domain for display.Most tone mapping algorithms are implemented in desktop-end. However, there are several GPU implementations made inrecent years, which demonstrate great computational efficiencywith the help of GPU’s parallel processing ability. Chen et. al achieved 50 Hz tone mapping by developing a newdata structure and paralleling their edge preserving filter onGPU [15]. Akyuz demonstrated a WDR imaging pipelinerealized by GPU [16]. It yields 2 to 3 orders of performanceimprovement when compared to the CPU implementation.Urena et. al evaluated both GPU and FPGA performances on anew tone mapping algorithm [17]. When compared with CPU a r X i v : . [ ee ss . I V ] F e b implementation, speed-up factors of 7.5 and 15 are achievedfor the GPU and FPGA, respectively.In this paper, we propose an algorithm that is inspiredby the HVS, it tone maps a pixel by taking into accountmultiple receptive fields that surround the pixel. The uti-lization of integral image and integral histogram make thewhole tone mapping process highly parallel. Experimentalresults including the image quality evaluation and mobile-end implementation performance are carried out to prove theeffectiveness and efficiency of our work.The rest of the paper is structured as follows. Section 2details the proposed algorithm and section 3 describes themobile-end implementation. Section 4 provides experimentalresults. The last section concludes this work.II. P ROPOSED A LGORITHM
When viewing WDR scenes, our HVS adopts the localadaptation mechanism to help us see the details in all partsof the scene. Local adaptation can be mainly explained asthe ability to accommodate the level of a certain visual fieldaround the current fixation point. Moreover, it also revealsthat different luminance intervals could result in overlappingreactions on the limited response range of the visual system,thus extending our visual response range to cope with thefull dynamic range of high contrast scenes. Inspired by theconcept of local adaptation, we design our tone mappingalgorithm as follows. The WDR image i is first transferred to l using logarithmic compression. For every pixel l ( x, y ) , one canalways find s different receptive fields w i , i = 1 , ...s where l ( x, y ) is the center of every receptive field. To adapt to everyvisual field, we can tone map them separately. Since HVS hasa logarithmic response to light intensity, we choose histogramadjustment [18] which is one of the simplest tone mappingmethods to tone map every receptive field. The processing flowof applying histogram adjustment to every receptive field w i issummarized as follow: first, a histogram of image luminance inthe logarithmic domain is constructed for every w i . Denoting f w i ( b i ) as the pixel count in a bin b i of the histogram, acumulative probability function is defined as P w i ( b ) = 1 T (cid:88) b i
The integral image is prominently used within the Viola-Jones object detection framework from 2001 [19]. It cansignificantly reduce the computation burden when an accumu-lative sum of image area is required. The following equationdefines an integral image. I ( x, y ) = (cid:88) x (cid:48) ≤ x, y (cid:48) ≤ y i ( x (cid:48) , y (cid:48) ) (5)where i ( x, y ) is the value of the pixel at ( x, y ) . A great featureof integral image I is that summation of any rectangularregion in the original image i can be computed efficiently ina single pass. For example, if there are four points A ( x , y ) , B ( x , y ) , C ( x , y ) and D ( x , y ) ( x < x , y < y ) inimage i , the accumulative summation of rectangular w that isenclosed by the four points is equal to: (cid:88) x In this part, we make a detailed analysis of how the proposedalgorithm compresses the dynamic range while preservinglocal details.The main reason that the algorithm adopts a simple tonemapping method (Eq. 2) rather than a more complicated one totone map every receptive field w i is because the local dynamicrange of a single receptive field w i is mostly much lower thanthe dynamic range of the entire WDR image. Fig. 1 showsan example that a simple tone mapping method is alreadysufficient to produce good results. The image on top of Fig. 1is the original WDR image. We tone map this image withdifferent algorithms. Two patches of the same size and atthe same location are selected from the corresponding tonemapped images. From left to right on the two bottom rowsof Fig. 1, the image patches are from Mantiuk et al. [8],Durand et al. [10], Reinhard et al. [21] and Drago et al. [3],respectively. The last column shows image patches from theoriginal WDR image and then tone mapped with a logarithmicresponse function. The image patches using local processingshow comparable or even better results when compared withother algorithms. In the lamp area, the image is not as saturatedas the other four images, and in the car area, the image isbrighter than the other four images. This example demonstratesthat with local adaptation, even a simple tone mapping methodcan reveal the local details of wide dynamic range scenes.However, revealing local detail is not enough for good tonemapping, because artifacts usually emerge if there is no con-sideration of global statistics. Figure. 2 shows a tone mappingexample of our algorithm with only one receptive field ( s = 1 in Eq. 3). It is apparent that the smaller the receptive field,the better the details, but there are many disturbing artifactsespecially at uniform areas. On the other hand, the larger thereceptive field, the fewer the details as well as the artifacts.Actually, the size of the receptive field can balance between theglobal and local statistics during tone mapping. Consider themost extreme situation when the receptive field is as large asthe WDR image itself, then the tone mapping becomes global. In this circumstance, there will be no artifacts because the tonemapping is global, it can preserve the brightness consistencyof the image based on the monotonic tone mapping curve. But,the details are also concealed due to the global compressionfeature. If the receptive field is so small that there are onlyseveral pixels, the details will be greatly exaggerated becauseevery receptive field will be given the maximum display rangebetween min ( L d ) and max ( L d ) regardless of its size. If thepixels have similar values, for instance, they are all backgroundpixels of clear sky, then, some pixel will still be tone mappedto min ( L d ) and become artifacts because of Eq. 2. The localcompression could sabotage the image brightness consistency,especially at uniform areas because the tone mapping functionof the entire WDR image is no longer monotonic. Although theresults are significantly different when tone mapped with largeand small receptive fields, it is also evident that the displayedvisual appearances are complimentary which contain localdetail and global brightness consistency. Since every pixel isincluded in s different receptive fields, one solution to obtainthese local and global appearances is to fuse all receptive fieldsusing a weighted summation as shown in Eq. 3.To reveal local detail and maintain brightness consistency,we should consider the two possible cases of local receptivefields. In a “flat” receptive field where pixel values are mostlythe same, we should tone map it more globally to maintainits image consistency and reduce possible artifacts. In a“high variance” receptive field where pixel values fluctuatesignificantly, we should tone map it more locally to revealdetails. Eq. 4 is taken from guided image filter [12] and itis a very efficient way to measure the “flatten” or “variance”degree of a local image area. If the variance value σ w i , ( x,y ) ismuch larger than (cid:15) , the weight value will be close to 1, whichindicates that the centre pixel of receptive field w i , ( x, y ) is“high variance”. If the variance is much less than (cid:15) , the weightvalue will be close to 0, and the center pixel is regarded asbelonging to a “flat” receptive field. Eq. 4 gives pixel-wiseand receptive-field-wise weight for each pixel and it has edgepreserving feature that is helpful for artifacts reduction. Fig. 3shows the weight map calculated with different receptive field.Bright pixels mean the computed weight value is close to oneand dark pixels mean that it is close to zero. It can be seenthat in smaller scales, the weight can better preserve detailssuch as edges and textures. C. Parameter Setting There are three free parameters involved in our algorithm,namely, the number of different receptive fields s , the numberof the histogram bins n , and the regularization term (cid:15) . Thenumber of receptive fields s is the most important parameterin the proposed algorithm. As we have discussed previously,to maintain the image brightness consistency, the largestreceptive field should be the same as the image size. Yetto obtain possible details in different receptive fields, weneed smaller receptive fields as well. We adopt the popularimage pyramid method and define the relationship for anytwo adjacent receptive fields w i +1 and w i as w i /w i +1 = 2 .Consider a × WDR image, s = 5 will yield the Fig. 3: Weight maps computed using Eq. 4. (a) w i is equal toquarter image height and width of image. (b) w i is equal to1/8 image height and width of image. (c) w i is equal to 1/16image height and width of image.Fig. 4: Tone mapped images obtained with various n and (cid:15) values.smallest receptive field of × which is enough for thetone mapping function of Eq. 2 to preserve local details. Theeffect of the number of bins n and the regularization term (cid:15) parameters for a tone mapped image are shown in Fig. 4. Ninetone mapped images are presented in a matrix with n varyingvertically and (cid:15) varying horizontally. It is apparent that theimage contrast increases as n increases. The decrease of (cid:15) inEq. 4 will result image with better local details. In Fig. 4, thetexture of the window glass is best preserved with smallest (cid:15) value. We find values for n > and (cid:15) = 0 . to usually producesatisfactory results, i.e. good brightness while preserving localcontrast and details.III. M OBILE - END I MPLEMENTATION The proposed algorithm is implemented in the iOS platformas an application. The application first captures four differentexposures and then the four images are used to recover a WDRimage using the algorithm proposed by Paul Debevec [22]. Fig. 5: Main building blocks of GPU implementation.The generated WDR image is converted to luminancechannel using the following equation: i = 0 . ∗ R + 0 . ∗ G + 0 . ∗ B (11)where R , G , and B are the red, green and blue channelof the WDR image, respectively. Then, the tone mappedimage is generated by applying the proposed algorithm on theluminance image i . At last, the color information is restoredusing the same color restoration function used in [23]: c out = ( C in L in ) sat L out (12)where C = R, G, B represents the three color channels, and L in , L out denote the luminance before and after WDR tonemapping, respectively. sat is a parameter controlling colorsaturation that is set as . . In the following, we first brieflyintroduce the programming model and then will focus on theGPU implementation of the tone mapping algorithm. A. Mobile-end GPU Implementation The algorithm is implemented using Apple’s GPU appli-cation programming interface, Metal. Metal allows for theparallel processing of data much like OpenGL or other shaderprogramming languages. In Metal, kernel function is the basicfunction that runs in SIMD fashion in GPU. Each instanceof kernel function is called a thread. Threads are furtherorganized into threadgroups that are executed together andcan share a common block of memory. Since the proposedalgorithm is pixel-parallel, then we can assign every pixel athread to tone map itself and all threads share the commonmemory which is the calculated integral image and integralhistogram. The overall implementation flow of the algorithmis shown in Fig. 5. The input and output of the algorithm arethe calculated lumniance image and tone mapped lumnianceimage, respectively. The blue blocks indicate functions thatexecute only once during the computation. They are used tocalculate the logarithmic image, integral histogram H andintegral image I ( i ) and I ( i ) , respectively. MPSImageIntegral Algorithm 1 Eq. 4 GPU implementation pseudocode Input: integral of luminance image I (cid:80) ; integral of luminanceimage I (cid:80) ; regularization term (cid:15) ; current scale j ; Output: weight, W w j ; for all pixel in L (cid:80) and L (cid:80) do x − = grid.x − ( s j .x + 1) x + = grid.x + s j .x y − = grid.y − ( s j .y + 1) y + = grid.y + s j .y x in = 2 s j .x + 1 y in = 2 s j .y + 1 | w | = x in ∗ y in A ← I (cid:80) ( i,j ) ( x − , y − ) B ← I (cid:80) ( i,j ) ( x + , y − ) C ← I (cid:80) ( i,j ) ( x + , y + ) D ← I (cid:80) ( i,j ) ( x − , y + ) µ = A + C − B − D A ← I (cid:80) ( i,j ) ( x − , Y − ) B ← I (cid:80) ( i,j ) ( x + , Y − ) C ← I (cid:80) ( i,j ) ( x + , Y + ) D ← I (cid:80) ( i,j ) ( x − , Y + ) µ = A + C − B − D σ w = ( µ − µ ) / | w | W w j = σ w / ( σ w + (cid:15) ) end for return W w j ;function that is provided by Metal Performance Shaders isused to calculate the integral image and integral histogram.The core functions are labeled with green color in Fig. 5.Our implementation calculates different scales in sequence,the functions that are required in this procedure are labeledas green, and they will be called for s times in a main loop(shaded gray). The pseudocode code of Algorithm 1 showshow the Eq. 4 is implemented in GPU as kernel function. I (cid:80) and I (cid:80) are the computed integral image and of i and i ,respectively. They can be accessed by all threads. grid.x and grid.y are the vertical and horizontal coordinates of currentthread. Line 2 to line 5 are used to calculate coordinates of thefour corner pixels of Eq. 6 and Eq. 9. | w | calculates the numberof pixels in the receptive fields. Eq. 6 and Eq. 9 are executedin line 13 and line 18. Line 19 implement Eq. 10 and Eq. 4 iscarried out in line 20. The implementation of algorithm 1 onlyemploys basic addition, subtraction and indexing operations ofGPU.Eq. 1, Eq. 2 and Eq. 3 are also implemented as kernelfunctions because they only depend on the pixel value andthe integral histogram. The parallel processing feature ofthe algorithm greatly reduce the computation complexity andcost of the implementation. Evaluation results of the GPUimplementation will be shown in the following section.IV. E XPERIMENTAL R ESULTS In this section, we present image quality comparison of ouralgorithm as well as the performance of the GPU implemen-tation. TABLE ITMQI NATURALNESS S CORES Image Durand et al. [10] Gu et al. [13] Paris et al. [14] ProposedBristolBridge 0.0802 0.3262 0.2160 ClockBuilding 0.8032 DomeBuilding 0.4134 0.9108 0.4570 FribourgGate 0.3923 StreetLamp 0.5590 0.7952 0.4141 Vernicular 0.4190 0.6256 TABLE IITMQI S TRUCTURAL S IMILARITY S CORES Image Durand et al. [10] Gu et al. [13] Paris et al. [14] ProposedBristolBridge 0.8368 0.7947 0.8387 ClockBuilding 0.8780 0.8101 0.8795 CrowFootGlacier 0.9274 0.8181 0.9422 DomeBuilding 0.7364 0.6968 0.7498 FribourgGate 0.9362 0.9045 0.9419 MontrealStore Oaks 0.9333 0.8832 0.9578 StreetLamp 0.8844 0.8694 0.8654 Vernicular 0.9117 0.8946 0.9255 Average 0.8892 0.8467 0.8958 Our algorithm is compared with three tone mapping algo-rithms — local Laplacian-based tone mapping by Paris et al. [14], the fast bilateral filtering tone mapping method by Du-rand et al. [10] and edge-preserving multi-scale decompositionalgorithm proposed by Gu et al. [13]. The three algorithmsreported state-of-the-art tone mapped image quality. We usethe codes of [14] and [10] provided by the authors themselves.For Gu’s method, we implemented the algorithm based on thecode of [12]. In the image quality assessment, all comparedalgorithms use their default parameter settings. Our algorithmuse parameter setting n = 5 , s = 5 and (cid:15) = 0 . . Forapplication performance evaluation, we tested our applicationon an iPhone 6 plus device. A. Image Quality Assessment It can be understood from the earlier explanation thatour algorithm utilizes all available display levels in eachlocal processing. This significantly increases image brightness,especially in dark regions. Our experiments tested differentWDR images to evaluate the image quality of the tone mapped TABLE IIITMQI O VERALL S CORES Image Durand et al. [10] Gu et al. [13] Paris et al. [14] ProposedBristolBridge 0.7921 0.8369 0.8265 ClockBuilding 0.9403 0.9334 0.9224 CrowFootGlacier 0.8187 0.8705 0.9120 DomeBuilding 0.8362 0.9038 0.8480 FribourgGate 0.8877 0.9570 0.9600 MontrealStore 0.8875 0.9692 0.8969 Moraine2 0.8254 0.9387 0.8666 Oaks 0.8525 0.9600 0.8954 StreetLamp 0.9034 0.9368 0.8731 Vernicular 0.8863 0.9171 images. One example is shown in Fig. 6. It shows one imagetone mapped with different algorithms. In reading order, theimages are tone mapped with Durand et al. , Gu et al. , Paris et al. , and the proposed algorithm, respectively. A visualimpression can tell that our algorithm gives an image withhigher brightness value over the other three images. In fact theaverage brightness value for the four images is 93.78, 120.92,98.01 and 122.14 respectively. Despite the fact that the imagegenerated with Gu et al. has similar overall brightness, it lacksglobal contrast and many unwanted details are enhanced whichmakes the image look unnatural.For objective assessment, we use the tone-mapped imagequality index (TMQI) [24] to calculate an overall quality scorethat combines a multi-scale structural fidelity measure and ameasure of image naturalness. The structural fidelity measureis a full-reference assessment based on the structural similarity(SSIM) index. The naturalness measure is a no-referenceassessment based on statistics of good-quality natural images.The results of the naturalness score, structural fidelity score,and the overall score are listed in Table I, Table II and TableIII. The winner algorithm’s score is shown in bold font. In thenaturalness score, our algorithm scores highest in 5 imagesand achieves an average value of 0.8221 for which is highestamong the tested algorithms. In terms of structural similarity,our algorithm wins 9 out of 10 images and achieves an averagescore of 0.9213. In terms of overall quality, our algorithmproduces the best scores for 9 images.Field assessment of the image quality has also been con-ducted using the developed iOS application. Fig. 6 showstwo sets of examples. The first scene is taken in a com-monly seen extreme light condition. The four images withdifferent exposure are shown on the two left columns andthe tone mapped image is shown on the right-most column.Our algorithm produces a bright image with clear details —For example, the outdoor cloud and the frame of skylightwindow are clearly visible. The inner structure of the buildingsuch as stairs and railing are also clearly shown in the tonemapped image. Another scene is a portrait image becausetaking portraits is the most common use of phone camera. Fig. 6: Field test results. The left two columns show fourdifferent exposure images, the right image shows the tonemapped image with proposed algorithm.The bottom image of Fig. 6 shows an example image of aportrait taken by our application. This image is taken underbacklight condition, it is a challenging situation because thestrong background light could make the portrait dark or eveninvisible. The tone mapped image in Fig. 6 shows the portraitclearly and the detail in the background is also well preserved.This experiment indicates that our application is very suitablefor imaging under extreme light conditions. B. Application Performance Assessment Ideally, we hope to compare the application performancewith some other works which are implemented also in mobile-end. However, most tone mapping algorithms reported areimplemented on desktop platforms. It would be unfair to TABLE IVM EASURED P ROCESSING TIME WITH DIFFERENT S ETTINGS Number of scales × 640 720 × × n = 3 s = 3 s = 4 s = 5 n = 4 s = 3 s = 4 s = 5 n = 5 s = 3 s = 4 s = 5 compare implementations on different platforms. Hence, wefocus our analysis on the mobile-end implementation.The proposed algorithm has three parameters, namely thenumber of scales s , the number of bins n and the regularizationterm (cid:15) . A detailed performance analysis should fully considerthe variation of the three parameters. The regularization term (cid:15) is a scalar value which does not affect neither the memoryrequirement nor the total amount of computation, so it willnot affect the performance of our application. However, thenumber of scales affects the computation burden and thenumber of bins n affects the required memory usage. Toevaluate the two parameters s and n ’s effect on the processingspeed, we tested parameter sets, varying s from 3 to 5 and n from 3 to 5. The evaluation results of three commonly usedresolutions × , × and × are shown inTable IV. For identical parameter settings, the processing timeincreases mostly linearly as the image resolution increases. Forexample, the processing times are about ms , ms and ms for the three different resolutions when n = 5 , s = 5 .The influence of the n parameter on the processing time islimited. Under same image resolution with a fixed number ofscales, the processing time only fluctuates in a very limitedrange. In × resolution, the processing time are ms , ms and ms when s equals to and n equalsto 3, 4 and 5, respectively. In × resolution, theprocessing time is ms , ms and ms when s equalsto and n equals to , and , respectively. From Table IV,we can see that the scale parameter s has greater influenceon the application processing time than the n parameter. In × resolution, the application processing time is , and ms when the n is equal to 3 an s is equal to3, 4, 5, respectively. As for the × and × resolution, the processing time can increase about whenscale s changes from 3 to 5. It is easy to conclude that thegreatest factor that affects the application processing time isthe image resolution. Parameter s affects the application timesecondarily while parameter n has only very little influence.A processing time breakdown using two different parametersettings is shown in Fig. 7. Fig. 7 (a) is the processing timebreakdown when n = 5 , s = 5 and image resolution is × . Fig. 7 (b) is the processing time breakdown when n = 3 , s = 3 and image resolution equals to × . We Fig. 7: Computational breakdown of the application.choose the two extreme cases to show the rough proportion ofeach function. Since Eq. 3, Eq. 4 and Eq. 1 & Eq. 2 will belooped for several times, hence, they consumes the majorityof processing time. They account for 23%, 28% and 24% ofthe total processing time in Fig. 7 (a) and 32%, 28% and 19%in Fig. 7(b). The integral histogram and integral images onlyneed to be computed once in our algorithm and they consume19% and 6% of processing time in Fig. 7 (a), 18% 3% in Fig.7(b) respectively. V. C ONCLUSION A tone-mapping algorithm based on integral image andintegral histogram for tone mapping of wide dynamic rangeimages is presented. The algorithm is motivated by the localprocessing feature of the human visual system. It adoptsmultiple receptive fields to combines global image consistencyand local image details into one final image. Quality evaluationas well as field testing were carried out and discussed in detail.In the objective assessment, results showed that the proposedalgorithm performed best in both structural similarity scoreand naturalness score. Hence, highest TMQI index scores wereachieved by our algorithm compared to the three other state ofthe art algorithms. In the application field test, our algorithmalso produced appealing image which displayed scene details.Mobile GPU implementation of the proposed algorithm waspresented, which can perform tone mapping of typical 1080PWDR color images at about 1 second, thus making it suitablefor mobile phone users. A CKNOWLEDGMENTS The authors would like to thank the Alberta Innovates Tech-nology Futures (AITF) and Natural Sciences and EngineeringResearch Council of Canada (NSERC) for supporting thisresearch. R EFERENCES[1] J. Tumblin and H. Rushmeier, “Tone reproduction for realistic images,” IEEE Computer graphics and Applications , vol. 13, no. 6, pp. 42–48,1993.[2] G. Ward, “A contrast-based scalefactor for luminance display,” Graphicsgems IV , pp. 415–421, 1994.[3] F. Drago, K. Myszkowski, T. Annen, and N. Chiba, “Adaptive logarith-mic mapping for displaying high contrast scenes,” in Computer GraphicsForum , no. 3, pp. 419–426. Wiley Online Library, 2003.[4] C. Schlick, “Quantization techniques for visualization of high dynamicrange pictures,” in Photorealistic Rendering Techniques , pp. 7–20.Springer, 1995.[5] J. Van Hateren, “Encoding of high dynamic range video with a modelof human cones,” ACM Transactions on Graphics (TOG) , vol. 25, no. 4,pp. 1380–1399, 2006.[6] A. Benoit, D. Alleysson, J. Herault, and P. Le Callet, “Spatio-temporaltone mapping operator based on a retina model,” Computational colorimaging , pp. 12–22, 2009.[7] E. Reinhard and K. Devlin, “Dynamic range reduction inspired byphotoreceptor physiology,” IEEE Transactions on Visualization andComputer Graphics , vol. 11, no. 1, pp. 13–24, 2005.[8] R. Mantiuk, S. Daly, and L. Kerofsky, “Display adaptive tone mapping,”in ACM Transactions on Graphics (TOG) , vol. 27, no. 3, p. 68. ACM,2008.[9] K. Ma, H. Yeganeh, K. Zeng, and Z. Wang, “High dynamic range imagetone mapping by optimizing tone mapped image quality index,” in , pp.1–6. IEEE, 2014.[10] F. Durand and J. Dorsey, “Fast bilateral filtering for the display ofhigh-dynamic-range images,” in ACM transactions on graphics (TOG) ,vol. 21, no. 3, pp. 257–266. ACM, 2002.[11] Z. Farbman, R. Fattal, D. Lischinski, and R. Szeliski, “Edge-preservingdecompositions for multi-scale tone and detail manipulation,” in ACMTransactions on Graphics (TOG) , vol. 27, no. 3, p. 67. ACM, 2008.[12] K. He, J. Sun, and X. Tang, “Guided image filtering,” in Europeanconference on computer vision , pp. 1–14. Springer, 2010.[13] B. Gu, W. Li, M. Zhu, and M. Wang, “Local edge-preserving multiscaledecomposition for high dynamic range image tone mapping,” IEEETransactions on image Processing , vol. 22, no. 1, pp. 70–79, 2013.[14] S. Paris, S. W. Hasinoff, and J. Kautz, “Local laplacian filters: edge-aware image processing with a laplacian pyramid,” Communications ofthe ACM , vol. 58, no. 3, pp. 81–91, 2015.[15] J. Chen, S. Paris, and F. Durand, “Real-time edge-aware image pro-cessing with the bilateral grid,” ACM Transactions on Graphics (TOG) ,vol. 26, no. 3, p. 103, 2007.[16] A. O. Aky¨uz, “High dynamic range imaging pipeline on the gpu,” Journal of Real-Time Image Processing , vol. 10, no. 2, pp. 273–287,2015.[17] R. Ure˜na, J. M. G´omez-L´opez, C. Morillas, F. Pelayo et al. , “Real-timetone mapping on gpu and fpga,” EURASIP Journal on Image and VideoProcessing , vol. 2012, no. 1, p. 1, 2012.[18] G. W. Larson, H. Rushmeier, and C. Piatko, “A visibility matchingtone reproduction operator for high dynamic range scenes,” IEEETransactions on Visualization and Computer Graphics , vol. 3, no. 4,pp. 291–306, 1997.[19] P. Viola and M. J. Jones, “Robust real-time face detection,” Internationaljournal of computer vision , vol. 57, no. 2, pp. 137–154, 2004.[20] F. Porikli, “Integral histogram: A fast way to extract histograms incartesian spaces,” in Computer Vision and Pattern Recognition, 2005.CVPR 2005. IEEE Computer Society Conference on , vol. 1, pp. 829–836. IEEE, 2005.[21] E. Reinhard, M. Stark, P. Shirley, and J. Ferwerda, “Photographic tonereproduction for digital images,” ACM transactions on graphics (TOG) ,vol. 21, no. 3, pp. 267–276, 2002.[22] P. E. Debevec and J. Malik, “Recovering high dynamic range radiancemaps from photographs,” in ACM SIGGRAPH 2008 classes , p. 31.ACM, 2008. [23] R. Fattal, D. Lischinski, and M. Werman, “Gradient domain highdynamic range compression,” in ACM Transactions on Graphics (TOG) ,vol. 21, no. 3, pp. 249–256. ACM, 2002.[24] H. Yeganeh and Z. Wang, “Objective quality assessment of tone-mappedimages,” IEEE Transactions on Image Processing , vol. 22, no. 2, pp.657–667, 2013.[25] E. Reinhard, W. Heidrich, P. Debevec, S. Pattanaik, G. Ward, andK. Myszkowski,