Design of Novel Algorithm and Architecture for Gaussian Based Color Image Enhancement System for Real Time Applications
DDesign of Novel Algorithm and Architecture forGaussian Based Color Image EnhancementSystem for Real Time Applications
M. C Hanumantharaju , M. Ravishankar and D. R Rameshbabu Department of Information Science & Engineering, Department of Computer Science & EngineeringDayananda Sagar College of Engineering, Bangalore, India. (mchanumantharaju, ravishankarmcn, bobrammysore)@gmail.com
No Institute Given
Abstract.
This paper presents the development of a new algorithm forGaussian based color image enhancement system. The algorithm hasbeen designed into architecture suitable for FPGA/ASIC implementa-tion. The color image enhancement is achieved by first convolving anoriginal image with a Gaussian kernel since Gaussian distribution is apoint spread function which smoothes the image. Further, logarithm-domain processing and gain/offset corrections are employed in order toenhance and translate pixels into the display range of 0 to 255. Theproposed algorithm not only provides better dynamic range compressionand color rendition effect but also achieves color constancy in an image.The design exploits high degrees of pipelining and parallel processingto achieve real time performance. The design has been realized by RTLcompliant Verilog coding and fits into a single FPGA with a gate countutilization of 321,804. The proposed method is implemented using XilinxVirtex-II Pro XC2VP40-7FF1148 FPGA device and is capable of pro-cessing high resolution color motion pictures of sizes of up to 1600 × Keywords:
Gaussian color image enhancement, Serpentine memory, 2DGaussian convolution, Logarithm, Field Programmable Gate Array.
Digital image enhancement [1] refers to accentuation, sharpening of image fea-tures such as edges, boundaries or contrast to make a graphic display more usefulfor display and analysis. The enhanced images with better contrast and detailsare required in many areas such as computer vision, remote sensing, dynamicscene analysis, autonomous navigation and medical image analysis. In the recent a r X i v : . [ c s . A R ] S e p M. C Hanumantharaju et al. years, color image enhancement has been becoming an increasingly importantresearch area with the widespread use of color images. Numerous methods [2]are available in the literature for color image enhancement. The color image en-hancement can be classified into two categories according to the color space: colorimage enhancement in RGB color space and color image enhancement based ontransformed space.The paper is organized as follows: Section 2 gives a brief review of pre-vious work done. Section 3 describes the proposed Gaussian based color im-age enhancement algorithm. Section 4 provides detailed system architecture forhardware realization of color image enhancement algorithm. The results anddiscussions follow these. Conclusion arrived at are presented in the final section.
Digital Signal Processors (DSPs) [3][4] have been employed for enhancement ofimages which provides some improvement compared to general purpose com-puters. Only marginal improvement has been achieved since parallelism andpipelining incorporated in the design are inadequate. This scheme uses opti-mized DSP libraries for complex operations and does not take full advantage ofinherent parallelism of image enhancement algorithm. The neural network basedlearning algorithm [5] provides an excellent solution for the color image enhance-ment with color restoration. The hardware implementation of these algorithmsparallelizes the computation and delivers real time throughput for color imageenhancement. However, its window related operations such as convolution, sum-mation and matrix dot products in an image enhancement architecture demandsenormous amount of hardware resources.Hiroshi Tsutsui et al. [6] proposed an FPGA implementation of adaptivereal-time video image enhancement based on variational model of the Retinextheory. The authors have claimed that the architectures developed in this schemeare efficient and can handle color picture of size 1900 × itle Suppressed Due to Excessive Length 3 adaptive and reusable hardware architecture for image enhancement. However,the histogram modification used in this scheme treats all regions of the imageequally and often results in poor local performance, which in turn affects the im-age details. The modified luminance based multiscale retinex algorithm proposedby Tao et al. [11] achieves optimal enhancement result with minimal complexityof hardware implementation. However, the algorithm works fine so long as thebackground is dark and the object is bright.The limitations mentioned earlier are overcome in the proposed method inan efficient way. To start with, the input image is convolved with 5 × In order to account for the smoothness, lightness, color constancy and dynamicrange properties of Human Visual System (HVS), Gaussian based image en-hancement algorithm has been proposed. The basic operational sequence of theproposed Gaussian based color image enhancement algorithm is shown in Fig. 1.To start with, the original image (which is of poor quality and needing enhance-ment) is read in RGB color space. The color components are separated followedby the selection of window size as 5 × × Convolution is a simple mathematical operation which is essential in many imageprocessing algorithms. The color image enhancement algorithm proposed in thispaper uses 5 × M. C Hanumantharaju et al.
Fig. 1.
Flow Sequence of the Proposed Gaussian based Color Image EnhancementAlgorithm image is convolved with Gaussian kernel of size 5 ×
5. The two dimension (2D)Gaussian function is defined by Eqn. (1): g ( x, y ) = 12 πσ e − x y σ (1)where σ is the standard deviation of the distribution; x and y are spatialcoordinates.The basic principle of Gaussian filters is to use 2D distribution as a pointspread function. This property is adopted in image enhancement application andis achieved by convolving the 2D Gaussian distribution function with the originalimage. A Gaussian kernel can be a 3 × × Fig. 2.
Gaussian Kernels: 3 × × The Gaussian convolution matrix is given by Eqn. (2). G ( x, y ) = I ( x, y ) ⊗ g ( x, y ) (2)where ⊗ denotes convolution, g(x, y) is a Gaussian kernel, I(x, y) is theoriginal image and G(x, y) is the convolved output.Mathematically, 2D convolution can be represented by the following Eqn.(3): G ( x, y ) = M (cid:88) i =1 N (cid:88) j =1 I ( i, j ) × g ( x − i, y − j ) (3) itle Suppressed Due to Excessive Length 5 where m = 5 and n = 5 for 5 × × × P ( x, y ) = (cid:80) i =0 W i × P i (cid:80) i =0 W i (4)where W i indicates the 5 × P i is the 5 × ×
3. Theconvolution mask of 5 × Fig. 3.
Two Dimensional Convolution Scheme
Next, the log transformation operation applied on the image compresses thedynamic range of gray level input values to manageable levels for further pro-cessing. The logarithmic processing on a 2D image is carried out by using Eqn.(5): G L ( x, y ) = K × log [1 + G ( x, y )] (5)where K is a constant arrived as 1.5 after conducting a number of experimentswith various test images. Further, the logarithm computed image is scaled by ascaling factor of 32 for getting pixel values in the range of 0 to 255. The finalstep in proposed approach is gain/offset correction. This gain/offset correctionis accomplished by using Eqn. (6). I (cid:48) ( x, y ) = d max G Lmax − G Lmin [ G L ( x, y ) − G Lmin ] (6)where d max is the maximum intensity which is chosen as 255 for an imagewith 8-bit representation, G L ( x, y ) is the log-transformed image, G Lmin is theminimum value of log transformed image, G Lmax is the maximum value of log
M. C Hanumantharaju et al. transformed image, I (cid:48) ( x, y ) is the enhanced image and, x and y represent spatialcoordinates. This section presents the architectures of the proposed color image enhancementsystem. It is composed of several components such as serpentine memory, 2Dconvolution, logarithm and gain/offset corrections.
The block diagram of a top level Gaussian based color image enhancement systemshown in Fig. 4, which performs a color image enhancement operation as impliedin the name. The detailed signal descriptions of this block are provided in TableI. The inputs ”rin”, ”gin”, and ”bin” are the red, green, and blue pixels, eachof size 8-bits. The input pixels are valid at the positive edge of the clock. Theoutputs ”ro”, ”go” and ”bo” represent the enhanced pixels of proposed imageenhancement system. The ”pixel valid” signal is asserted, When the enhancedpixel data is valid. The enhancement system can be reset at any point of timeby asserting the asynchronous, active low signal, ”reset n”.
Fig. 4.
Block Diagram of Top Level Gaussian Based Color Image Enhancement System
The architecture proposed for color image enhancement system consists oftwelve modules, each color component (R/G/B) comprising four basic modules:Sliding window, 2D Gaussian Convolution, Logarithm Base-2, and Gain/OffsetCorrection as shown in Fig. 5. Pipelining and parallel processing techniqueshave been adopted in the design in order to increase the processing speed of theproposed system.
The signal diagram of a serpentine memory for each of the color components R,G and B is presented in Fig. 6(a). The serpentine memory is commonly referred itle Suppressed Due to Excessive Length 7
Table 1.
Signal Description for the Top Module of Gaussian Based Image EnhancementSystem
Signals Description clk This is the global clock signalreset n Active low system resetrin [7:0] Red color componentgin [7:0] Green color componentbin [7:0] Blue color componentro [7:0] Enhanced red color componentgo [7:0] Enhanced Green color componentbo [7:0] Enhanced Blue color componentpixel valid Valid signal for enhanced RGB pixel
Fig. 5.
Architecture of Gaussian Based Color Image Enhancement System M. C Hanumantharaju et al. to as a sliding window. The proposed method uses 5 × × (a) (b) Fig. 6. (a) Signal Diagram of Serpentine Memory for R/G/B Color Channels (b)Schematic Diagram of a 5 × In this work, one pixel is read from memory in one clock cycle with no latency.The pixels are read row by row in a raster scan order. For a 5 × The design of the 2D Gaussian convolution processor in hardware is more dif-ficult than that of the sliding window. The convolution algorithm uses adders,multipliers and dividers in order to calculate its output. The signal diagram ofthe 2D Gaussian convolution processor is shown in Fig. 7.The input pixels W11 to W55 are scaled to 16 bits in order to match theGaussian coefficients. The input pixels are valid on the positive edge of the”clk” with ”window valid” asserted. The coefficient values of the convolutionmask employed in most of the image processing applications remain constant forthe entire processing. Therefore, constant coefficient multiplier is employed inorder to implement convolution efficiently. The twenty five Gaussian coefficientsG11 [15:0] to G55 [15:0] represent the convolution mask shown earlier in Fig. 2. itle Suppressed Due to Excessive Length 9
Fig. 7.
Signal Diagram of 2D Gaussian Convolution Processor for R/G/B Color Chan-nels
The output signal ”conv out” is 8-bits in width and is valid only on the positiveedge of signal, ”conv valid”.
The signal diagram of logarithm base-2 module for R/G/B color channels isshown in Fig. 8(a). The convolution output comprises the input for the logarithmblock. The architecture for logarithmic module is shown in Fig. 8(b). (a) (b)
Fig. 8. (a) Signal Diagram of Logarithm Base-2 Module for R/G/B Color Channels(b) Architecture of Logarithm Base-2 Processor for R/G/B Color Channels
The log of base-2 is computed in two steps: Integer portion computation andfractional portion computation. Accordingly, the architecture consists of twomodules.
The gain/offset correction is accomplished using Eqn. (6). The hardware imple-mentation of gain and offset correction uses a multiplier and subtractor module.This step steers pixels into the correct display range: 0 to 255. The multiplierdesign presented in this work incorporates a high degree of parallel circuits andpipelining of five levels. The multiplier performs the multiplication of two 8-bitsunsigned numbers n1 and n2 as shown in Fig. 9(a). The multiplier result is ofwidth 16-bits. The detailed architecture for the multiplier is shown in Fig. 9(b). (a) (b)
Fig. 9. (a) Signal Diagram of a Multiplier Block for R/G/B Color Channels(b) DetailedArchitecture of Pipelined Multiplier Design for R/G/B Color Channels
Over the years, many image enhancement techniques have been reported, majorones being Multiscale Retinex with Color Restoration. The Multiscale Retinexis the most popular technique that works well under most lighting conditions. Inthis paper, the most popular image enhancement techniques namely, histogramequalization[12], MSRCR [13] and improved MSRCR [14] are chosen in order tovalidate the proposed scheme.The image enhancement algorithms mentioned earlier have been coded inMatlab by the present authors and the reconstructed images are shown in Fig.10. The original image of resolution 256 ×
256 pixels is shown in Fig. 10(a). Theimage enhancement using conventional histogram equalization of R, G and Bchannels is shown in Fig. 10(b). The histogram equalization method improvesthe global contrast of the original image. However, the overexposed regions ofthe original image are highly enhanced in this approach. Image enhancementbased on Multiscale Retinex with Color Restoration is shown in Fig. 10(c). Theenhanced image obtained by MSRCR is better compared with histogram equal-ization. MSRCR method fails to produce good color rendition for a class ofimages that contain violation of gray world assumption. itle Suppressed Due to Excessive Length 11
The image enhanced based on improved multiscale retinex using hue-saturation-value (HSV) color space shown in Fig. 10(d) improves the visual quality betterthan MSRCR method. However, this approach is complex from computationpoint of view. The limitations expressed in the above algorithms are conspic-uous by its absence in the proposed Gaussian based color image enhancementmethod as can be seen from the result displayed in Fig. 10(e). The proposed algo-rithm outperforms the other image enhancement techniques in terms of qualityof the reconstructed picture. The images enhanced by our method are clearer,more vivid, and more brilliant than that achieved by other researchers. (a) (b) (c) (d) (e)
Fig. 10.
Comparison of Reconstructed Pictures Using Image Enhancement Algorithms:(a) Original Image of resolution 256 ×
256 pixels (b) Image enhanced based on His-togram Equalization of Ref. [12], PSNR: 31.2 dB (c) Multiscale Retinex with ColorRestoration of Ref. [13], PSNR: 30 dB (d) Improved Multiscale Retinex in HSV ColorSpace of Ref. [14], PSNR: 29.5 dB (e) Proposed Gaussian Color Image Enhancement,PSNR: 31.8 dB.
The proposed FPGA implementation of Gaussian based Color Image En-hancement System has been coded and tested in Matlab (Version 8.1) first inorder to ensure the correct working of the algorithm. Subsequently, the com-plete system has been coded in Verilog HDL so that it may be implementedon an FPGA or ASIC. The proposed scheme has been coded in RTL compliantVerilog and the hardware simulation results are compared with Matlab resultsdescribed earlier in order to validate the hardware design. The system simu-lation has been done using ModelSim (Version SE 6.4) and Synthesized usingXilinx ISE 9.2i. The algorithm has been implemented on Xilinx Virtex-II ProXC2VP40-7FF1148 FPGA device. In the proposed work, window size is chosenas 5 × ×
256 pixels. The images (b), (e) and (h) of Fig. 13 show theenhanced images using the proposed method based on Matlab approach. Theimages in Fig. 11 (c), (f) and (i) show the enhanced images using the proposedGaussian method based on hardware (Verilog) approach.
Fig. 11.
Experimental Results for Gaussian based Color Image Enhancement System:(a) Original couple image (256 ×
256 pixels) (b) Reconstructed couple image usingMatlab, PSNR: 36.9 dB (c) Reconstructed couple image using Verilog, PSNR: 36.2dB. (d) Original dark road image (256 ×
256 pixels) (e) Reconstructed dark road imageusing Matlab, PSNR: 40.8 dB (f) Reconstructed dark road image using Verilog, PSNR:40.3 dB. (g) Original eyeball image (256 ×
256 pixels) (h) Reconstructed eyeball imageusing Matlab, PSNR: 36.8 dB (i) Reconstructed eyeball image using Verilog, PSNR:36.1 dB
The ModelSim simulation waveforms for inputting the image pixel data for allthe three color components is shown in Fig. 12(a). The enhancement processstarts when the ”din valid” signal is asserted at the positive edge of the clock.The RGB image is separated into R, G and B data and the algorithm is appliedconcurrently to all the color components. The reconstructed pixels are issued atthe output pins ”dout” with ”dvm” signal asserted as presented in Fig. 12(b).The outputs are issued out continuously one pixel every clock cycle after a la-tency of 535 clock cycles.
The design was Synthesized, Placed and Routed using Xilinx ISE 9.2i. TheFPGA target device chosen was Xilinx Virtex-II Pro XC2VP40-7FF1148. Thesynthesis and place and route results for the design are presented in the following.The ISE generated RTL view of the top level module ”Gaussian IE” is shownin Fig. 13(a) and the zoomed top view modules are presented in Fig. 13(b). Thedetailed zoomed view of R/G/B processor is shown in Fig. 14.The device utilization and the performance summary for Gaussian basedcolor image enhancement system are presented in Fig. 15. The system utilizesabout 321,804 gates as reported by the ISE tool. The maximum frequency ofoperation is 224.840 MHz. This works out to a frame rate of 117 per second itle Suppressed Due to Excessive Length 13(a) (b)
Fig. 12. (a) Waveforms for Gaussian Based Color Image Enhancement System: In-putting Image Data (b) Waveforms for Gaussian based Color Image EnhancementSystem: Enhanced Pixel Data(a) (b)
Fig. 13.
Experimental Results for Gaussian based Color Image Enhancement System:(a) RTL View of the Top Module ”Gaussian IE” (b) Zoomed View of ”Gaussian IE”Module where
U1:
Red Color,
U2:
Green Color and
U3:
Blue Color ComponentProcessors
Fig. 14.
Zoomed View of U1 or U2 or U3 Module.4 M. C Hanumantharaju et al. for a picture size of 1600 × Fig. 15.
FPGA Resource Utilization for Gaussian Based Color Image EnhancementDesign
The timing summary for the design as reported by Xilinx ISE tool is asfollows:Speed Grade : -7Minimum period: 4.448ns (Maximum Frequency: 224.840 MHz)Minimum input arrival time before clock: 1.418nsMaximum output required time after clock: 3.293nsClock period: 4.448ns (frequency: 224.840 MHz)Total number of paths /destination ports: 68988 / 30564Delay: 4.448ns (Levels of Logic = 12)
A new algorithm and architecture for Gaussian based color image enhancementsystem for real time applications has been presented in this paper. The Gaussianconvolution used in this scheme not only smoothes the image but also removesthe noise present in the image. Further, the pixels are processed in log-domain inorder compress the dynamic range. Finally, linear operation such as gain/offsetcorrection is used in order to obtain image in display range. The Verilog designof the color image enhancement system is implemented on Xilinx Virtex-II ProXC2VP40-7FF1148 FPGA device and is capable of processing high resolutionvideos up to 1600 × itle Suppressed Due to Excessive Length 15 References
1. Anil K. Jain.:Fundamentals of Digital Image Processing, Prentice-Hall, Inc. UpperSaddle River, NJ, USA (1989).2. An, C., and Yu, M.:Fast Color Image Enhancement based on Fuzzy Multiple-scaleRetinex, In: Strategic Technology (IFOST-2011), 6 thth