Zhenglin Wang

On-Tree Mango Fruit Size Estimation Using RGB-D Images

In-field mango fruit sizing is useful for estimation of fruit maturation and size distribution, informing the decision to harvest, harvest resourcing (e.g., tray insert sizes), and marketing. In-field machine vision imaging has been used for fruit count, but assessment of fruit size from images also requires estimation of camera-to-fruit distance. Low cost examples of three technologies for assessment of camera to fruit distance were assessed: a RGB-D (depth) camera, a stereo vision camera and a Time of Flight (ToF) laser rangefinder. The RGB-D camera was recommended on cost and performance, although it functioned poorly in direct sunlight. The RGB-D camera was calibrated, and depth information matched to the RGB image. To detect fruit, a cascade detection with histogram of oriented gradients (HOG) feature was used, then Otsu’s method, followed by color thresholding was applied in the CIE L*a*b* color space to remove background objects (leaves, branches etc.). A one-dimensional (1D) filter was developed to remove the fruit pedicles, and an ellipse fitting method employed to identify well-separated fruit. Finally, fruit lineal dimensions were calculated using the RGB-D depth information, fruit image size and the thin lens formula. A Root Mean Square Error (RMSE) = 4.9 and 4.3 mm was achieved for estimated fruit length and width, respectively, relative to manual measurement, for which repeated human measures were characterized by a standard deviation of 1.2 mm. In conclusion, the RGB-D method for rapid in-field mango fruit size estimation is practical in terms of cost and ease of use, but cannot be used in direct intense sunshine. We believe this work represents the first practical implementation of machine vision fruit sizing in field, with practicality gauged in terms of cost and simplicity of operation.

A Study of Video Coding by Reusing Compressive Sensing Measurements

Compressive Sensing (CS) is gaining popularity in video codec applications because of its low-complexity encoding procedure. However, traditional motion estimation is unable to be adopted to reduce the inter-frame redundancy in CS. Therefore, how to reduce the inter-frame redundancy is becoming the top priority for CS. In this article, similar redundancy is also discovered in excessive CS measurements via analyzing the relationship between the CS measurements and the original signal. Consequently, a proposed scheme is to conditionally reuse CS measurements to reduce the redundancy among CS measurements and increase video compression ratio. The experimental results show the proposed scheme can increase 20% compression rate. At the same time, the proposed scheme still maintains the low-complexity characteristic.

Sorted Random Matrix for Orthogonal Matching Pursuit

Orthogonal Matching Pursuit (OMP) algorithm is widely applied to compressive sensing (CS) image signal recovery because of its low computation complexity and its ease of implementation. However, OMP usually needs more measurements than some other recovery algorithms in order to achieve equal-quality reconstructions. This article firstly illustrates the fundamental idea of OMP and the specific algorithm steps. And then, two limitations leading to the previous issue are addressed. Finally, a sorted random matrix is proposed to be used as a measurement matrix to improve those two limitations. The experimental results show the proposed measurement matrix is able to help OMP make a great progress on the quality of recovered approximations.

Machine vision assessment of mango orchard flowering

Abstract Machine vision assessment of mango orchard flowering involves detection of an inflorescence (a panicle) with flowers at various stages of development. Two systems were adopted contrasting in camera, illumination hardware and image processing. The image processing paths were: (i) colour thresholding of pixels followed by SVM classification to estimate inflorescence associated pixel number (panicle area), and panicle area relative to total canopy area (‘flowering intensity’) using two images per tree (‘dual view’), and (ii) a faster R-CNN for panicle detection, using either ‘dual-view’ or ‘multi-view’ tracking of panicles between consecutive images to achieve a panicle count per tree. The correlation coefficient of determination between the machine vision flowering intensity and area estimate (path i) and in field human visual counts of panicles (past ‘asparagus’ stage) per tree was 0.69 and 0.81, while that between the machine vision (path ii) and human panicle count per tree was 0.78 and 0.84 for the dual and multi-view detection approaches, respectively (n = 24), while that for repeat human counts was 0.86. The use of such information is illustrated in context of (i) monitoring the time of peak flowering based on repeated measures of flowering intensity, for use as the start date within heat sum models of fruit maturation, (ii) identification and mapping of early flowering trees to enable selective early harvest and (iii) exploring relationships between flowering and fruit yield. For the current orchard and season, the correlation coefficient of determination between machine vision estimates of panicle area and multi-view panicle count and fruit yield per tree was poor (R2 of 0.19 and 0.28, respectively, n = 44), indicative of variable fruit set per panicle and retention between trees.

Backprojection regularization with weighted ramp filter for tomographic reconstruction

Although filtered backprojection (FBP) is popular, backprojection then filtering (BPF) still receives a few attentions. Usually, BPF is inferior to FBP in terms of reconstruction quality. There are two main causes. First, BPF has to use a 2-dimensional discrete ramp filter formed by sampling the continuous ramp filter, resulting in DC shift and aliasing artefacts. Second, the common ramp filter amplifies high frequency noise much. To address such two issues, a weighted ramp filter is investigated to reduce the amplification of high frequency noise, and then a total-variation based backprojection regularization (BPR) method is developed to mitigate the DC shift and improve the robustness to noise. The experimental results show that BPR outperforms FBP for low-dose CT imaging reconstruction.

In Field Fruit Sizing Using A Smart Phone Application

In field (on tree) fruit sizing has value in assessing crop health and for yield estimation. As the mobile phone is a sensor and communication rich device carried by almost all farm staff, an Android application (“FruitSize”) was developed for measurement of fruit size in field using the phone camera, with a typical assessment rate of 240 fruit per hour achieved. The application was based on imaging of fruit against a backboard with a scale using a mobile phone, with operational limits set on camera to object plane angle and camera to object distance. Image processing and object segmentation techniques available in the OpenCV library were used to segment the fruit from background in images to obtain fruit sizes. Phone camera parameters were accessed to allow calculation of fruit size, with camera to fruit perimeter distance obtained from fruit allometric relationships between fruit thickness and width. Phone geolocation data was also accessed, allowing for mapping fruits of data. Under controlled lighting, RMSEs of 3.4, 3.8, 2.4, and 2.0 mm were achieved in estimation of avocado, mandarin, navel orange, and apple fruit diameter, respectively. For mango fruit, RMSEs of 5.3 and 3.7 mm were achieved on length and width, benchmarked to manual caliper measurements, under controlled lighting, and RMSEs of 5.5 and 4.6 mm were obtained in-field under ambient lighting.

Automated mango flowering assessment via refinement segmentation

An automated flowering assessment system for mango orchards was proposed. Segmentation of flowers from a complex background (i.e. leaves, branches and ground) was achieved based on (i) colour correction via adjustment of the brightness and contrast to a reference level, to rectify the illumination variability spatially within and between images; (ii) colour thresholding with fixed thresholds to separate flowers, although with some branches and trunks; and (iii) SVM classification to refine the segmentation results, removing the branch and trunk errors. Mango tree canopy images (n=160) were acquired during a five-week flowering period, with 15 of the images used in calibration and 145 used in validation. The proposed method had a good correlation with human scoring, with coefficient of determination (R2) of 0.87.

Iterative Weighted DCT-SVD for Compressive Imaging

This paper proposes iterative weighted discrete cosine transform and singular value decomposition (DCT-SVD) transform for compressive sensing (CS) reconstruction. The idea of weight utilizes the priori that the components of the transform representation of an image usually are unequally important. Sequentially, larger weights are assigned to more important components to improve reconstruction quality. Besides, iterative DCT-SVD can be regarded as a sequence of adaptive transforms. DCT starts a recovery procedure as an initial transform. SVD is then performed on previous reconstruction to obtain a pair of transform bases for next recovery, and the mechanism is repeated until the reconstructions remain unchanged. The proposal does not introduce extra cost to CS sampling, but improves reconstruction quality much according to the numerical simulations.

Interleaving and Sparse Random Coded Aperture for Lens-Free Visible Imaging

Coded aperture has been applied to short wavelength imaging (e.g., gamma-ray), and it suffers from diffraction and interference for taking longer wavelength images. This paper investigates an interleaving and sparse random (ISR) coded aperture to reduce the impact of diffraction and interference for visible imaging. The interleaving technique treats coded aperture as a combination of many small replicas to reduce the diffraction effects and to increase the angular resolution. The sparse random coded aperture reduces the interference effects by increasing the separations between adjacent open elements. These techniques facilitate the analysis of the imaging model based only on geometric optics. Compressed sensing is applied to recover the coded image by coded aperture, and a physical prototype is developed to examine the proposed techniques.