Fruits yield estimation using Faster R-CNN with MIoU

Abstract

Fruit yield estimation is one of the challenging tasks using on-plant images to support smart farming and provide information about the product so that storage and export facility can be arranged. The detection and counting of on-plant fruits is a challenging task in complex vision. In this paper, we modify the intersection of union (IoU) in original Faster R-CNN (FR-CNN) for on-plant fruit detection. The modified IoU (MIoU) introduces good distance metric with the minimum area containing ground truth and predicted bounding box. Again, the MIoU pays extra attention to overlapping areas, which overcome the inefficiency of original R-CNN and enhance the detection accuracy. The proposed FR-CNN with MIoU achieved the correlation coefficient (R2) of mango, pomegranate, tomato, apple & orange are 0.98,0.92, 0.96, 0.98 & 0.95 respectively with the variation of imaging condition. In the same image sample, FR-CNN achieved the correlation coefficient (R2) of mango, pomegranate, tomato, apple & orange are 0.81,0.91,0.89, 0.90 & 0.92 respectively. So, the FR-CNN with MIoU enhances the detection accuracy compared to original FR-CNN. Again, the F1 score for apple, orange, tomato, pomegranate and mango is 0.9534, 0.9794, 0.9424, 0.9534 and mango 0.9383 respectively. In addition, the proposed method is efficient enough with less complex compared to state-of-art models for fruit detection. Again, the proposed methodology is evaluated using other state-of-art fruits datasets, namely ACFR dataset and KFuji RGB-DS dataset. The proposed methodology achieved F1 score of 0.9523 & 0.9432 for yield estimation of apple & mango of ACFR dataset and 0.8912 for KFuji RGB-DS dataset.