Naoshi Kondo

A Machine Vision for Tomato Cluster Harvesting Robot

Dutch style greenhouses for tomato production are recently becoming popular in many countries while fruit cluster harvesting is also becoming popular in the Netherland and other countries where the Dutch system is introduced due to higher workability and fruit freshness. In the large scale Dutch production system, it is desirable to replace human operations into automated machines. In this paper, a machine vision system for a tomato fruit cluster harvesting robot is described. This machine vision system consisted of two identical color TV cameras (VGA class), four lighting devices with PL filters, and two image capture boards. Two images were acquired at a time and RGB color component images were converted into HSI images. Using colors on the HSI images, main stems, peduncles, and fruits were discriminate and an end-effector grasping point on the main stem was recognized based on physical properties of the tomato plant. Since difficulty to recognize the grasping point depended on exposure of plant parts and on robot access angle, acquired images were classified into three groups; Group A was images in which the fruit cluster, the stem, and the peduncle were isolated from the other plant parts. Group B was images in which they existed with adjacent plant parts. Group C was images in which some of them were occluded. From an experiment, results showed that 73% of grasping points on main stems were successfully recognized excluding Group C which was not able to be recognized also by human eyes.

n End-Effector and Manipulator Control for Tomato Cluster Harvesting Robot

An end-effector and a control method for tomato-cluster harvesting manipulator are proposed in this study. When fruit cluster harvesting is conducted, peduncle direction is necessary to cut, but is not easy to detect because peduncles are often occluded by leaves, stems and fruits. The end-effector needs to grasp the peduncle without its direction information. An end-effector which can surround main stem and can grasp and cut the peduncle by fingers was made as a trial. When a tomato cluster is transported into a container with the manipulator, both its transportation speed and vibration damping are required. Such a control problem is generally called a motion and vibration control (MOVIC). An input shaping method is one of the representative control methods for the MOVIC. It requires accurate natural frequencies of the manipulating target fruit cluster to damp the flexible vibration when the robot is accelerating the target. The tomato clusters, however, have individual variation with natural frequencies; hence, it is not easy to apply the input shaping method directly. To overcome this problem, identification method of natural frequency was combined with the input shaping method in the proposed method. This identification was based on real time sensing data from a machine vision and a force sensor and database of physical properties of the tomato clusters. Usefulness of the proposed method was verified through both numerical simulations and hardware experiments.

Strawberry Harvesting Robot on Table-top Culture

In this paper, it is reported that a robot was developed for harvesting strawberry grown on table top culture. The robot mainly consisted of a 4 DOF manipulator, a harvesting end-effector using sucking force and a visual sensor. As its manipulator, a Cartesian coordinate type was adopted and it was suspended under the planting bed of strawberry. The robot was capable of moving along the planting bed without a traveling device because one prismatic joint of the manipulator played the role of a traveling device. The end-effector could suck a fruit using a vacuum device and it could compensate detecting errors caused by the visual sensor. The visual sensor gave the robot two dimensional information based on an acquired image and fruit depth was calculated as an average value of previously harvested fruit depths obtained from end-effector positions when the robot actually harvested. The end-effector moved toward a target fruit based on the three dimensional position of the target fruit until the fruit was detected by three pairs of photo-interrupters on sucking head. After cutting the peduncle by using the robots wrist joint, the fruits passed the tube and were transported to the tray. From the results of the harvesting experiments, it was observed that the robot could harvest all target fruits with no injury, and that depth measurement by a visual sensor was simplified because a distance between the robot and the fruits was kept an approximately constant by suspending the robot under the planting bed.

Strawberry harvesting robot for fruits grown on table top culture

We have been developing strawberry harvesting robot for harvesting in a table top culture greenhouse. A strawberry harvesting robot consisted of a 3 DOF manipulator, an end-effecter, machine vision system and a traveling device. An end-effecter, which consisted of a suction head connected to a blower and tow fingers could hold on fruit by suction head and cut and grasp peduncle by two fingers rotate according to peduncle inclination. Machine vision system, which consisted of three identical color cameras (stereo vision system and center camera) and five LED lighting device could calculate fruit’s location of 3 dimension and recognize target fruit and peduncle details. From harvesting experiments at May 2007, it was observed that 38% of fruits were harvested. Corresponding problems were miss-stereo matching and false recognition of fruit and peduncle.

Three Dimensional Sensing System Using Laser Scanner

A three-dimensional sensing system that consisted of a laser scanner and a lift was ndeveloped for agricultural robot to capture precise external information. The laser scanner was nmoved vertically by the lift to collect three-dimensional distance information around the robots. In the nexperiments by using a laser scanner and a color camera, each tomato fruit in a same cluster was ndiscriminates successfully. Furthermore, the sensing system detected not only objects but also nhuman motion around a robot when infrared sensors were added to the sensing system.

Feasibility of using Polarizing Filters to reduce Halation Effects during Image Acquisition in the Field

Machine vision is widely applied in automated inspection and monitoring systems. One of nthe most important components of machine vision systems is illumination. To remove and avoid nhalation on images, indirect lighting devices such as diffusers or domes with several lamps have nbeen studied for post-harvest operations. However, it is difficult to control natural illumination in the nfield, because sunlight constitutes a point source, which causes shiny and shady parts on objects, nand its intensity fluctuates with time. In addition, most parts of plants have a cuticular layer, causing nhalation effects, which reduce image quality. nIn this study, a tractor-mounted imaging system was developed to acquire high quality crop nimages. The system consisted of a frame box (1m X 1m X 1m), covered by a polarizing film (1m X n1m), 2 identical cameras in the box, and a PC with two image capture boards. All 4 sides of the nframe box were covered with black curtains so that only polarized sunlight could enter. Polarizing nfilters were also fitted to the lens of one camera to compare 2 images from the cameras at a time. In nexperiments, images of coffee plants were acquired under varying sunlight conditions. From the nresults, it was observed that the polarizing filtering technique was capable not only reduce by 97% of nhalation on the surfaces of leaves but also of achieving high accuracy color representation of the nleaves.

3-D Maps of Japanese Citrus Information for Precision Agriculture

Most of mandarin oranges (Citrus Unshu) are manually harvested and sorted sorted based on size, color, shape and internal qualities at cooperative grading facilities in Japan. 80% of the oranges are marketed as fresh fruits. Since higher quality oranges can be given higher prices, it is important to produce high quality fruits and to grade them properly. It is known that fruit quality in each tree is different from place to place because environmental conditions are different even in a same orchard. There are not many researches on this variability from a viewpoint of precision agriculture in Japanese orange production. In this research, the 2-D map and 3-D map were developed in order to create the visualized maps of features on color, size, sugar content, yield, canopy size, and others for optimum tree management by measuring the qualities of the harvested fruits using a mobile citrus fruit grading machine. This machine could travel in citrus orchards and measure appearance and internal qualities of fruits harvested by human in front of citrus trees. 2-D maps can show a relationship between 2 types of data. 3-D map can show the relationships among 3 features. It was observed that 2-D maps were effective for overlaying aerial images and a soil information map and that 3-D maps were effective for tree management changing with complicated environmental conditions. In conclusion, field maps using circles and cylinders were effective for tree management in Japanese agriculture.

Machine vision system for detecting tomato fruits

. Recently, tomato production in large-sized greenhouse facilities has been popular in Japan. In large-scale greenhouse, however, harvesting tomatoes by hand is tough work. Therefore, robotic agricultural system has been addressed. In order to promote robotic agricultural system, it is important to develop the machine vision system.

An illumination system for machine vision inspection of agricultural products

An effect of PL filter on imaging that eliminates halation on object surface is well known. nHowever, it has not been easy to use the filter in front of halogen, incandescent, or HID lamps which nradiate a lot of heat, because PL film is melted with a temperature of about 60 degree centigrade. nAlthough fluorescence lamps or LEDs can be used with PL filters, their intensity and color rendering nare inferior to them. Diffusers or domes with reflection plates are often used when halogen lamps nare used, but unavoidable halation and surroundings reflection occur on glossy surface. nIn this presentation, a direct lighting device (called DL) developed for fruit grading systems is nintroduced. DL mainly consisted of a PL filter, two heat absorption filters, a halogen lamp, and a fan. nSince the absorption wavelength bands of the two filters were different, major heat did not reach the nPL filter. In addition, cold air was introduced to the PL filter by its fan. nMost parts of plants have cuticular layers on their surfaces to keep moisture within themselves. In nthis study, three categories are created for objects corresponding to degree of gloss on surfaces and nsurface materials: category 1 (apples, tomato, eggplants, bell peppers), category 2 (pear, kiwi, npeach), and category 3 (potatoes, radish). A color CCD camera was set at the top of object with four nDLs and images were acquired. To compare the PL filtering images, a dome and diffusers were also nused for the image acquisition. From experiments through three types of illumination devices, it was nobserved that PL filtering images not only could eliminate halation on products of the category 1 but nalso could express true colors of the products. Microscopic images showed proportional relation nbetween thickness of cuticular layers and halation on surfaces.

A double image acquisition system with visible and UV LEDs for citrus fruit

There are many types of citrus fruit grading machine with machine vision capability. While most of them sort fruit by size, shape, and color, detection of rotten fruit remains challenging because their appearances are similar to normal parts. Objectives of this research were to investigate if fluorescence would be a good indicator of the fruit rot, and to develop an economical solution to add the rot inspection capability to an existing machine vision fruit inspection station. A machine vision system consisting of a pair of white and ultra violet (UV) LED lighting devices and a color CCD camera was proposed for the citrus fruit grading task. Since the time lag between the color and fluorescence image captures was short (14ms), it was possible to inspect color, shape, size, and rot of a fruit on the move before it leaves an existing industrial inspection chamber.