H. Isil Bozma

Visual processing and classification of items on a moving conveyor: a selective perception approach

Abstract Many industrial applications require some sort of automated visual processing and classification of items placed on a moving conveyor. In this paper, we present a selective perception based approach to visual processing. The novelty of this approach is that instead of processing the whole image, only areas that are deemed ‘‘interesting’’ and hence calling for attention are analyzed. The attentional sequences thus constructed can then be used for a variety of tasks including shape determination. Since only a small portion of the whole image is processed, visual processing can be real-time and flexible without requiring special hardware. Two different applications based on this approach are described. In a defective item detection task, we explain in detail how attentional sequences can be used. As a second application, the approach has been implemented in an automated remote controller sorter in a TV manufacturing plant—thus confirming its practical applicability.

Assembly as a noncooperative game of its pieces: analysis of 1D sphere assemblies

We propose an event-driven algorithm for the control of simple robot assembly problems based on noncooperative game theory. We examine rigorously the simplest setting – three bodies with one degree of freedom and offer extensive simulations for the 2 DOF extension. The initial analysis and the accompanying simulations suggest that this approach may indeed, offer an attractive means of building robust event driven assembly systems.

Bubble space and place representation in topological maps

This paper presents bubble space based representation of “places” (nodes) in topological maps. Bubble space simultaneously provides for detailed (bubble surfaces) and holistic (bubble descriptors) representation of places. It is based on bubble memory where visual feature values and their local S2-metric relations from robot’s viewpoint are simultaneously encoded on a deformable spherical surface. Bubble surfaces extend bubble memory to accommodate varying robot pose and multiple features. They are transformed into bubble descriptors that are rotationally invariant with respect to heading changes while being computable in an incremental manner as each new set of visual observations is made. We use bubble descriptors for place learning and recognition with support vector machines in both indoor and outdoor environments and provide analysis results on recognition, recall and precision rates and time performance including a comparative study with the state-of-the-art descriptors.

Fovea Based Coding for Video Streaming

Attentive robots, inspired by human-like vision – are required to have visual systems with fovea-periphery distinction and saccadic motion capability. Thus, each frame in the incoming image sequence has nonuniform sampling and consecutive saccadic images have temporal redundancy. In this paper, we propose a novel video coding and streaming algorithm for low bandwidth networks that exploits these two features simultaneously. Our experimental results reveal improved video streaming in applications like robotic teleoperation. Furthermore, since the algorithm employs the Gaussian-like resolution of human visual system and is extremely simple to integrate with the standard coding schemes, it can also be used in applications such as cellular phones with video.

Coalition formation games for dynamic multirobot tasks

In this paper we study the problem of forming coalitions for dynamic tasks in multirobot systems. As robots, either individually or in groups, encounter new tasks for which individual or group resources do not suffice, robot coalitions that are collectively capable of meeting these requirements need to be formed. We propose a hybrid approach to this problem where coalitions proceed with the task if they have sufficient resources after liberating redundant members while they report it to a task coordinator in cases where their resources do not suffice. In turn, the task coordinator forms capable coalitions based on coalition formation games in which groups of robots are evaluated together in regards to each task’s required resources and cost of forming a coalition. The resulting coalitions are such that no group of robots has a viable alternative to staying within their assigned coalition. Thus, as new tasks are confronted, coalitions merge and split so that the resulting coalitions are capable of the newly encountered tasks. Simulations and experiments performed on groups of heterogeneous mobile robots demonstrate the effectiveness of the proposed approach.

Long-term topological place learning

In this work, we consider long-term topological place learning and present an approach that enables the robot to learn in an unsupervised, organized and incremental manner. The knowledge associated with the previously visited places is internally stored in the form of bubble descriptor semantic tree (BDST) using the previously proposed bubble space representation. The BDST is generated and maintained without any external supervision. It organizes the learned knowledge where the terminal nodes are viewed as corresponding to distinct places while its structure encodes their semantic hierarchy. In case the robot is not able to recognize a place with its current BDST, it learns it via updating the BDST incrementally based on the hierarchical single link clustering algorithm SLINK. The proposed approach is evaluated experimentally using combined benchmark datasets from indoor and outdoor settings with recognition rates comparable to those of state-of-the-art approaches while the robot is able to retain efficiently and use the knowledge associated with the learned places.

Place representation in topological maps based on bubble space

Place representation is a key element in topological maps. This paper presents bubble space - a novel representation for “places” (nodes) in topological maps. The novelties of this model are two-fold: First, a mathematical formalism that defines bubble space is presented. This formalism extends previously proposed bubble memory to accommodate two new variables - varying robot pose and multiple features. Each bubble surface preserves the local S2-metric relations of the incoming sensory data from the robots viewpoint. Secondly, for learning and recognition, bubble surfaces can be transformed into bubble descriptors that are compact and rotationally invariant, while being computable in an incremental manner. The proposed model is evaluated with support vector machine based decision making in two different settings: first with a mobile robot placed in a variety of locations and secondly using benchmark visual data.

Model reference adaptive approach to sliding mode control

In this paper, model reference adaptive control (MRAC) and sliding mode control (SMC) are combined to obtain the proposed adaptive sliding mode control (ASMC) algorithm which is a new approach to the control of single input nonlinear systems with imprecise models. The contribution of this method is that the constant parameters of the sliding surface are replaced by time-varying parameters whose values are calculated by an adaptation algorithm, which forces the tracking errors to follow the behavior of a reference error model. Simulation results show that the proposed method not only improves the performance of the system but also reduces the chattering problem. Consequently it is expected that the new ASMC algorithm will be suitable for various practical applications.

RGB-D based place representation in topological maps

With the recent developments in sensor technology including Microsoft Kinect, it has now become much easier to augment visual data with three-dimensional depth information. In this paper, we propose a new approach to RGB-D based topological place representation—building on bubble space. While bubble space representation is in principle transparent to the type and number of sensory inputs employed, practically, this has been only verified with visual data that are acquired either via a two degrees of freedom camera head or an omnidirectional camera. The primary contribution of this paper is of practical nature in this perspective. We show that bubble space representation can easily be used to combine RGB and depth data while affording acceptable recognition performance even with limited field of view sensing and simple features.

Reliable topological place detection in bubble space

This paper introduces a novel approach to topo-logical place detection. The approach is based on previously proposed bubble space representation - where all sensory features and their relative S2- geometry are encoded in a manner that is implicitly dependent on robot pose. Its novelty is that ensuring sensory data reliability is integrated with place detection. This is achieved via checking for informativeness, coherence and plenitude using only the bubble space representation of the incoming sensory data. The stringency of these checks is controllable via a set of associated parameters. Experimental results with benchmark datasets indicate correct detection rates comparable to state-of-the-art approaches in place detection. Furthermore, the detected places can then be immediately used to generate the nodes in topological maps.