Barna Reskó

Virtual Collaboration Arena

This paper presents a system for high level collaboration between humans and intelligent agents in a virtual reality. The proposed system is capable of creating a virtual world including its visualization and simulation of Newtonian physics. Furthermore, with voice recognition and synthesization conversations can be made with intelligent agents in a user friendly way. The proposed system is able to connect the different communication modalities of the human user and the intelligent agents, allowing to control them using the brain-in-the-loop control method. Finally the capabilities of the system is demonstrated on two examples.

Reference signal control of the TORA system: a TP model transformation based approach

This paper presents a case study of the TP (tensor product) model transformation in the control of a nonlinear benchmark problem. We design a nonlinear controller of translational oscillation with an eccentric rotational proof mass actuator (TORA) system via TP model transformation and LMI (linear matrix inequality) based controller design technique that is also capable of the reference signal tracking control. The main contribution of the paper is to show that both numerical methods the TP model transformation and the LMI can readily be executed by computer independently on the given problem and without analytical derivations, which, hence, leads to a fast way of controller designs for a class of engineering control problems. Numerical simulation is used in the paper to provided empirical validation of the control results.

A cognitive robot supervision system

This paper applies new cognitive infocommunication channels in human-machine interaction to develop a new paradigm of robot teaching and supervision. The robot is considered as an unskilled worker who is strong and capable for precise manufacturing. It has a special kind of intelligence but it is handicapped in a sense, which requires it to be supervised. If people can learn how to communicate to this “new worker” they can get a new, capable “colleague”. The goal is that the boss is able to give the daily task to a robot in a similar way as he/she gives the jobs to the human workers, for example using CAD documentations, gestures and some verbal explanation. This paper presents an industrial robot supervision system inspired by research results of cognitive infocommunication. The operator can steer the remote manipulator by certain gestures using a motion capture suit as input device. Every gesture has its own meaning, which corresponds to a specific movement of the robot. The manipulator interprets and executes the instructions invoking its on-board artificial intelligence, while feedback through a 3D visualization unit closes the supervisory loop. The system was designed to be independent of the geographical distance between the user and the manipulated environment, allowing to establish control loops spanning through countries and continents. Successful results have been achieved between Norway, France and Hungary.

Opto-Mechanical Filtering Applied for Orientation and Length Selective Contour Detection

Cognitive informatics is a very hot topic today, trying to provide solutions to problems in computation that are easily solved by the brain but are hard for a computer. The information processing properties of the brain are taken as the basis for such artificial systems. The first step of visual information processing, the extraction of orientation selective contours, is done in the primary visual cortex. A similar, orientation based filtering can be the first step in many vision systems of cognitive informatics. This paper presents a new approach to orientation selective contour detection. Oriented motion blur is used to filter for the desired orientations. The motion blur operation is performed before digitalization, using a vibrating mirror. The proposed technique not only provides a very high speed, high resolution contour detector, but also solves the problem of contour discontinuities.

Artificial neural network based stereo matching in Intelligent Space

Intelligent Space is a space which has distributed sensory intelligence and is equipped with actuators. The various devices of sensory intelligence cooperate with each other autonomously, and the whole space has high intelligence, where we can easily interact with computers and robots, and get useful services from them. The most important sensors are video cameras achieving stereo vision, where stereo matching is necessary. This paper presents an artificial neural network for the adjustment of two cameras in order to picture the same spot for means of stereoscopic vision. The two cameras are in a master-slave relationship, where the optical center of the slave camera is adjusted to picture the same point as the optical center of the master camera. The task is performed by a biologically inspired artificial neural network, which can be embedded in an analog neural hardware to produce a real time solution for camera control in stereo vision that is faster than existing solutions.

Distributed Image Processing System using the RT-Middleware Framework

In conventional robot system development the different robot parts (sensors, processing elements, actuators) are combined together in a compact, self contained system. The need for faster development, system reconfigurability and flexibility required the introduction of middleware frameworks for robot systems. One promising middleware framework is the RT-middleware technology (and its implementation named OpenRTM-aist), which is getting more and more popular in robot system design. Vision sensors are one of the most important sensors in robot systems. When constructing a new robot system, it is desirable that vision and image processing components are just as easily integrated as any other robot components. Unfortunately, RT- Middleware is not yet equipped with such RT-Components. This need can be satisfied by introducing a new image processing toolbox, directly obtained from DIMAN, a existing software previously developed by the authors. DIMAN is a distributed framework for image processing, with a modular architecture very similar to RT-middleware. The modules of DIMAN satisfy the needs of robot systems with respect to vision related tasks. This paper will show how the modules of DIMAN can be deployed in RT-Components, and as such, directly integrated into RT-Middleware based robot systems.

Cognitive Informatics based DIND for Corner and Crossing Detection in Intelligent Space

Intelligent space is a space of distributed sensory intelligence and actuators. The basic component of intelligent space is the distributed intelligent network device (DIND), responsible for intelligent sensing and estimation. It is important for intelligent space to be aware of state of its internal environment, such as the class and position of objects it contains. The ability of object recognition is thus necessary in intelligent space. Existing cognitive computational systems like the mammalian cortex perform extremely well when it comes to visual object recognition. Psychological experiments have shown that image features such as edges and especially corners are of great importance in cognitive object recognition. Inspired by cognitive recognition systems, the proposed cognitive informatics model addresses the problem of vertex and corner detection. The model integrates the previously developed visual feature array (VFA), which is a cognitive model of oriented edge detection. The ultimate goal of the presented model is to provide suitable input for a cognitive object recognition system

Visual Feature Array based Cognitive polygon recognition using the UFEX text categorizer

This paper presents a cognitive vision based approach to recognize polygons on a natural image. The approach is based on the visual feature array (VFA), which is a cognitive computational model of the mammalian primary visual processing. VFA, as a multidimensional orthogonal data structure, contains data about the line segment and vertex features in the edge detected input image. Based on the features available in VFA, using the universal feature extractor classifier (UFEX), the problem of polygon categorization and recognition is addressed. The results are compared to solutions by conventional neural networks, such as the learning vector quantization network

Substitution of haptic gloves using audio interfaces in robot control

We refer to channels of communication which link the user to various electronic appliances and computers as cognitive communication channels. One especially interesting research topic related to cognitive communication channels deals with a special application called sensory substitution, when information is conveyed through a channel other than the one that is normally used. Besides offering a glimpse of hope to those living with injured sensory organs, sensory substitution can have benefits when designing user interfaces in terms of cost-effectiveness, the reduction of network delays, as well as the prevention of channel overloading. One such application in robot control could be the substitution of haptic feedback gloves (which are costly, bulky and have relatively few vibration states) using parameter-rich audio interfaces. In this paper, we propose the foundations of a framework for user-directed, interactive experimentation with haptic feedback using auditory cognitive communiation channels. Within the presented framework, the paper deals with an approach to describe the relationship between physical parameters and coding parameters that can be used to convey control information through auditory cognitive channels.

Sonification of surface parameters for remote tactile sensing

We refer to channels of communication which link the user to various electronic appliances and computers as cognitive communication channels. One especially interesting research topic related to cognitive communication channels deals with a special application called sensory substitution, when information is conveyed through a channel other than the one that is normally used. Besides offering a glimpse of hope to those living with injured sensory organs, sensory substitution can have benefits when designing user interfaces both in terms of cost-effectiveness, as well as the prevention of channel overloading. In this paper, we propose a unified approach to study the relationship between physical parameters and coding parameters that can be used to convey control information through auditory cognitive channels. The approach is demonstrated through an application in which the physical properties of a surface are conveyed to a remote teleoperator through sound. Plans for experiments to be conducted in future investigations are also layed out.