Danijela Ristic-Durrant

A robust markerless vision-based human gait analysis system

In this paper, a novel robust markerless image processing system capable of extracting gait features which can be used for gait analysis is presented. The presented system can deal with images of persons captured in natural indoor scenes. The systems robustness against external influences and different person appearance is achieved by employing the idea of improving the image processing robustness by including feedback control at the image segmentation level. The effectiveness of the proposed system is demonstrated by the comparison of gait features, namely knee angles, extracted automatically with the features directly measured using a goniometer. Also a small database is created to extract the gait pattern of healthy subjects. The obtained data is compared to data from medical literature and is also compared to data obtained from persons having a pathological gait.

VF-SIFT: very fast SIFT feature matching

Feature-based image matching is one of the most fundamental issues in computer vision tasks. As the number of features increases, the matching process rapidly becomes a bottleneck. This paper presents a novel method to speed up SIFT feature matching. The main idea is to extend SIFT feature by a few pairwise independent angles, which are invariant to rotation, scale and illumination changes. During feature extraction, SIFT features are classified based on their introduced angles into different clusters and stored in multidimensional table. Thus, in feature matching, only SIFT features that belong to clusters, where correct matches may be expected are compared. The performance of the proposed methods was tested on two groups of images, realworld stereo images and standard dataset images, through comparison with the performances of two state of the arte algorithms for ANN searching, hierarchical k-means and randomized kd-trees. The presented experimental results show that the performance of the proposed method extremely outperforms the two other considered algorithms. The experimental results show that the feature matching can be accelerated about 1250 times with respect to exhaustive search without losing a noticeable amount of correct matches.

Stereo Vision-Based Human Tracking for Robotic Follower

Abstract This paper addresses the problem of real-time vision-based human tracking to enable mobile robots to follow a human co-worker. A novel approach to combine stereo vision-based human detection with human tracking using a modified Kalman filter is presented. Stereo vision-based detection combines features extracted from 2D stereo images with reconstructed 3D object features to detect humans in a robots environment. For human tracking a modified Kalman filter recursively predicts and updates estimates of the 3D coordinates of a human in the robots camera coordinate system. This prediction enables human detection to be performed on the image region of interest contributing to cost effective human tracking. The performance of the presented method was tested within a working scenario of a mobile robot intended to follow a human co-worker in indoor applications as well as in outdoor applications.

A Supportive FRIEND at Work: Robotic Workplace Assistance for the Disabled

This article presents the evolution of an assistive robotic system, the Functional Robot with Dexterous Arm and User-Friendly Interface for Disabled People (FRIEND), from a robot supporting disabled people in their activities of daily living (ADL) into a robot supporting people with disabilities in real workplaces. In its fourth generation, FRIEND supports the end user, a quadriplegic individual, to work as a librarian with the task of retrospectively cataloging collections of old books. All of the book manipulation tasks, such as grasping the book from the book cart and placing it on the specially designed book holder for reading by the end user, are carried out autonomously by the FRIEND system. The retrospective cataloging itself is done by the end user. This article discusses all of the technical adjustments and improvements to the FRIEND system that are necessary to meet the challenges of a robot supporting a disabled person working on a regular basis. These challenges concern the shared autonomy between system and user, system effectiveness, safety in interaction with the user, and user acceptability. The focus is on both the vision-based control of book manipulation as a key factor for autonomous robot functioning and on an advanced human-machine interface (HMI), which enables the end user to intervene if the autonomous book manipulation fails. The experimental results of an in-depth evaluation of the system performance in supporting the end user to perform the librarian task are presented. It has been shown that working together, the FRIEND system and the end user had an overall success rate of 95%. These results may help to raise interest in the research field of workplace assistive robotics, establish new projects, and, eventually, supply such systems to the people whose working lives they could greatly improve.

ROVIS: Robust machine vision for service robotic system FRIEND

In this paper the vision architecture, named ROVIS, of the robotic system FRIEND is presented. The main concept of the ROVIS is the inclusion of feedback structures between different components of the vision system as well as between the vision and other modules of the robotic system to achieve high robustness against external influences of the individual system units as well as of the system as whole. The novelty of this work lies in the inclusion of feedback control at different levels of the 2D object recognition system to provide reliable inputs to the 3D object reconstruction and object manipulation modules of the robotic system FRIEND. The idea behind this approach is to change the processing parameters in a closed-loop manner so that the current image processing result at a particular processing level is driven to a desired result. The effectiveness of the ROVIS system is demonstrated through the presentation of experimental results on 3D reconstruction of different objects from FRIEND environment.

Mobile robotic gait rehabilitation system CORBYS - overview and first results on orthosis actuation

In this paper the novel mobile robotic gait rehabilitation system CORBYS is presented. The system consists of a mobile platform and a powered orthosis attached to the platform. Beside the mobility, i.e. overground walking, due to introduced degrees of freedom (DOFs) CORBYS gait rehabilitation system, in contrast to existing gait rehabilitation robotic systems will enable more physiological movements including turning. The focus in the paper is on novel push-pull control (PPC) cables based actuation system of the orthosis. The first results on orthosis actuation obtained in the control experiments with the powered orthosis test-stand are shown. The results prove the advantages of using the push-pull cables for the dislocation of the actuators from the powered orthosis and for providing a flexible power actuation with a bi-directional (pushing and pulling) force transfer.

Closed-Loop Control in Image Processing for Improvement of Object Recognition

This paper presents the employment of a novel idea of inclusion of feedback control at different image processing levels to improve the robustness of object recognition. The proposed approach is intended to cope with object image color uncertainty that arises from changes in the illumination conditions during image acquisition. The focus is on the control of quality of binary segmented image, which represents the input to higher processing levels, feature extraction and recognition, of the image processing chain. The main idea behind this is that introduced closed-loops drive the current segmented image to the image of the desired, reference, quality so that higher processing levels are provided with reliable input image data. In this way, the reliability and robustness against external influences of the overall object recognition system is improved. The specifics and benefit of closed-loop control in image processing are considered throughout the presentation of the closed-loop object recognition in the robotic system FRIEND II. The benefit of the closed-loop image segmentation in service robotics is considered through the demonstration of results achieved for recognition of the object of interest from a working scenario of the system FRIEND II.

CORBYS cognitive control architecture for robotic follower

In this paper the novel generic cognitive robot control architecture CORBYS is presented. The objective of the CORBYS architecture is the integration of high-level cognitive modules to support robot functioning in dynamic environments including interacting with humans. This paper presents the preliminary integration of the CORBYS architecture to support a robotic follower. Experimental results on high-level empowerment-based trajectory planning have demonstrated the effectiveness of ROS-based communication between distributed modules developed in a multi-site research environment as typical for distributed collaborative projects such as CORBYS.

Robust stereo-vision based 3D modelling of real-world objects for assistive robotic applications

This paper addresses the problem of recognizing and reconstructing real-world objects in cluttered environments to enable service robot to grasp the objects and manipulate them. A novel approach to combine disparity segmentation method with the closed-loop color region based segmentation is presented. Disparity map segmentation leads to definition of object region of interest (ROI) enabling autonomous functioning of robot system in cluttered environments. Closed-loop object region segmentation is robust against variable illumination providing reliable operation of robot system in different lighting conditions. Starting from the segmented object in both stereo images, the 3D contour of the object is generated and the object geometry is recovered from it. The proposed method needs no a-priori knowledge about the object color, its appearance or geometry. The performance of the presented method has been tested within the working scenario of the assistive robotic system FRIEND.

Learning gait by therapist demonstration for natural-like walking with the CORBYS powered orthosis

The number of mechanical degrees of freedom (DoFs) within rehabilitation robots directly influences the scope of the movements that a subject can perform when training walking. Currently, gait rehabilitation robots have a limited number of mechanical DoFs, as a consequence this limits the movements these robots can make possible. In this paper, the novel gait rehabilitation system CORBYS is presented which consists of the mobile platform and a powered orthosis which is attached to the platform. The CORBYS powered orthosis has 16 DoFs enabling more physiological movements, making it a state-of-the-art gait rehabilitation robotic system. With the sufficient number of DoFs to enable natural-like walking, the CORBYS robotic system enables the integration of the “learning gait by therapist demonstration” paradigm. This paper presents the fully integrated functional CORBYS gait rehabilitation system, with the focus on the implementation aspects which enable generation of the reference gait trajectory through learning by therapist demonstration, and the use of the generated trajectory in the robotic therapy session. The results of the initial evaluation of the robotic system obtained in tests with a selected patient are given in the paper.