Kimiaki Shirahama

Classification of environmental microorganisms in microscopic images using shape features and support vector machines

Environmental Microorganisms (EMs) are currently recognised using molecular biology (DNA, RNA) or morphological methods. The first ones are very time-consuming and expensive. The second ones require a very experienced laboratory operator. To overcome these problems, we introduce an automatic classification method for EMs in the framework of content-based image analysis in this paper. To describe the shapes of EMs observed in microscopic images, we use Edge Histograms, Fourier Descriptors, extended Geometrical Features, as well as introduce Internal Structure Histograms. For classification, multi-class Support Vector Machine is applied to EMs represented by the above features. In order to quantitatively evaluate discriminative properties of the feature spaces we have introduced, we perform comprehensive experiments with a ground truth of manually segmented microscopic EM images. The best classification result of 89.7% proves a high robustness of our method in this application domain.

Event retrieval in video archives using rough set theory and partially supervised learning

This paper develops a query-by-example method for retrieving shots of an event (event shots) using example shots provided by a user. The following three problems are mainly addressed. Firstly, event shots cannot be retrieved using a single model as they contain significantly different features due to varied camera techniques, settings and so forth. This is overcome by using rough set theory to extract multiple classification rules with each rule specialized to retrieve a portion of event shots. Secondly, since a user can only provide a small number of example shots, the amount of event shots retrieved by extracted rules is inevitably limited. We thus incorporate bagging and the random subspace method. Classifiers characterize significantly different event shots depending on example shots and feature dimensions. However, this can result in the potential retrieval of many unnecessary shots. Rough set theory is used to combine classifiers into rules which provide greater retrieval accuracy. Lastly, counter example shots, which are a necessity for rough set theory, are not provided by the user. Hence, a partially supervised learning method is used to collect these from shots other than example shots. Counter example shots, which are as similar to example shots as possible, are collected because they are useful for characterizing the boundary between event shots and the remaining shots. The proposed method is tested on TRECVID 2009 video data.

Object matching with hierarchical skeletons

The skeleton of an object provides an intuitive and effective abstraction which facilitates object matching and recognition. However, without any human interaction, traditional skeleton-based descriptors and matching algorithms are not stable for deformable objects. Specifically, some fine-grained topological and geometrical features would be discarded if the skeleton was incomplete or only represented significant visual parts of an object. Moreover, the performance of skeleton-based matching highly depends on the quality and completeness of skeletons. In this paper, we propose a novel object representation and matching algorithm based on hierarchical skeletons which capture the shape topology and geometry through multiple levels of skeletons. For object representation, we reuse the pruned skeleton branches to represent the coarse- and fine-grained shape topological and geometrical features. Moreover, this can improve the stability of skeleton pruning without human interaction. We also propose an object matching method which considers both global shape properties and fine-grained deformations by defining singleton and pairwise potentials for similarity computation between hierarchical skeletons. Our experiments attest our hierarchical skeleton-based method a significantly better performance than most existing shape-based object matching methods on six datasets, achieving a 99.21% bulls-eye score on the MPEG7 shape dataset. HighlightsIt represents the coarse- and fine-grained shape topological and geometrical features.It improves the stability of skeleton pruning without human interaction.It considers both global and fine shape properties by different potential functions.It achieves a better performance than most existing methods on six datasets.Experiments attest our method a better performance than most related approaches.We achieve a 99.21% bulls-eye score on the MPEG7 shape dataset.

Shape-Based Classification of Environmental Microorganisms

Occurrence of certain environmental microorganisms and their species is a very informative indicator to evaluate environmental quality. Unfortunately, their manual recognition in microbiological laboratories is very time-consuming and expensive. Therefore, we work on an automatic method for shape-based classification of EMs in microscopic images. First, we segment the microorganisms from the background. Second, we describe their shapes by discriminative feature vectors. Third, we perform the EM classification using Support Vector Machines. The most important scientific contribution of this paper, in comparison to the state-of-the-art and to our previous publications in this field, is the introduction of a completely new and very robust 2D feature descriptor for EM shapes. Experimental results certify the effectiveness and practicability of our automatic EM classification system emphasising the benefits achieved with the new shape descriptor proposed in this work.

Multiple human detection in depth images

Most human detection algorithms in depth images perform well in detecting and tracking the movements of a single human object. However, their performance is rather poor when the person is occluded by other objects or when there are multiple humans present in the scene. In this paper, we propose a novel human detection technique which analyzes the edges in depth image to detect multiple people. The proposed technique detects a human head through a fast template matching algorithm and verifies it through a 3D model fitting technique. The entire human body is extracted from the image by using a simple segmentation scheme comprising a few morphological operators. Our experimental results on three large human detection datasets and the comparison with the state-of-the-art method showed an excellent performance achieving a detection rate of 94.53% with a small false alarm of 0.82%.

Environmental microbiology aided by content-based image analysis

Environmental microorganisms (EMs) such as bacteria and protozoa are found in every imaginable environments. To explore functions of EMs is an important research field for environmental assessment and treatment. However, EMs are traditionally investigated through morphological analysis using microscopes or DNA analysis, which is time and money consuming. To overcome this, we introduce an innovative method which applies content-based image analysis (CBIA) to environmental microbiology. Our method classifies EMs into different categories based on features extracted from microscopic images. Specifically, it consists of three steps: The first is image segmentation which accurately extracts the region of an EM in a microscopic image with a small amount of user interaction. The second step is feature extraction where multiple features are extracted to describe different characteristics of the EM. In particular, we develop an internal structure histogram descriptor which captures the structure of the EM using angles defined on its contour. The last step is fusion which combines classification results by different features to improve the performance. Experimental results validate the effectiveness and practicability of our environmental microbiology method aided by CBIA.

Video data mining: rhythms in a movie

The task to discover useful editing patterns from a professional video, such as a movie, is one of the main purposes of video data mining. These patterns successfully convey an editors intentions to the viewer. However, data mining on multimedia data like a movie is a challenging task, due to the complicated contents of multimedia data. Particularly, the discovered patterns need to be supported by their semantic features, because these features tell amateur editors how to use the corresponding patterns during the process of editing a new video. We focus on the rhythm in a movie, consisting of the durations of a target characters appearance and disappearance. Based on this rhythm, we divide the movie into topics. Each topic corresponds to one meaningful episode of the character. By investigating such topics, we can discover useful editing patterns of a characters rhythm, supported by their semantic features. Also, these rhythms can be used to annotate certain types of topics

Shape-based Object Matching Using Point Context

This paper proposes a novel object matching algorithm based on shape contours. In order to ensure low computational complexity in shape representation, our descriptor is composed by a small number of interest points which are generated by considering both curvatures and the overall shape trend. To effectively describe each point of interest, we introduce a simple and highly discriminative point descriptor, namely Point Context, which represents its geometrical and topological location. For shape matching, we observed that the correspondences are not only dependent on the similarities between these single points in different objects, but they are also related to the geometric relations between multiple points of interest in the same object. Therefore, a high-order graph matching formulation is introduced to merge the single point similarities and the similarities between point triangles. The main contributions of this paper include (i) the introduction of a novel shape descriptor with robust shape points and their descriptors and (ii) the implementation of a high-order graph matching algorithm that solves the shape matching problem. Our method is validated through a series of object retrieval experiments on four datasets demonstrating its robustness and accuracy.

Environmental Microorganism Classification Using Sparse Coding and Weakly Supervised Learning

Environmental Microorganisms (EMs), such as Epistylis and Rotifera, are very tiny living beings in human environments and decompose pollutants as their nutrition. The classification of EMs plays a fundamental role for establishing sustainable ecosystem. However, this is traditionally done by molecular methods using DNA or RNA analysis, or morphological methods using microscopes. These methods require huge monetary costs or manual efforts. In this paper, we propose an EM classification method by directly processing microscopic images. We especially address two problems: The first is a small training dataset problem. Because EM samples are collected from unstable real-world environments, it is difficult to stably obtain a large number of EM images. To effectively represent scarce training images, we use Sparse Coding (SC) which extracts sufficient local features from an image and reconstructs it by a sparse linear combination of bases. The second is a noisy image problem. Because EM samples are often found in dirty regions, their images often show a noisy background. To overcome this, we use Weakly Supervised Learning (WSL) which jointly performs the localisation and classification of EMs by examining the local information in training images. Experimental results on real-world images show the effectiveness and potential of our system.

Environmental microorganism classification using conditional random fields and deep convolutional neural networks

Abstract The labeling of Environmental Microorganisms (EM) which help decomposing pollutants, plays a fundamental role for establishing sustainable ecosystem. We propose an environmental microorganism classification engine that can automatically analyze microscopic images using Conditional Random Fields (CRF) and Deep Convolutional Neural Networks (DCNN). First, to effectively represent scarce training images, a DCNN pre-trained for image classification using a large amount of data is re-purposed to our feature extractor that distils pixel-level features in microscopic images. In addition, pixel-level classification results by such features can be refined using global features that describe the whole image in toto. Finally, our CRF model localizes and classifies EMs by considering the spatial relations among DCNN-based features, and their relations to global features. The experimental results have shown 94.2% of overall segmentation accuracy and up to 91.4% mean average precision of the results.