Itsuo Kumazawa

A neural network for recovering 3D shape from erroneous and few depth maps of shaded images

In this paper, we present a new neural network (NN) for three-dimensional (3D) shape reconstruction. This NN provides an analytic mapping of an initial 3D polyhedral model into its projection depth images. Through this analytic mapping, the NN can analytically refine vertices position of the model using error back-propagation learning. This learning is based on shape-from-shading (SFS) depth maps taken from multiple views. The depth maps are obtained by Tsai-Shah SFS algorithm. They are considered as partial 3D shapes of the object to be reconstructed. The task is to reconstruct an accurate and complete representation of a given object relying only on a limited number of views and erroneous SFS depth maps. Through hierarchical reconstruction and annealing reinforcement strategies, our reconstruction system gives more exact and stable results. In addition, it corrects and smoothly fuses the erroneous SFS depth maps. The implementation of this neural network algorithm used in this paper is available at http://kumazawa-www.cs.titech.ac.jp/~fanany/MV-SPRNN/mv-sprnn.html.

OPTICAL FUZZY IMAGE-PROCESSING BASED ON SHADOW-CASTING

Abstract An optical system based on an area-coded scheme and the shadow-casting technique is proposed for fuzzy image processing. All of the fuzzy logic functions of two images can be implemented in parallel. The access of the fuzzy logic functions can be easily achieved by programming an LED source array in the system. Above all, no thresholding device is required. The experimental results are also given.

Compact and parametric shape representation by a tree of sigmoid functions for automatic shape modeling

Abstract A modeling method to represent an object shape by a tree of sigmoid functions, and a gradient-descent-based searching procedure to estimate shape parameters are described. The models unique feature is that it can produce a shape parameterized with a gain parameter determining the resolution of the represented shape. With a large gain value, the model represents a detailed shape with a sharp edge, which is described by a piecewise combination of lines and curves. With a small gain value, it generates a blurred shape, which has explicit (analytic) forms of partial derivaties with respect to parameters describing the shape. Using the models differentiable property, the gradient-descent-based searching method updates the shape parameters and the gain parameter simultaneously so that an optimal set of shape parameters which makes the model fit to a target shape is found through a coarse-to-fine search. Previous shape description techniques were either contour-based (spline, active contour, wireframe, polygon and so on) or expansion-based (wavelet, radial basis function, eigenfunction and so on). The method has a dual modal nature with its capability of dealing with sharp edges and its differentiable and resolution adjustable nature. The models performance was evaluated by some coarse-to-fine shape modeling simulations and its efficiency and robustness were verified.

A scheme for reconstructing face from shading using smooth projected polygon representation NN

In this paper, we present a neural-network learning scheme for face reconstruction. This scheme, which we called the smooth projected polygon representation neural network (SPPRNN), is able to successively refine the polygons vertices parameter of an initial 3D shape based on depth-maps of several calibrated images taken from multiple views. The depth-maps, which are obtained by deploying the Tsai-Shah shape-from-shading (SFS) algorithm, can be considered as partial 3D shapes of the face to be reconstructed. The reconstruction is finalized by mapping the texture of face images to the initial 3D shape. There are three interesting issues investigated in this paper concerning the effectiveness of this scheme. First, how effective the SFS provides partial 3D shapes compared to if we simply used 2D images. Secondly, how essential a smooth projected polygonal model is in order to approximate the face structure and enhance the convergence rate of this scheme. Thirdly, how an appropriate initial 3D shape should be selected and used in order to improve model resolution and learning stability. By carefully addressing these three issues, it was shown from our experiment that a compact and realistic 3D model of a human (mannequin) face could be obtained.

Background images generation based on the nelder-mead simplex algorithm using the eigenbackground model

The Eigenbackground model is often stated to perform better than pixel-based methods when illumination variations occur. However, it has originally one demerit, that foreground objects must be small. This paper presents an original improvement of the Eigenbackground model, dealing with large and fast moving foreground objects. The method generates background images using the Nelder-Mead Simplex algorithm and a dynamic masking procedure. Experiments show that the proposed method performs as well as the state-of-the-art Eigenbackground improvements in the case of slowly moving objects, and achieves better results for quickly moving objects.

Multiple-view shape extraction from shading as local regression by analytic NN scheme

We introduce a multiple-view 3D-shape-reconstruction system. This system is able to fuse few-view and erroneous depth maps into a more complete and more accurate shape representation using a unique neural network (NN). The NN provides analytic mapping and learning of a polyhedron model to approximate the true shape of an object based on multiple-view depth maps. The depth maps are obtained by a widely used Tsai-Shah shape-from-shading (SFS) algorithm. They are considered as partial 3D shapes of the object to be reconstructed. The main insight of this work is that the NN minimizes the depth map error in one view using depth maps information from other views observed under nonfixed light source positions relative to the object. Theoretically, we formulate our problem as nonparametric (local) regression in depth space formed by multiple view observations. Experimentally, we obtain exact and stable results through hierarchical reconstruction and annealing reinforcement. We provide the implementation of the NN used in this paper at .

Image restoration using a stochastic variant of the alternating direction method of multipliers

We propose an efficient image restoration framework based on stochastic optimization. Image restoration usually requires some iterative methods for solving optimization problems that characterize restored images, where the multiplication of the observation matrix Φ ϵ Rm × n and variables has to be computed at each iteration. If an efficient implementation of the multiplication (e.g., using FFT) is unavailable, its computational cost becomes O(MN), which is quite expensive since both N and M are usually large in image restoration. Our method needs to load and apply only a part of the observation matrix of size M/b × N (b: the number of parts), so that the computational cost is only O(MN/b). Moreover, the proposed method accepts various nonsmooth objectives effective for image restoration. Experiments on compressed sensing reconstruction and non-uniform deblurring show the advantage of the proposed method over state-of-the-art proximal optimization methods.

Online Boosting Algorithm Based on Two-Phase SVM Training

We describe and analyze a simple and effective two-step online boosting algorithm that allows us to utilize highly effective gradient descent-based methods developed for online SVM training without the need to fine-tune the kernel parameters, and we show its efficiency by several experiments. Our method is similar to AdaBoost in that it trains additional classifiers according to the weights provided by previously trained classifiers, but unlike AdaBoost, we utilize hinge-loss rather than exponential loss and modify algorithm for the online setting, allowing for varying number of classifiers. We show that our theoretical convergence bounds are similar to those of earlier algorithms, while allowing for greater flexibility. Our approach may also easily incorporate additional nonlinearity in form of Mercer kernels, although our experiments show that this is not necessary for most situations. The pre-training of the additional classifiers in our algorithms allows for greater accuracy while reducing the times associated with usual kernel-based approaches. We compare our algorithm to other online training algorithms, and we show, that for most cases with unknown kernel parameters, our algorithm outperforms other algorithms both in runtime and convergence speed.

Haptic mouse with quick and flexible tactile feedback generated by double control loop

This paper introduces a mouse with a tactile display that generates tactile feedback for multi-modal user interface. The control system to generate tactile feedback imitates the biological nervous system that has multiple feedback loops for quick and flexible reaction against sensory inputs. The control system, inspired by the biological system, has double feedback loops, one for the local feedback performed within the mouse for quick and fixed reaction and the other for the global feedback performed through the communication with an external computer for slow and flexible reaction. The conflicting demands of quickness and flexibility can be satisfied by combining these two feedback loops. The control system is applied to the multi-modal user interface that uses tactile information in addition to the auditory information to assist visually handicapped people to operate a web browser that uses the tactile information to guide cursor operation.

Object tracking with shape representation network using color information

In this article, we propose an object tracking method using a neural network which represents the shape of an object based on the objects color information. We previously proposed a specific form of multiple-layered neural network which has a suitable structure to represent an objects shape. This network (shape representation network, SRN) originally was developed to deal with black and white images but it is extended for color images in this article. SRN is capable of representing objects of various kinds of shape and color with an arbitrary degree of blurring. Its learning capability enables automatic model construction for various shapes including their color information. To perform object tracking with color information, we introduce Mahalanobis distance in color space and improve the tracking performance. Some experiments are performed to evaluate the performance of the proposed method using real image sequences.