Yusuf Sinan Akgul

Extraction and tracking of the tongue surface from ultrasound image sequences

This paper presents a system for automatic extraction and tracking of 2D contours of the tongue surfaces from digital ultrasound image sequences. The input to the system is provided by a Head and Transducer Support System (HATS), which is developed for use in ultrasound imaging of the tongue movement. We developed a novel active contour (snakes) model that uses several temporally adjacent images during the extraction of the tongue surface contour for an image frame. The user supplies an initial contour model for a single image frame in the whole sequence. Using optical flow and multi-resolution methods, this initial contour is then used to find the candidate contour points in the temporally immediate adjacent images. Subsequently, the new snake mechanism is applied to estimate optimal contours for each image frame using these candidate points. In turn, the extracted contours are used as models for the extraction process of new adjacent frames. Finally, the system uses a novel postprocessing technique to refine the positions of the contours. We tested the system on 11 different speech sequences, each containing about 25 images. Visual inspection of the detected contours by the speech experts shows that the results are very promising and this system can be effectively employed in speech and swallowing research.

A coarse-to-fine deformable contour optimization framework

This paper introduces a novel coarse-to-fine deformable contour optimization framework, which is composed of two main components. The first component uses scale-space and information theories to produce a coarser representation of the input image to be used in a coarse-to-fine optimization scheme. The employment of information theory ensures that maximal image information is propagated to the coarse images and employment of scale spaces provides a mechanism to change the image coarseness locally based on the deformable contour model definition. The second component of this framework uses a novel combination of dynamic programming and gradient descent methods to optimize the contour energy on coarser representations and then use the obtained coarse contour positions in finer optimizations. The motivation in using a combination of dynamic programming and gradient descent method is to take advantage of each methods efficiency and avoid their drawbacks. In order to verify the performance of this framework, we constructed a deformable contour model for the spatiotemporal tracking of closed contours and optimized the model energy under this framework. Experiments on this system performed using synthetic images and real world echocardiographic sequences demonstrated the effectiveness and practicality of this framework.

Vision-based railroad track extraction using dynamic programming

Most of the common driver assistant systems for detection of obstacles work on unstructured environments. These environments generally include many non-planar surfaces which pose a big challenge for vision systems. Similar problems exist for railroad environments which often contain complex shapes and surfaces like hills and vegetation along railroad tracks. In railroad transportation, the main task of a train driver is to carefully focus on the track. Therefore the field of view of a train driver must contain the space between two rails in front of the train and the near lateral area (left and right side) of these rails. In this paper, we present an algorithm to extract the train course and railroad track space in front of the train using dynamic programming in railroad environments. We use dynamic programming to compute the optimal path which gives the minimum cost to extract the railroad track space. The proposed algorithm extracts the left and right rails using dynamic programming simultaneously. Our method does not need any static calibration process. For this purpose, a camera system was installed in front of a locomotive. Experimental results show the effectiveness of the algorithm.

Localization of the Lumbar discs using machine learning and exact probabilistic inference

We propose a novel fully automatic approach to localize the lumbar intervertebral discs in MR images with PHOG based SVM and a probabilistic graphical model. At the local level, our method assigns a score to each pixel in target image that indicates whether it is a disc center or not. At the global level, we define a chain-like graphical model that represents the lumbar intervertebral discs and we use an exact inference algorithm to localize the discs. Our main contributions are the employment of the SVM with the PHOG based descriptor which is robust against variations of the discs and a graphical model that reflects the linear nature of the vertebral column. Our inference algorithm runs in polynomial time and produces globally optimal results. The developed system is validated on a real spine MRI dataset and the final localization results are favorable compared to the results reported in the literature.

Simultaneous Localization of Lumbar Vertebrae and Intervertebral Discs With SVM-Based MRF

This paper presents a method for localizing and labeling the lumbar vertebrae and intervertebral discs in mid-sagittal MR image slices. The approach is based on a Markov-chain-like graphical model of the ordered discs and vertebrae in the lumbar spine. The graphical model is formulated by combining local image features and semiglobal geometrical information. The local image features are extracted from the image by employing pyramidal histogram of oriented gradients (PHOG) and a novel descriptor that we call image projection descriptor (IPD). These features are trained with support vector machines (SVM) and each pixel in the target image is locally assigned a score. These local scores are combined with the semiglobal geometrical information like the distance ratio and angle between the neighboring structures under the Markov random field (MRF) framework. An exact localization of discs and vertebrae is inferred from the MRF by finding a maximum a posteriori solution efficiently using dynamic programming. As a result of the novel features introduced, our system can scale-invariantly localize discs and vertebra at the same time even in the existence of missing structures. The proposed system is tested and validated on a clinical lumbar spine MR image dataset containing 80 subjects of which 64 have disc- and vertebra-related diseases and abnormalities. The experiments show that our system is successful even in abnormal cases and our results are comparable to the state of the art.

Adaptive regularization parameter for graph cut segmentation

Graph cut minimization formulates the segmentation problem as the liner combination of data and smoothness terms. The smoothness term is included in the energy formulation through a regularization parameter. We propose that the trade-off between the data and the smoothness terms should not be balanced by the same regularization parameter for the whole image. In order to validate the proposed idea, we build a system which adaptively changes the effect of the regularization parameter for the graph cut segmentation. The method calculates the probability of being part of the boundary for each pixel using the Canny edge detector at different hysteresis threshold levels. Then, it adjusts the regularization parameter of the pixel depending on the probability value. The experiments showed that adjusting the effect of the regularization parameter on different image regions produces better segmentation results than using a single best regularization parameter.

A New Adaptive Focus Measure for Shape From Focus.

This paper proposes a new focus measure operator for Shape From Focus to recover a dense depth map of a scene. The method can handle depth discontinuities effectively by using adaptively shaped and weighted support windows. The support window shapes and weights are determined from the image characteristics of the all-focused image of the scene. Similar and closer pixels in the support window get higher weights, which inhibits the problems caused by the depth discontinuities. The size of the support window can be increased conveniently for a more robust depth estimation without introducing any window size related Shape From Focus problems. The large support window sizes also addresses the edge bleeding problem. The experiments on the real and synthetically refocused images show that the introduced ideas work effectively and efficiently in real world applications.

Recovery and tracking of continuous 3D surfaces from stereo data using a deformable dual-mesh

We propose a novel method for continuous 3D depth recovery and tracking using calibrated stereo. The method integrates stereo correspondence, surface reconstruction and tracking by using a new single deformable dual mesh optimization, resulting in simplicity, robustness and efficiency. In order to combine stereo correspondence and structure recovery, the method introduces an external energy function defined for a 3D volume based on cross-correlation between the stereo pairs. The internal energy functional of the deformable dual mesh imposes smoothness on the surfaces and it serves as a communication tool between the two meshes. Under the forces produced by the energy terms, the dual mesh deforms to recover and track the 3D surface. The newly introduced dual-mesh model, which is one of the main contributions of this paper, makes the system robust against local minima and yet it is efficient. A coarse-to-fine minimization approach makes the system even more efficient. Tracking is achieved by using the recovered surface as an initial position for the next time frame. Although the system can effectively utilize initial surface positions and disparity data, they are not needed for a successful operation, which makes this system applicable to a wide range of areas. We present the results of a number of experiments on stereo human face and cloud images, which proves that our new method is very effective.

A Computerized Recognition System for the Home-Based Physiotherapy Exercises Using an RGBD Camera

Computerized recognition of the home based physiotherapy exercises has many benefits and it has attracted considerable interest among the computer vision community. However, most methods in the literature view this task as a special case of motion recognition. In contrast, we propose to employ the three main components of a physiotherapy exercise (the motion patterns, the stance knowledge, and the exercise object) as different recognition tasks and embed them separately into the recognition system. The low level information about each component is gathered using machine learning methods. Then, we use a generative Bayesian network to recognize the exercise types by combining the information from these sources at an abstract level, which takes the advantage of domain knowledge for a more robust system. Finally, a novel postprocessing step is employed to estimate the exercise repetitions counts. The performance evaluation of the system is conducted with a new dataset which contains RGB (red, green, and blue) and depth videos of home-based exercise sessions for commonly applied shoulder and knee exercises. The proposed system works without any body-part segmentation, bodypart tracking, joint detection, and temporal segmentation methods. In the end, favorable exercise recognition rates and encouraging results on the estimation of repetition counts are obtained.

An occlusion insensitive adaptive focus measurement method

This paper proposes a new focus measurement method for Depth From Focus to recover depth of scenes. The method employs an all-focused image of the scene to address the focus measure ambiguity problem of the existing focus measures in the presence of occlusions. Depth discontinuities are handled effectively by using adaptively shaped and weighted support windows. The size of the support window can be increased conveniently for more robust depth estimation without introducing any window size related Depth From Focus problems. The experiments on the real and synthetically refocused images show that the introduced focus measurement method works effectively and efficiently in real world applications.