Hsin Chen

Model-based measurement of food portion size for image-based dietary assessment using 3D/2D registration.

Dietary assessment is important in health maintenance and intervention in many chronic conditions, such as obesity, diabetes, and cardiovascular disease. However, there is currently a lack of convenient methods for measuring the volume of food (portion size) in real-life settings. We present a computational method to estimate food volume from a single photographical image of food contained in a typical dining plate. First, we calculate the food location with respect to a 3D camera coordinate system using the plate as a scale reference. Then, the food is segmented automatically from the background in the image. Adaptive thresholding and snake modeling are implemented based on several image features, such as color contrast, regional color homogeneity and curve bending degree. Next, a 3D model representing the general shape of the food (e.g., a cylinder, a sphere, etc.) is selected from a pre-constructed shape model library. The position, orientation and scale of the selected shape model are determined by registering the projected 3D model and the food contour in the image, where the properties of the reference are used as constraints. Experimental results using various realistically shaped foods with known volumes demonstrated satisfactory performance of our image based food volume measurement method even if the 3D geometric surface of the food is not completely represented in the input image.

3D/2D model-to-image registration for quantitative dietary assessment

Image-based dietary assessment is important for health monitoring and management because it can provide quantitative and objective information, such as food volume, nutrition type, and calorie intake. In this paper, a new framework, 3D/2D model-to-image registration, is presented for estimating food volume from a single-view 2D image containing a reference object (i.e., a circular dining plate). First, the food is segmented from the background image based on Otsus thresholding and morphological operations. Next, the food volume is obtained from a user-selected, 3D shape model. The position, orientation and scale of the model are optimized by a model-to-image registration process. Then, the circular plate in the image is fitted and its spatial information is used as constraints for solving the registration problem. Our method takes the global contour information of the shape model into account to obtain a reliable food volume estimate. Experimental results using regularly shaped test objects and realistically shaped food models with known volumes both demonstrate the effectiveness of our method.

Automatic and quantitative measurement of collagen gel contraction using model-guided segmentation

Quantitative measurement of collagen gel contraction plays a critical role in the field of tissue engineering because it provides spatial-temporal assessment (e.g., changes of gel area and diameter during the contraction process) reflecting the cell behaviors and tissue material properties. So far the assessment of collagen gels relies on manual segmentation, which is time-consuming and suffers from serious intra- and inter-observer variability. In this study, we propose an automatic method combining various image processing techniques to resolve these problems. The proposed method first detects the maximal feasible contraction range of circular references (e.g., culture dish) and avoids the interference of irrelevant objects in the given image. Then, a three-step color conversion strategy is applied to normalize and enhance the contrast between the gel and background. We subsequently introduce a deformable circular model (DCM) which utilizes regional intensity contrast and circular shape constraint to locate the gel boundary. An adaptive weighting scheme was employed to coordinate the model behavior, so that the proposed system can overcome variations of gel boundary appearances at different contraction stages. Two measurements of collagen gels (i.e., area and diameter) can readily be obtained based on the segmentation results. Experimental results, including 120 gel images for accuracy validation, showed high agreement between the proposed method and manual segmentation with an average dice similarity coefficient larger than 0.95. The results also demonstrated obvious improvement in gel contours obtained by the proposed method over two popular, generic segmentation methods.

Automatic Insall–Salvati ratio measurement on lateral knee x-ray images using model-guided landmark localization

The Insall-Salvati ratio (ISR) is important for detecting two common clinical signs of knee disease: patella alta and patella baja. Furthermore, large inter-operator differences in ISR measurement make an objective measurement system necessary for better clinical evaluation. In this paper, we define three specific bony landmarks for determining the ISR and then propose an x-ray image analysis system to localize these landmarks and measure the ISR. Due to inherent artifacts in x-ray images, such as unevenly distributed intensities, which make landmark localization difficult, we hence propose a registration-assisted active-shape model (RAASM) to localize these landmarks. We first construct a statistical model from a set of training images based on x-ray image intensity and patella shape. Since a knee x-ray image contains specific anatomical structures, we then design an algorithm, based on edge tracing, for patella feature extraction in order to automatically align the model to the patella image. We can estimate the landmark locations as well as the ISR after registration-assisted model fitting. Our proposed method successfully overcomes drawbacks caused by x-ray image artifacts. Experimental results show great agreement between the ISRs measured by the proposed method and by orthopedic clinicians.

A Joint-Constraint Model-Based System for Reconstructing Total Knee Motion

Comprehending knee motion is an essential requirement for studying the causes of knee disorders. In this paper, we propose a new 2-D-3-D registration system based on joint-constraint model for reconstructing total knee motion. The proposed model that contains bone geometries and an articulated joint mechanism is first constructed from multipostural magnetic resonance volumetric images. Then, the bone segments of the model are hierarchically registered to each frame of the given single-plane fluoroscopic video that records the knee activity. The bone posture is iteratively optimized using a modified chamfer matching algorithm to yield the simulated radiograph which is the best fit to the underlying fluoroscopic image. Unlike conventional registration methods computing posture parameters for each bone independently, the proposed femorotibial and patellofemoral joint models properly maintain the articulations between femur, tibia, and patella during the registration processes. As a result, we can obtain a sequence of registered knee postures showing smooth and reasonable physiologic patterns of motion. The proposed system also provides joint-space interpolation to densely generate intermediate postures for motion animation. The effectiveness of the proposed method was validated by computer simulation, animal cadaver, and in vivo knee testing. The mean target registration errors for femur, tibia, and patella were less than 1.5 mm. In particular, small out-of-plane registration errors [less than 1 mm (translation) and 2° (rotation)] were achieved in animal cadaver assessments.Comprehending knee motion is an essential requirement for studying the causes of knee disorders. In this paper, we propose a new 2-D-3-D registration system based on joint-constraint model for reconstructing total knee motion. The proposed model that contains bone geometries and an articulated joint mechanism is first constructed from multipostural magnetic resonance volumetric images. Then, the bone segments of the model are hierarchically registered to each frame of the given single-plane fluoroscopic video that records the knee activity. The bone posture is iteratively optimized using a modified chamfer matching algorithm to yield the simulated radiograph which is the best fit to the underlying fluoroscopic image. Unlike conventional registration methods computing posture parameters for each bone independently, the proposed femorotibial and patellofemoral joint models properly maintain the articulations between femur, tibia, and patella during the registration processes. As a result, we can obtain a sequence of registered knee postures showing smooth and reasonable physiologic patterns of motion. The proposed system also provides joint-space interpolation to densely generate intermediate postures for motion animation. The effectiveness of the proposed method was validated by computer simulation, animal cadaver, and in vivo knee testing. The mean target registration errors for femur, tibia, and patella were less than 1.5 mm. In particular, small out-of-plane registration errors [less than 1 mm (translation) and 2° (rotation)] were achieved in animal cadaver assessments.

Computer Aided Quantification of Pathological Features for Flexor Tendon Pulleys on Microscopic Images

Quantifying the pathological features of flexor tendon pulleys is essential for grading the trigger finger since it provides clinicians with objective evidence derived from microscopic images. Although manual grading is time consuming and dependent on the observer experience, there is a lack of image processing methods for automatically extracting pulley pathological features. In this paper, we design and develop a color-based image segmentation system to extract the color and shape features from pulley microscopic images. Two parameters which are the size ratio of abnormal tissue regions and the number ratio of abnormal nuclei are estimated as the pathological progression indices. The automatic quantification results show clear discrimination among different levels of diseased pulley specimens which are prone to misjudgments for human visual inspection. The proposed system provides a reliable and automatic way to obtain pathological parameters instead of manual evaluation which is with intra- and interoperator variability. Experiments with 290 microscopic images from 29 pulley specimens show good correspondence with pathologist expectations. Hence, the proposed system has great potential for assisting clinical experts in routine histopathological examinations.

A coarse-to-fine auto-focusing algorithm for microscopic image

Auto-focusing is an important step in development a computer-aided evaluation system for diagnostic microscope. An auto-focusing algorithm can identify the best focus reliably and objectively, and prevents the tedious and time consuming steps of manual focusing. In this paper, a coarse-to-fine auto-focusing method is proposed based on a multi-resolution scheme which locates the focusing range by quadratic polynomial fitting in low resolution and achieves accurate focal point by using binary search in fine resolution. An algorithm was developed to identify the focal point under multiple image resolution with the Sum of Modified Laplacian (SML) as the focusing value. The main contribution of this system is to reduce the auto-focusing time. Comparing with the conventional binary search, the coarse-to-fine method only needs half of the time to accomplish auto-focusing. Using the proposed technique, the efficiency of auto-focusing system has been significantly improved.

Multiresolution Image Analysis for Automatic Quantification of Collagen Gel Contraction

Quantifying collagen gel contraction is important in tissue engineering and biological research because it provides spatial-temporal assessments of cell behaviors and tissue material properties. However, these assessments currently rely on manual processing, which is time-consuming and subjective to personal opinions. We present a multiresolution image analysis system for automatic quantification of gel contraction. This system includes a color conversion process to normalize and enhance the contrast between gel and background. Then, a deformable circular model is activated automatically to capture details of gel boundaries. These steps are coordinated by a multiresolution strategy. The target measurements are obtained after gel segmentation. Our experiments using 30 images demonstrated a high consistency between the proposed and manual segmentation methods. The system can process large-size images (4000×3000) at a rate of approximately one second per image. It thus serves as a useful tool for analyzing large biological and biomaterial imaging datasets efficiently and objectively.

Muscular motion estimation from 4-D ultrasound image using Kalman filter and rotation-invariant feature descriptor

Muscular motion estimation in ultrasound images is of great importance for investigating causes of musculoskeletal conditions in pathological examinations. However, the quality of ultrasound images is usually depressed due to speckle noises and temporal decorrelation of speckle patterns, making certain difficulties in motion estimation. To resolve the problem, this paper presents a new model-based tracking method for estimating the perimysium motion from 4-D ultrasound images. From the first frame of the given motion images, the proposed method builds a perimysium model, which consists of 3-D surface and rotation-invariant feature descriptor (RIFD) to characterize its structural and image appearances. Then, the model is applied to the next frame using Kalman filter for estimating the best matching position with the highest similarity of RIFD. The estimation is used to update the motion state for predicting and refining the model position in the next frame. The Kalman filtering is iteratively performed until the entire image sequence is processed. Overall, the proposed method efficiently combines the structure, image and motion priors, so it can overcome the aforementioned difficulties. Experimental results showed that the proposed method can provide reliable and accurate estimation of perimysium motion with tracking errors 6.26 voxels using three 4-D ultrasound volumes.