Hsuan-Tien Lin

A note on Platt's probabilistic outputs for support vector machines

Abstract Platt’s probabilistic outputs for Support Vector Machines (Platt, J. in Smola, A., et al. (eds.) Advances in large margin classifiers. Cambridge, 2000) has been popular for applications that require posterior class probabilities. In this note, we propose an improved algorithm that theoretically converges and avoids numerical difficulties. A simple and ready-to-use pseudo code is included.

Reduction from cost-sensitive ordinal ranking to weighted binary classification

We present a reduction framework from ordinal ranking to binary classification. The framework consists of three steps: extracting extended examples from the original examples, learning a binary classifier on the extended examples with any binary classification algorithm, and constructing a ranker from the binary classifier. Based on the framework, we show that a weighted 0/1 loss of the binary classifier upper-bounds the mislabeling cost of the ranker, both error-wise and regret-wise. Our framework allows not only the design of good ordinal ranking algorithms based on well-tuned binary classification approaches, but also the derivation of new generalization bounds for ordinal ranking from known bounds for binary classification. In addition, our framework unifies many existing ordinal ranking algorithms, such as perceptron ranking and support vector ordinal regression. When compared empirically on benchmark data sets, some of our newly designed algorithms enjoy advantages in terms of both training speed and generalization performance over existing algorithms. In addition, the newly designed algorithms lead to better cost-sensitive ordinal ranking performance, as well as improved listwise ranking performance.

Unsupervised Semantic Feature Discovery for Image Object Retrieval and Tag Refinement

We have witnessed the exponential growth of images and videos with the prevalence of capture devices and the ease of social services such as Flickr and Facebook. Meanwhile, enormous media collections are along with rich contextual cues such as tags, geo-locations, descriptions, and time. To obtain desired images, users usually issue a query to a search engine using either an image or keywords. Therefore, the existing solutions for image retrieval rely on either the image contents (e.g., low-level features) or the surrounding texts (e.g., descriptions, tags) only. Those solutions usually suffer from low recall rates because small changes in lighting conditions, viewpoints, occlusions, or (missing) noisy tags can degrade the performance significantly. In this work, we tackle the problem by leveraging both the image contents and associated textual information in the social media to approximate the semantic representations for the two modalities. We propose a general framework to augment each image with relevant semantic (visual and textual) features by using graphs among images. The framework automatically discovers relevant semantic features by propagation and selection in textual and visual image graphs in an unsupervised manner. We investigate the effectiveness of the framework when using different optimization methods for maximizing efficiency. The proposed framework can be directly applied to various applications, such as keyword-based image search, image object retrieval, and tag refinement. Experimental results confirm that the proposed framework effectively improves the performance of these emerging image retrieval applications.

Do all the patients with gastric parietal cell antibodies have pernicious anemia

OBJECTIVE This study evaluated whether all the patients with serum gastric parietal cell antibody (GPCA) positivity had pernicious anemia (PA). MATERIALS AND METHODS The blood hemoglobin (Hb), iron, and vitamin B12 concentrations, and mean corpuscular volume (MCV) in 124 GPCA-positive patients were measured and compared with the corresponding data in 124 age- and sex-matched healthy controls. PA was defined by World Health Organization (WHO) as having an Hb concentration < 13 g dl(-1) for men and < 12 g dl(-1) for women, an MCV ≥ 100 fl, and a serum vitamin B12 level < 200 pg ml(-1) . RESULTS We found that 20, 25, and 20 GPCA-positive patients had deficiencies of Hb (men < 13 g dl(-1) , women < 12 g dl(-1) ), iron (<60 μg dl(-1) ), and vitamin B12 (<200 pg ml(-1) ), respectively. Moreover, 16 GPCA-positive patients had abnormally high MCV (≥ 100 fl). GPCA-positive patients had a significantly higher frequency of Hb, iron, or vitamin B12 deficiency and of abnormally high MCV (all P-values < 0.001) than healthy controls. However, only 12.9% of 124 GPCA-positive patients were diagnosed as having PA by the WHO definition. CONCLUSION Only 12.9% of GPCA-positive patients are discovered to have PA by the WHO definition.

Significant reduction of homocysteine level with multiple B vitamins in atrophic glossitis patients

OBJECTIVE This study evaluated whether supplementations of different vitamins and iron could reduce the serum homocysteine levels in 91 atrophic glossitis (AG) patients. MATERIALS AND METHODS Atrophic glossitis (AG) patients with concomitant deficiencies of vitamin B12 only (n = 39, group I), folic acid only (n = 10, group II), iron only (n = 9, group III), or vitamin B12 plus iron (n = 19, group IV) were treated with vitamin BC capsules plus deficient hematinics. AG patients without definite hematinic deficiencies (n = 14, group V) were treated with vitamin BC capsules only. The blood homocysteine and hematinic levels at baseline and after treatment till all oral symptoms had disappeared were measured and compared by paired t-test. RESULTS Supplementations with vitamin BC capsules plus corresponding deficient hematinics for groups I, II, III, IV patients and with vitamin BC capsules only for group V patients could reduce the high serum homocysteine levels to significantly lower levels after a mean treatment period of 8.3-11.6 months (all P-values < 0.05). CONCLUSION Supplementations with vitamin BC capsules plus corresponding deficient hematinics or with vitamin BC capsules only can reduce the high serum homocysteine levels to significantly lower levels in AG patients.

A note on the decomposition methods for support vector regression

The dual formulation of support vector regression involves two closely related sets of variables. When the decomposition method is used, many existing approaches use pairs of indices from these two sets as the working set. Basically, they select a base set first and then expand it so all indices are pairs. This makes the implementation different from that for support vector classification. In addition, a larger optimization subproblem has to be solved in each iteration. We provide theoretical proofs and conduct experiments to show that using the base set as the working set leads to similar convergence (number of iterations). Therefore, by using a smaller working set while keeping a similar number of iterations, the program can be simpler and more efficient.

Unsupervised auxiliary visual words discovery for large-scale image object retrieval

Image object retrieval–locating image occurrences of specific objects in large-scale image collections–is essential for manipulating the sheer amount of photos. Current solutions, mostly based on bags-of-words model, suffer from low recall rate and do not resist noises caused by the changes in lighting, viewpoints, and even occlusions. We propose to augment each image with auxiliary visual words (AVWs), semantically relevant to the search targets. The AVWs are automatically discovered by feature propagation and selection in textual and visual image graphs in an unsupervised manner. We investigate variant optimization methods for effectiveness and scalability in large-scale image collections. Experimenting in the large-scale consumer photos, we found that the the proposed method significantly improves the traditional bag-of-words (111% relatively). Meanwhile, the selection process can also notably reduce the number of features (to 1.4%) and can further facilitate indexing in large-scale image object retrieval.

Improving generalization by data categorization

In most of the learning algorithms, examples in the training set are treated equally. Some examples, however, carry more reliable or critical information about the target than the others, and some may carry wrong information. According to their intrinsic margin, examples can be grouped into three categories: typical, critical, and noisy. We propose three methods, namely the selection cost, SVM confidence margin, and AdaBoost data weight, to automatically group training examples into these three categories. Experimental results on artificial datasets show that, although the three methods have quite different nature, they give similar and reasonable categorization. Results with real-world datasets further demonstrate that treating the three data categories differently in learning can improve generalization.

Infinite ensemble learning with support vector machines

Ensemble learning algorithms such as boosting can achieve better performance by averaging over the predictions of base hypotheses. However, existing algorithms are limited to combining only a finite number of hypotheses, and the generated ensemble is usually sparse. It is not clear whether we should construct an ensemble classifier with a larger or even infinite number of hypotheses. In addition, constructing an infinite ensemble itself is a challenging task. In this paper, we formulate an infinite ensemble learning framework based on SVM. The framework can output an infinite and nonsparse ensemble, and can be used to construct new kernels for SVM as well as to interpret some existing ones. We demonstrate the framework with a concrete application, the stump kernel, which embodies infinitely many decision stumps. The stump kernel is simple, yet powerful. Experimental results show that SVM with the stump kernel is usually superior than boosting, even with noisy data.

Large-Margin thresholded ensembles for ordinal regression: theory and practice

We propose a thresholded ensemble model for ordinal regression problems. The model consists of a weighted ensemble of confidence functions and an ordered vector of thresholds. We derive novel large-margin bounds of common error functions, such as the classification error and the absolute error. In addition to some existing algorithms, we also study two novel boosting approaches for constructing thresholded ensembles. Both our approaches not only are simpler than existing algorithms, but also have a stronger connection to the large-margin bounds. In addition, they have comparable performance to SVM-based algorithms, but enjoy the benefit of faster training. Experimental results on benchmark datasets demonstrate the usefulness of our boosting approaches.