MTBR: Multi-Target Boosting for Regression

Abstract

Gradient boosting method has been successfully used for single target prediction problems. In real world applications, however, problems involving the prediction of multiple target attributes are often of interest. In this paper, a multi-target boosting method for regression problems, named as MTBR, is proposed. Although MTBR builds one model for each target attribute separately, all the target attributes are utilized when building each model. In each boosting iteration, the base learner, the regression tree in particular, is learned by selecting the best models from all the target attributes. We also introduce a novel knowledge transfer approach. That is, the tree structure learned from one target attribute, representing a way to partition the feature space, is used to predict another target attribute. Experiments with six data sets compare MTBR to other ensemble regression methods, and prove the effectiveness of MTBR in leveraging the knowledge of multiple target attributes and improving the model accuracy.