Marius Kloft

Predicting MOOC Dropout over Weeks Using Machine Learning Methods

With high dropout rates as observed in many current larger-scale online courses, mechanisms that are able to predict student dropout become increasingly important. While this problem is partially solved for students that are active in online forums, this is not yet the case for the more general student population. In this paper, we present an approach that works on click-stream data. Among other features, the machine learning algorithm takes the weekly history of student data into account and thus is able to notice changes in student behavior over time. In the later phases of a course (i.e., once such history data is available), this approach is able to predict dropout significantly better than baseline methods.

Toward supervised anomaly detection

Anomaly detection is being regarded as an unsupervised learning task as anomalies stem from adversarial or unlikely events with unknown distributions. However, the predictive performance of purely unsupervised anomaly detection often fails to match the required detection rates in many tasks and there exists a need for labeled data to guide the model generation. Our first contribution shows that classical semi-supervised approaches, originating from a supervised classifier, are inappropriate and hardly detect new and unknown anomalies. We argue that semi-supervised anomaly detection needs to ground on the unsupervised learning paradigm and devise a novel algorithm that meets this requirement. Although being intrinsically non-convex, we further show that the optimization problem has a convex equivalent under relatively mild assumptions. Additionally, we propose an active learning strategy to automatically filter candidates for labeling. In an empirical study on network intrusion detection data, we observe that the proposed learning methodology requires much less labeled data than the state-of-the-art, while achieving higher detection accuracies.

A unifying view of multiple kernel learning

Recent research on multiple kernel learning has lead to a number of approaches for combining kernels in regularized risk minimization. The proposed approaches include different formulations of objectives and varying regularization strategies. In this paper we present a unifying optimization criterion for multiple kernel learning and show how existing formulations are subsumed as special cases. We also derive the criterions dual representation, which is suitable for general smooth optimization algorithms. Finally, we evaluate multiple kernel learning in this framework analytically using a Rademacher complexity bound on the generalization error and empirically in a set of experiments.

Automatic feature selection for anomaly detection

A frequent problem in anomaly detection is to decide among different feature sets to be used. For example, various features are known in network intrusion detection based on packet headers, content byte streams or application level protocol parsing. A method for automatic feature selection in anomaly detection is proposed which determines optimal mixture coefficients for various sets of features. The method generalizes the support vector data description (SVDD) and can be expressed as a semi-infinite linear program that can be solved with standard techniques. The case of a single feature set can be handled as a particular case of the proposed method. The experimental evaluation of the new method on unsanitized HTTP data demonstrates that detectors using automatically selected features attain competitive performance, while sparing practitioners from a priori decisions on feature sets to be used.

Active and semi-supervised data domain description

Data domain description techniques aim at deriving concise descriptions of objects belonging to a category of interest. For instance, the support vector domain description (SVDD) learns a hypersphere enclosing the bulk of provided unlabeled data such that points lying outside of the ball are considered anomalous. However, relevant information such as expert and background knowledge remain unused in the unsupervised setting. In this paper, we rephrase data domain description as a semi-supervised learning task, that is, we propose a semi-supervised generalization of data domain description (SSSVDD) to process unlabeled and labeled examples. The corresponding optimization problem is non-convex. We translate it into an unconstraint, continuous problem that can be optimized accurately by gradient-based techniques. Furthermore, we devise an effective active learning strategy to query low-confidence observations. Our empirical evaluation on network intrusion detection and object recognition tasks shows that our SSSVDDs consistently outperform baseline methods in relevant learning settings.

Variable effects of biochar and P solubilizing microbes on crop productivity in different soil conditions

ABSTRACT An expanding body of literature informs that biochar improves soil quality and agricultural productivity. However, there are some reports of little, or even negative, effect of biochar on crop yield, depending on the type of biochar feedstock, pyrolysis process, soil nutrient status, and crop species. Biochar is known to adsorb ammonia and phosphates in soil and facilitate growth and activities of phosphorus (P) solubilizing microbes (PSM), which mobilize P for uptake by plant roots. Using slow-pyrolyzed wood biochar and PSM in different soil conditions in three countries, our experiements show that soil nutrient status is more determinant of beneficial agronomic effect of biochar than the feedstock species and the type of crop. Treatments with biochar and PSM entail significant yield increase in P-deficient soil, whereas in soils with high P content, biochar has no significant effect on crop yield, regardless of addition of PSM. Based on published empirical data as well as our own findings, we also present a mathematical model of plant uptake of bioavailable P at different soil P concentrations, which explains that biochar is ineffective to enhance PSM activity for P mobilization in phosphate-rich soil, but significantly improves crop productivity in P-deficient soil.

A critical assessment of the importance of seedling age in the system of rice intensification (sri) in Eastern India

A survey of the system of rice intensification (SRI)-related literature indicates that different authors have drawn conflicting inferences about rice yield performances under the SRI, chiefly because the SRI methodology has been variously advocated, interpreted and implemented in the field using different rice varieties, seedling ages at transplantation, cultivation seasons and nutrient management regimes. In particular, the SRI method of single-seedling transplantation (SST) has potential economic advantage due to reduced seed costs, but it is not clear whether SST is an effective management strategy across a range of seedling ages, and whether there is any specific seedling age that is optimal for yield improvement of a given rice variety. This is an important consideration in rain-fed ecosystems where variable rainfall patterns and lack of controlled irrigation make it difficult to reliably transplant at a specific seedling age as recommended for the SRI. We conducted a five year-long experiment on a rain-fed organic farm using a short-duration upland and a medium-duration lowland landrace, following the SRI methodology. Rice seedlings of different ages (6, 10, 14, 18 and 28 days after establishment) were transplanted at 25 cm × 25 cm spacing in three replicated plots. The performance for each landrace was examined with respect to productive tillers, panicle density, total grain counts per hill and grain yield per unit area. Performances of seedlings of different ages were compared with that of control plots that employed all SRI practices with the exception that 28-day-old seedlings were transplanted with three seedlings per hill. The results indicate that (1) the SRI can improve mean panicle density if seedling age ≤ 18 days, but that responses differ between varieties; (2) the number of productive tillers per hill is significantly less in SST than that of multiple seedling transplants (MST) of 28-day-old seedlings of both upland and lowland varieties; (3) the total grain numbers per hill of the lowland variety is significantly greater for 14-day-old SST than 28-day-old MST; (4) the grain yield per unit area from young SRI transplants is significantly greater than that from 28-day-old MST for the lowland variety, although the magnitude of the improvement was small; (5) for the upland variety, grain yields declined with the oldest seedlings, but planting multiple seedlings per hill made the yield of the oldest transplants on par with that of younger seedlings planted singly. Our findings suggest that transplanting younger seedlings under the SRI management may not necessarily enhance grain yields.

Feature selection for density level-sets

A frequent problem in density level-set estimation is the choice of the right features that give rise to compact and concise representations of the observed data. We present an efficient feature selection method for density level-set estimation where optimal kernel mixing coefficients and model parameters are determined simultaneously. Our approach generalizes one-class support vector machines and can be equivalently expressed as a semi-infinite linear program that can be solved with interleaved cutting plane algorithms. The experimental evaluation of the new method on network intrusion detection and object recognition tasks demonstrate that our approach not only attains competitive performance but also spares practitioners from a priori decisions on feature sets to be used.

Insights from Classifying Visual Concepts with Multiple Kernel Learning

Combining information from various image features has become a standard technique in concept recognition tasks. However, the optimal way of fusing the resulting kernel functions is usually unknown in practical applications. Multiple kernel learning (MKL) techniques allow to determine an optimal linear combination of such similarity matrices. Classical approaches to MKL promote sparse mixtures. Unfortunately, 1-norm regularized MKL variants are often observed to be outperformed by an unweighted sum kernel. The main contributions of this paper are the following: we apply a recently developed non-sparse MKL variant to state-of-the-art concept recognition tasks from the application domain of computer vision. We provide insights on benefits and limits of non-sparse MKL and compare it against its direct competitors, the sum-kernel SVM and sparse MKL. We report empirical results for the PASCAL VOC 2009 Classification and ImageCLEF2010 Photo Annotation challenge data sets. Data sets (kernel matrices) as well as further information are available at http://doc.ml.tu-berlin.de/image_mkl/(Accessed 2012 Jun 25).

Efficient Algorithms for Exact Inference in Sequence Labeling SVMs

The task of structured output prediction deals with learning general functional dependencies between arbitrary input and output spaces. In this context, two loss-sensitive formulations for maximum-margin training have been proposed in the literature, which are referred to as margin and slack rescaling, respectively. The latter is believed to be more accurate and easier to handle. Nevertheless, it is not popular due to the lack of known efficient inference algorithms; therefore, margin rescaling - which requires a similar type of inference as normal structured prediction - is the most often used approach. Focusing on the task of label sequence learning, we here define a general framework that can handle a large class of inference problems based on Hamming-like loss functions and the concept of decomposability for the underlying joint feature map. In particular, we present an efficient generic algorithm that can handle both rescaling approaches and is guaranteed to find an optimal solution in polynomial time.