Nadia Magnenat Thalmann

Time-aware point-of-interest recommendation

The availability of user check-in data in large volume from the rapid growing location based social networks (LBSNs) enables many important location-aware services to users. Point-of-interest (POI) recommendation is one of such services, which is to recommend places where users have not visited before. Several techniques have been recently proposed for the recommendation service. However, no existing work has considered the temporal information for POI recommendations in LBSNs. We believe that time plays an important role in POI recommendations because most users tend to visit different places at different time in a day, \eg visiting a restaurant at noon and visiting a bar at night. In this paper, we define a new problem, namely, the time-aware POI recommendation, to recommend POIs for a given user at a specified time in a day. To solve the problem, we develop a collaborative recommendation model that is able to incorporate temporal information. Moreover, based on the observation that users tend to visit nearby POIs, we further enhance the recommendation model by considering geographical information. Our experimental results on two real-world datasets show that the proposed approach outperforms the state-of-the-art POI recommendation methods substantially.

Enhancing naive bayes with various smoothing methods for short text classification

Partly due to the proliferance of microblog, short texts are becoming prominent. A huge number of short texts are generated every day, which calls for a method that can efficiently accommodate new data to incrementally adjust classification models. Naive Bayes meets such a need. We apply several smoothing models to Naive Bayes for question topic classification, as an example of short text classification, and study their performance. The experimental results on a large real question data show that the smoothing methods are able to significantly improve the question classification performance of Naive Bayes. We also study the effect of training data size, and question length on performance.

A Generalized Stereotypical Trust Model

Stereotypical trust modeling can be adopted by a buyer to effectively evaluate trustworthiness of a seller who has little or no past experience in e-marketplaces. The buyer forms trust stereotypes based on her past experience with other sellers. However, when the buyer has limited past experience with sellers, the formed stereotypes cannot accurately reflect her trust evaluation towards sellers. To address this issue, we propose a novel generalized stereotypical trust model. Specifically, we first build a semantic ontology to represent hierarchical relationships among seller attribute values. We then propose a fuzzy semantic decision tree (FSDT) learning method to construct trust stereotypes that generalizes over seller non-nominal attributes by splitting their values in a fuzzy manner, and generalizes over nominal attributes by replacing their specific values with more general terms according to the ontology. Experimental results confirm that our proposed model can more accurately measure the trustworthiness of sellers in simulated e-marketplaces where buyers have limited experience with sellers.

Physical simulation of wet clothing for virtual humans

We present a technique that simulates wet garments for virtual humans with realistic folds and wrinkles. Our approach combines three new models to allow realistic simulation of wet garments: (1) a simplified saturation model that modifies the masses, (2) a nonlinear friction model derived from previously reported, real-world measurements, and (3) a wrinkle model based on imperfection sensitivity theory. In contrast to previous approaches to wet cloth, the proposed models are supported by the experimental results reported in the textile literature with parameters varying over the course of the simulation. As a result, the wet garment motions simulated by our method are comparable to that of real wet garments. Our approach recognizes the special, practical importance of contact models with human skin and provides a specific skin-cloth friction solution. We evaluate our approach by draping a rotating sphere and simulating a typical garment on a virtual human in the rain. Both of these examples are typical scenarios within computer graphics research.

Modelling multi-party interactions among virtual characters, robots, and humans

3D virtual humans and physical human-like robots can be used to interact with people in a remote location in order to increase the feeling of presence. In a telepresence setup, their behaviors are driven by real participants. We envision that in the absence of the real users, when they have to leave or they do not want to do a repetitive task, the control of the robots can be handed to an artificial intelligence component to sustain the ongoing interaction. At the point when human-mediated interaction is required again, control can be returned to the real users. One of the main challenges in telepresence research is the adaptation of 3D position and orientation of the remote participants to the actual physical environment to have appropriate eye contact and gesture awareness in a group conversation. In case the human behind the robot and/or virtual human leaves, multi-party interaction should be handed to an artificial intelligence component. In this paper, we discuss the challenges in autonomous multi-party interaction among virtual characters, human-like robots, and real participants, and describe a prototype system to study these challenges.

Progressive surface reconstruction for heart mapping procedure

The composite imaging of measured cardiac properties like electrical activation and contractile properties on a reconstructed endocardial surface allows for the diagnosis of cardiac arrhythmia and myocardial infarct. However, it is difficult for an interventionalist to acquire heart surface contacts by navigating a catheter to the desired region of interest under minimal visual aid. This paper discusses a new method for the progressive reconstruction of an endocardial surface during a heart mapping procedure. A generic mesh is first aligned with a set of anchor points to obtain a first approximation of the surface. Subsequent deformations are constrained by the preservation of local surface characteristics and the fidelity of new contact points. The mesh is refined by local subdivision and its geometrical shape is further improved by edge swapping. Compared to prior art, the new method can reconstruct a realistic surface from a set of sparse and random data. It can advantageously provide a smooth reconstruction at initial acquisition and ensure a geometrical consistency between consecutive reconstructions. The accurate reconstruction of a heart chamber provides important visual cues for an interventionalist to decide on the next mapping site, thus constructively influencing the final diagnosis.

Category hierarchy maintenance: a data-driven approach

Category hierarchies often evolve at a much slower pace than the documents reside in. With newly available documents kept adding into a hierarchy, new topics emerge and documents within the same category become less topically cohesive. In this paper, we propose a novel automatic approach to modifying a given category hierarchy by redistributing its documents into more topically cohesive categories. The modification is achieved with three operations (namely, sprout, merge, and assign) with reference to an auxiliary hierarchy for additional semantic information; the auxiliary hierarchy covers a similar set of topics as the hierarchy to be modified. Our user study shows that the modified category hierarchy is semantically meaningful. As an extrinsic evaluation, we conduct experiments on document classification using real data from Yahoo! Answers and AnswerBag hierarchies, and compare the classification accuracies obtained on the original and the modified hierarchies. Our experiments show that the proposed method achieves much larger classification accuracy improvement compared with several baseline methods for hierarchy modification.

Multiple foreground recognition and cosegmentation: An object-oriented CRF model with robust higher-order potentials

Localizing, recognizing, and segmenting multiple foreground objects jointly from a general users photo stream that records a specific event is an important task with many useful applications. As argued in recent Multiple Foreground Cosegmentation (MFC) work by Kim and Xing, this task is very challenging in that it contrasts substantially from the classical cosegmentation problem, and aims to parse a set of realistic event photos but each containing irregularly occurring multiple foregrounds with high appearance and scene configuration variations. Inspired by the impressive advance in scene understanding and object recognition, this paper casts the multiple foreground recognition and cosegmentation (MFRC) problem within a conditional random fields (CRFs) framework in a principled manner. We capitalize centrally on the key objective that MFRC is to segment out and annotate foreground objects or “things” rather than “stuff”. To this end, we exploit a few complementary objectness cues (e.g. contours, object detectors and layout) and propose novel and efficient methods to capture object-level information. Integrating object potentials as soft constraints (e.g. robust higher-order potentials defined over detected object regions) with low-level unary and pairwise terms holistically, we solve the MFRC task with a probabilistic CRF model. The inference for such a CRF model is performed efficiently with graph cut based move making algorithms. With a minimal amount of user annotations on just a few example photos, the proposed approach produces spatially coherent, boundary-aligned segmentation results with correct and consistent object labeling. Experiments on the FlickrMFC dataset justify that our method achieves state-of-the-art performance.

Object-Level Image Segmentation Using Low Level Cues

This paper considers the problem of automatically segmenting an image into a small number of regions that correspond to objects conveying semantics or high-level structure. Although such object-level segmentation usually requires additional high-level knowledge or learning process, we explore what low level cues can produce for this purpose. Our idea is to construct a feature vector for each pixel, which elaborately integrates spectral attributes, color Gaussian mixture models, and geodesic distance, such that it encodes global color and spatial cues as well as global structure information. Then, we formulate the Potts variational model in terms of the feature vectors to provide a variational image segmentation algorithm that is performed in the feature space. We also propose a heuristic approach to automatically select the number of segments. The use of feature attributes enables the Potts model to produce regions that are coherent in color and position, comply with global structures corresponding to objects or parts of objects and meanwhile maintain a smooth and accurate boundary. We demonstrate the effectiveness of our algorithm against the state-of-the-art with the data set from the famous Berkeley benchmark.

AR in Hand: Egocentric Palm Pose Tracking and Gesture Recognition for Augmented Reality Applications

Wearable devices such as Microsoft Hololens and Google glass are highly popular in recent years. As traditional input hardware is difficult to use on such platforms, vision-based hand pose tracking and gesture control techniques are more suitable alternatives. This demo shows the possibility to interact with 3D contents with bare hands on wearable devices by two Augmented Reality applications, including virtual teapot manipulation and fountain animation in hand. Technically, we use a head-mounted depth camera to capture the RGB-D images from egocentric view, and adopt the random forest to regress for the palm pose and classify the hand gesture simultaneously via a spatial-voting framework. The predicted pose and gesture are used to render the 3D virtual objects, which are overlaid onto the hand region in input RGB images with camera calibration parameters for seamless virtual and real scene synthesis.