Gaurush Hiranandani

Summarizing Multimedia Content

Today multimedia content comprising both text and images is growing at a rapid pace. There has been a body of work to summarize text content, but to the best of our knowledge, no method has been developed to summarize multimedia content. We propose two methods for summarizing multimedia content. Our novel approach explicitly recognizes two desirable, normative characteristics of a summary - good coverage and diversity of the respective text and images, and that text and images should be coherent with each other. Two methods are examined - graph based and a modification to the submodular approach. Moreover, we propose a metric to measure the quality of a multimedia summary which captures coverage and diversity of text and images as well as coherence between the text and images in the summary. We experimentally demonstrate that the proposed methods achieve good quality multimedia summaries.

[POSTER] Enhanced Personalized Targeting Using Augmented Reality

Augmented Reality (AR) based applications have existed for some time; however, their true potential in digital marketing remains unexploited. To bridge this gap we create a novel consumer targeting system. First, we analyze consumer interactions on AR-based retail apps to identify her preferred purchase viewpoint during the session. We then target the consumer through a personalized catalog, created by embedding recommended products in her viewpoint visual. The color and style of the embedded product are matched with the viewpoint to create recommendations, and personalized text content is created using visual cues from the AR data. Evaluation with user studies show that our system is able to identify the viewpoint, our recommendations are better than tag-based recommendations, and targeting using the viewpoint is better than that of usual product catalogs.

Improved Classification and Reconstruction by Introducing Independence and Randomization in Deep Neural Networks

This paper deals with a novel way of improving classification as well as reconstructions obtained from deep neural networks. The underlying ideas that have been used throughout are Independence and Randomization. The idea is to expose the inherent properties of neural network architectures and to make simpler models that are easy to implement rather than creating highly fine-tuned and complex neural network architectures. For the most basic type of deep neural network i.e. fully connected, it has been shown that dividing the data into independent components and training each component separately not only reduces the parameters to be learned but also the training is more efficient. And if the predictions are fused appropriately the overall accuracy also increases. Using the orthogonality of LAB colour space, it is shown that L,A and B components trained separately produce better reconstructions than RGB components taken together which in turn produce better reconstructions than LAB components taken together. Based on a similar approach, randomization has been injected into the networks so as to make different networks as independent as possible. Again fusing predictions appropriately increases accuracy. The best error on MNISTs test data set was 1.91% which is a drop by 1.05% in comparison to architectures that we created similar to [1]. As the technique is architecture independent it can be applied to other networks - for example CNNs or RNNs.