Nisarg Raval

MarkIt: privacy markers for protecting visual secrets

The increasing popularity of wearable devices that continuously capture video, and the prevalence of third-party applications that utilize these feeds have resulted in a new threat to privacy. In many situations, sensitive objects/regions are maliciously (or accidentally) captured in a video frame by third-party applications. However, current solutions do not allow users to specify and enforce fine grained access control over video feeds. In this paper, we describe MarkIt, a computer vision based privacy marker framework, that allows users to specify and enforce fine grained access control over video feeds. We present two example privacy marker systems -- PrivateEye and WaveOff. We conclude with a discussion of the computer vision, privacy and systems challenges in building a comprehensive system for fine grained access control over video feeds.

What You Mark is What Apps See

Users are increasingly vulnerable to inadvertently leaking sensitive information through cameras. In this paper, we investigate an approach to mitigating the risk of such inadvertent leaks called privacy markers. Privacy markers give users fine-grained control of what visual information an app can access through a devices camera. We present two examples of this approach: PrivateEye, which allows a user to mark regions of a two-dimensional surface as safe to release to an app, and WaveOff, which does the same for three-dimensional objects. We have integrated both systems with Androids camera subsystem. Experiments with our prototype show that a Nexus 5 smartphone can deliver near realtime frame rates while protecting secret information, and a 26-person user study elicited positive feedback on our prototypes speed and ease-of-use.

LSH based outlier detection and its application in distributed setting

In this paper, we give an approximate algorithm for distance based outlier detection using Locality Sensitive Hashing (LSH) technique. We propose an algorithm for the centralized case wherein the entire dataset is locally available for processing. However, in case of very large datasets collected from various input sources, often the data is distributed across the network. Accordingly, we show that our algorithm can be effectively extended to a constant round protocol with low communication costs, in a distributed setting with horizontal partitioning.

Protecting Visual Secrets Using Adversarial Nets

Protecting visual secrets is an important problem due to the prevalence of cameras that continuously monitor our surroundings. Any viable solution to this problem should also minimize the impact on the utility of applications that use images. In this work, we build on the existing work of adversarial learning to design a perturbation mechanism that jointly optimizes privacy and utility objectives. We provide a feasibility study of the proposed mechanism and present ideas on developing a privacy framework based on the adversarial perturbation mechanism.

Image retrieval using eigen queries

Category based image search, where the goal is to retrieve images of a specific category from a large database, is becoming increasingly popular. In such a setting, the query is often a classifier. However, the complexity of the classifiers (often SVMs) used for this purpose hinders the use of such a solution in practice. Problem becomes paramount when the database is huge and/or the dimensionality of the feature representation is also very large. In this paper, we address this issue by proposing a novel method which decomposes the query classifier into set of known eigen queries. We use their precomputed results (or scores) for computing the ranked list corresponding to novel queries. We also propose an approximate algorithm which accesses only a fraction of the data to perform fast retrieval. Experiments on various datasets show that our method reports high accuracy and efficiency. Apart from retrieval, the proposed method can also be used to discover interesting new concepts from the given dataset.

Privacy Preserving Outlier Detection Using Locality Sensitive Hashing

In this paper, we give approximate algorithms for privacy preserving distance based outlier detection for both horizontal and vertical distributions, which scale well to large datasets of high dimensionality in comparison with the existing techniques. In order to achieve efficient private algorithms, we introduce an approximate outlier detection scheme for the centralized setting which is based on the idea of Locality Sensitive Hashing. We also give theoretical and empirical bounds on the level of approximation of the proposed algorithms.

On methods for privacy-preserving energy disaggregation

Household energy monitoring via smart-meters motivates the problem of disaggregating the total energy usage signal into the component energy usage and operating patterns of individual appliances. While energy disaggregation enables useful analytics, it also raises privacy concerns because sensitive household information may also be revealed. Our goal is to preserve analytical utility while mitigating privacy concerns by processing the total energy usage signal. We consider processing methods that attempt to remove the contribution of a set of sensitive appliances from the total energy signal. We show that while a simple model-based approach is effective against an adversary making the same model assumptions, it is much less effective against a stronger adversary employing neural networks in an inference attack. We also investigate the performance of employing neural networks to estimate and remove the energy usage of sensitive appliances. The experiments used the publicly available UK-DALE dataset that was collected from actual households.

ePrivateeye: to the edge and beyond!

Edge computing offers resource-constrained devices low-latency access to high-performance computing infrastructure. In this paper, we present ePrivateEye, an implementation of PrivateEye that offloads computationally expensive computer-vision processing to an edge server. The original PrivateEye locally processed video frames on a mobile device and delivered approximately 20 fps, whereas ePrivateEye transfers frames to a remote server for processing. We present experimental results that utilize our campus Software-Defined Networking infrastructure to characterize how network-path latency, packet loss, and geographic distance impact offloading to the edge in ePrivateEye. We show that offloading video-frame analysis to an edge server at a metro-scale distance allows ePrivateEye to analyze more frames than PrivateEyes local processing over the same period to achieve realtime performance of 30 fps, with perfect precision and negligible impact on energy efficiency.

Demo: What You Mark is What Apps See

The proliferation of camera-equipped computers creates a dilemma. On one hand, cameras enable many useful applications, including video chat, document scanning, and QR-code reading. At the same time, sensitive information in the physical environment can inadvertently leak through image data shared with applications. Preventing leaks by a determined attacker is likely impossible, but as camera-based applications become more central to our work and personal lives, it is imperative to develop tools that provide greater control over what information applications can access through cameras. We have designed and implemented two systems, PrivateEye and WaveOff, that use privacy markers to provide fine-grained access control of information within a cameras view. Our demonstration highlights the accuracy, performance, and usability of these systems.

Efficient Evaluation of SVM Classifiers using Error Space Encoding

Many computer vision tasks require efficient evaluation of Support Vector Machine (SVM) classifiers on large image databases. Our goal is to efficiently evaluate SVM classifiers on a large number of images. We propose a novel Error Space Encoding (ESE) scheme for SVM evaluation which utilizes large number of classifiers already evaluated on the similar data set. We model this problem as an encoding of a novel classifier (query) in terms of the existing classifiers (query logs). With sufficiently large query logs, we show that ESE performs far better than any other existing encoding schemes. With this method we are able to retrieve nearly 100% correct top-k images from a dataset of 1 Million images spanning across 1000 categories. We also demonstrate application of our method in terms of relevance feedback and query expansion mechanism and show that our method achieves the same accuracy 90 times faster than exhaustive SVM evaluations.