Karamjit Singh

Comparative benchmarking of causal discovery algorithms

In this paper we present a comprehensive view of prominent causal discovery algorithms, categorized into two main categories (1) assuming acyclic and no latent variables, and (2) allowing both cycles and latent variables, along with experimental results comparing them from three perspectives: (a) structural accuracy, (b) standard predictive accuracy, and (c) accuracy of counterfactual inference. For (b) and (c) we train causal Bayesian networks with structures as predicted by each causal discovery technique to carry out counterfactual or standard predictive inference. We compare causal algorithms on two publicly available and one simulated datasets having different sample sizes: small, medium and large. Experiments show that structural accuracy of a technique does not necessarily correlate with higher accuracy of inferencing tasks. Further, surveyed structure learning algorithms do not perform well in terms of structural accuracy in case of datasets having large number of variables.

Automated Product-Attribute Mapping

Aggregate analysis, such as comparing country-wise sales versus global market share across product categories, is often complicated by the unavailability of common join attributes, e.g., category, across diverse datasets from different geographies or retail chains. Sometimes this is a missing data issue, while in other cases it may be inherent, e.g., the records in different geographical databases may actually describe different product ‘SKUs’, or follow different norms for categorization. Often a tedious manual mapping process is often employed in practice. We focus on improving such a process using machine-learning driven automation. Record linkage techniques, such as [5] can be used to automatically map products in different data sources to a common set of global attributes, thereby enabling federated aggregation joins to be performed. Traditional record-linkage techniques are typically unsupervised, relying textual similarity features across attributes to estimate matches. In this paper, we present an ensemble model combining minimal supervision using Bayesian network models together with unsupervised textual matching for automating such ‘attribute fusion’. We present results of our approach on a large volume of real-life data from a market-research scenario and compare with a standard record matching algorithm. Our approach is especially suited for practical implementation since we also provide confidence values for matches, enabling routing of items for human intervention where required.

Group-Based Pricing to Shape Demand in Real-Time Electricity Markets

Maintaining the balance between electricity supply and demand is one of the major concerns of utility operators. With the increasing contribution of renewable energy sources in the typical supply portfolio of an energy provider, volatility in supply is increasing while the control is decreasing. Real time pricing based on aggregate demand, unfortunately cannot control the non-linear price sensitivity of deferrable/flexible loads and leads to other peaks [4, 5] due to overly homogenous consumption response. In this paper, we present a day-ahead group-based real-time pricing mechanism for optimal demand shaping. We use agent-based simulations to model the system-wide consequences of deploying different pricing mechanisms and design a heuristic search mechanism in the strategy space to efficiently arrive at an optimal strategy. We prescribe a pricing mechanism for each groups of consumers, such that even though consumption synchrony within each group gives rise to local peaks, these happen at different time slots, which when aggregated result in a flattened macro demand response. Simulation results show that our group-based pricing strategy out-performs traditional real-time pricing, and results in a fairly flat peak-to-average ratio.