Martin Boissier

Data structures for mixed workloads in in-memory databases

Traditionally, enterprise data management is divided into separate systems. Online Transaction Processing (OLTP) systems are focused on the day to day business by being optimized for retrieving and modifying complete entities. Online Analytical Processing (OLAP) systems initiate queries on specific attributes as these applications are optimized to support decision making based on the information gathered from many instances. In parallel both hardware and database applications are subject to steady improvements. For example, todays size of main memory in combination with the column oriented organization of data offer completely new possibilities such as real time analytical ad hoc queries on transactional data. Especially latest development in the area of main memory database systems raises the question whether those databases are capable of handling both OLAP and OLTP workloads in one system. This Paper discusses requirements for main memory database systems managing both workloads and analyses using appropriate data structures.

Main memory databases for enterprise applications

Enterprise applications are traditionally divided in transactional and analytical processing. This separation was essential as growing data volume and more complex requests were no longer performing feasibly on conventional relational databases.

An Interactive Platform to Simulate Dynamic Pricing Competition on Online Marketplaces

E-commerce marketplaces are highly dynamic with constant competition. While this competition is challenging for many merchants, it also provides plenty of opportunities, e.g., by allowing them to automatically adjust prices in order to react to changing market situations. For practitioners however, testing automated pricing strategies is time-consuming and potentially hazardously when done in production. Researchers, on the other side, struggle to study how pricing strategies interact under heavy competition. As a consequence, we built an open continuous time framework to simulate dynamic pricing competition called Price Wars. The microservice-based architecture provides a scalable platform for large competitions with dozens of merchants and a large random stream of consumers. Our platform stores each event in a distributed log. This allows to provide different performance measures enabling users to compare profit and revenue of various repricing strategies in real-time. For researchers, price trajectories are shown which ease evaluating mutual price reactions of competing strategies. Furthermore, merchants can access historical marketplace data and apply machine learning. By providing a set of customizable, artificial merchants, users can easily simulate both simple rule-based strategies as well as sophisticated data-driven strategies using demand learning to optimize their pricing strategies.

Analyzing Data Relevance and Access Patterns of Live Production Database Systems

Access to real-world database systems and their workloads is an invaluable source of information for database researchers. However, usually such full access is not possible due to tracing overheads, data protection, or legal reasons. In this paper, we present a tool set to analyze and compare synthetic and real-world database workloads, their characteristics, and access patterns. This tool set processes SQL workload traces and collects fine-grained access information without requiring direct read access to the production system. To gain insights into large real-world systems, we traced a live production enterprise system of a Global 2000 company and compare it with the synthetic benchmarks TPC-C and TPC-E.

Data-Driven Inventory Management and Dynamic Pricing Competition on Online Marketplaces

Online markets are characterized by competition and limited demand information. In E-commerce, firms compete against each other using data-driven dynamic pricing and ordering strategies. Successfully managing both inventory levels as well as offer prices is a highly challenging task as (i) demand is uncertain, (ii) competitors strategically interact, and (iii) optimized pricing and ordering decisions are mutually dependent. Currently, retailers lack the possibility to test and evaluate their algorithms appropriately before releasing them into the real world. To study joint dynamic ordering and pricing competition on online marketplaces, we built an interactive simulation platform. To be both flexible and scalable, the platform has a microservice-based architecture and allows handling dozens of competing merchants and streams of consumers with configurable characteristics. Further, we deployed and compared different pricing and ordering strategies, from simple rule-based ones to sophisticated datadriven strategies which are based on state-of-the-art demand learning techniques and efficient dynamic optimization models. 1 Joint Ordering and Pricing Competition E-commerce has become highly dynamic and competitive. Merchants are able to automatically adjust prices to react to changing market situations within milliseconds, [Greenwald and Kephart, 1999]. Similarly, they can flexibly reorder items taking into account (i) estimated demand, (ii) inventory holding costs, and (iii) delivery times. To derive effective pricing and ordering decisions over time is challenging considering that numbers of combinations are enormous, demand is uncertain, and markets are steadily changing (cf. [Tsai and Hung, 2009], [Adida and Perakis, 2010], [Chen and Chen, 2015]). Moreover, pricing and ordering strategies mutually affect each other [Rajan et al., 1992]. Simulating the performance of automated ordering and pricing strategies is important as testing is potentially hazardous when done in production. To the best of our knowledge, however, flexible simulation platforms that allow evaluating specific strategies under competition do not exist. Existing platforms, e.g., [Kephart et al., 2000], [DiMicco et al., 2003], are limited in their capabilities: Simulations run on a single machine, offer a limited set of consumer behaviors, simulate solely short sales horizons, and pricing updates or orders are restricted to predefined discrete points in time. We built a continuous time framework (mimicking production marketplaces such as Amazon or eBay [Boissier et al., 2017; PriceWars, 2018]) to simulate dynamic pricing and ordering under competition. Our setup supports customers with heterogeneous buying behaviors. The competitors’ offers include multiple dimensions such as price and quality. The platform allows an arbitrary number of merchants to compete simultaneously. Each merchant can run his preferred repricing and ordering strategy to adjust prices on the marketplace and to order products, respectively. Due to the strategic interaction of competing merchants’ price reactions market situations steadily change. Simulating streams of customer decisions allows generating realized sales events and the firms’ sales revenues. The firms’ inventory levels, their holding costs as well as their ordering costs are determined by their ordering strategies. Finally, the user can easily study the complex interplay of ordering and repricing strategies and, most importantly, their performance (e.g., short and long-term profits). Our system supports self-adapting learning strategies. The platform logs each interaction such as orders, price updates, stock-outs, new offers, sales, etc. This historical data – which is defined as partially observable (sales are private knowledge) – is requested and numerically analyzed by data-driven merchants. Various state-of-the-art machine learning approaches can be applied to quantify how demand (i.e., sales probabilities) is affected by a merchant’s pricing decisions. Further, merchants are able to deploy optimization models [Transchel and Minner, 2009; Schlosser and Boissier, 2017; Yabe et al., 2017]. which are calibrated by estimated sales probabilities to compute optimized data-driven pricing and ordering strategies. In this context, it can be even tried to learn about competitors’ strategies in order to take anticipated price reactions into account. Finally, our framework also allows controlling and measuring the influence of (i) the customers’ buying behavior, (ii) price adjustment frequencies, as well as (iii) the exit or entry of competitors on a strategy’s performance. In addition, different demand learning techniques and optimization approaches can be compared regarding their accuracy and efficiency. Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18)

An Integrated Data Management for Enterprise Systems

Over past decades, higher demands on performance for enterprise systems have led to an increased architectural complexity. Demands as real-time analytics or graph computation add further complexity to the technology stack by adding redundancy and distributing business data over multiple components. We argue that enterprises need to simplify data management and reduce complexity as well as data redundancy. We propose a structured approach using the shearing layer concept with a unified data management to improve adaptability as well as maintainability.

And All of a Sudden: Main Memory Is Less Expensive Than Disk

Even today, the wisdom for storage still is that storing data in main memory is more expensive than storing on disks. While this is true for the price per byte, the picture looks different for price per bandwidth. However, for data driven applications with high throughput demands, I/O bandwidth can easily become the major bottleneck. Comparing costs for different storage types for a given bandwidth requirement shows that the old wisdom of inexpensive disks and expensive main memory is no longer valid in every case. The higher the bandwidth requirements become, the more cost efficient main memory is. And all of sudden: main memory is less expensive than disk.

Improving Box Office Result Predictions for Movies Using Consumer-Centric Models

Recent progress in machine learning and related fields like recommender systems open up new possibilities for data-driven approaches. One example is the prediction of a movies box office revenue, which is highly relevant for optimizing production and marketing. We use individual recommendations and user-based forecast models in a system that forecasts revenue and additionally provides actionable insights for industry professionals. In contrast to most existing models that completely neglect user preferences, our approach allows us to model the most important source for movie success: moviegoer taste and behavior. We divide the problem into three distinct stages: (i) we use matrix factorization recommenders to model each users taste, (ii) we then predict the individual consumption behavior, and (iii) eventually aggregate users to predict the box office result. We compare our approach to the current industry standard and show that the inclusion of user rating data reduces the error by a factor of 2x and outperforms recently published research.

Dynamic Pricing under Competition on Online Marketplaces: A Data-Driven Approach

Most online markets are characterized by competitive settings and limited demand information. Due to the complexity of such markets, efficient pricing strategies are hard to derive. We analyze stochastic dynamic pricing models in competitive markets with multiple offer dimensions, such as price, quality, and rating. In a first step, we use a simulated test market to study how sales probabilities are affected by specific customer behaviors and the strategic interaction of price reaction strategies. Further, we show how different state-of-the-art learning techniques can be used to estimate sales probabilities from partially observable market data. In a second step, we use a dynamic programming model to compute an effective pricing strategy which circumvents the curse of dimensionality. We demonstrate that the strategy is applicable even if the number of competitors is large and their strategies are unknown. We show that our heuristic can be tuned to smoothly balance profitability and speed of sales. Further, our approach is currently applied by a large seller on Amazon for the sale of used books. Sales results show that our data-driven strategy outperforms the rule-based strategy of an experienced seller by a profit increase of more than 20%.

Dealing with the Dimensionality Curse in Dynamic Pricing Competition: Using Frequent Repricing to Compensate Imperfect Market Anticipations

Most sales applications are characterized by competition and limited demand information. For successful pricing strategies, frequent price adjustments as well as anticipation of market dynamics are crucial. Both effects are challenging as competitive markets are complex and computations of optimized pricing adjustments can be time-consuming. We analyze stochastic dynamic pricing models under oligopoly competition for the sale of perishable goods. To circumvent the curse of dimensionality, we propose a heuristic approach to efficiently compute price adjustments. To demonstrate our strategys applicability even if the number of competitors is large and their strategies are unknown, we consider different competitive settings in which competitors frequently and strategically adjust their prices. For all settings, we verify that our heuristic strategy yields promising results. We compare the performance of our heuristic against upper bounds, which are obtained by optimal strategies that take advantage of perfect price anticipations. We find that price adjustment frequencies can have a larger impact on expected profits than price anticipations. Finally, our approach has been applied on Amazon for the sale of used books. We have used a sellers historical market data to calibrate our model. Sales results show that our data-driven strategy outperforms the rule-based strategy of an experienced seller by a profit increase of more than 20%.