Michael Cavaretta

How to reverse-engineer quality rankings

A good or bad product quality rating can make or break an organization. However, the notion of “quality” is often defined by an independent rating company that does not make the formula for determining the rank of a product publicly available. In order to invest wisely in product development, organizations are starting to use intelligent approaches for determining how funding for product development should be allocated. A critical step in this process is to “reverse-engineer” a rating company’s proprietary model as closely as possible. In this work, we provide a machine learning approach for this task, which optimizes a certain rank statistic that encodes preference information specific to quality rating data. We present experiments on data from a major quality rating company, and provide new methods for evaluating the solution. In addition, we provide an approach to use the reverse-engineered model to achieve a top ranked product in a cost-effective way.

Experience from hosting a corporate prediction market: benefits beyond the forecasts

Prediction markets are virtual stock markets used to gain insight and forecast events by leveraging the wisdom of crowds. Popularly applied in the public to cultural questions (election results, box-office returns), they have recently been applied by corporations to leverage employee knowledge and forecast answers to business questions (sales volumes, products and features, release timing). Determining whether to run a prediction market requires practical experience that is rarely described. Over the last few years, Ford Motor Company obtained practical experience by deploying one of the largest corporate prediction markets known. Business partners in the US, Europe, and South America provided questions on new vehicle features, sales volumes, take rates, pricing, and macroeconomic trends. We describe our experience, including both the strong and weak correlations found between predictions and real world results. Evaluating this methodology goes beyond prediction accuracy, however, since there are many side benefits. In addition to the predictions, we discuss the value of comments, stock price changes over time, the ability to overcome bureaucratic limits, and flexibly filling holes in corporate knowledge, enabling better decision making. We conclude with advice on running prediction markets, including writing good questions, market duration, motivating traders and protecting confidential information.

Data mining challenges in the automotive domain

Automotive companies, such as Ford Motor Company, have no shortage of large databases with abundant opportunities for cost reduction and revenue enhancement. The Data Mining Group at Ford has worked in the areas of Quality, Customer Satisfaction and Warranty Analytics for close to ten years. In this time, we have developed a number of methods for building systems to help the business. One area of particular success has been in warranty analysis. While traditional hazard analysis has been applied at Ford for a number of years, we have used techniques from other industries (e.g. retail), as well as text mining to view warranty analytics in a new way. However, our success has been tempered by serious challenges particularly in the areas of data understanding, computing meaningful aggregations and implementation. Case studies from the automobile industry (warranty, quality, forecasting, etc.) as well as from other industries will be used.

Using data mining to improve supplier release stability

Communications of material requirements between a manufacturer and its supply base is fraught with inefficiencies. Suppliers complain that variation in material quantity requires them to keep extra capacity on hand, as well as preventing optimization of their labor and equipment. The manufacturer experiences issues with instability, also preventing optimization of labor and equipment. This paper proposes using data mining, a series of statistical and artificial intelligence techniques for extracting knowledge from large databases, to identify opportunities for reducing material requirement variation.