Finding Hierarchical Structures of Disordered Systems: An Application for Market Basket Analysis

Abstract

Complex systems can be characterized by their level of order or disorder. An ordered system is related to the presence of system properties that are correlated with each other. For example, it has been found in crisis periods that the financial systems tend to be synchronized, and symmetry appears in financial assets’ behavior. In retail, the collective purchasing behavior tends to be highly disorderly, with a diversity of correlation patterns appearing between the available market supply. In those cases, it is essential to understand the hierarchical structures underlying these systems. For the latter, community detection techniques have been developed to find similar behavior clusters according to some similarity measure. However, these techniques do not consider the inherent interactions between the multitude of system elements. This paper proposes and tests an approach that incorporates a hierarchical grouping process capable of dealing with complete weighted networks. Experiments show that the proposal is superior in terms of the ability to find minimal energy clusters. These minimum energy clusters are equivalent to system states (market baskets) with a higher probability of occurrence; therefore, they are interesting for marketing and promotion activities in retail environments.