Hierarchical Posture Representation for Robust Action Recognition

Abstract

By modeling an action as the evolution of postures, Posture-based recognition algorithms possess interpretative patterns. However, there are two limitations resulted from individual diversity. First, the same action can be performed by different organs. Such actions are denoted as ambiguous actions. Second, the postures of the same action can be badly influenced by personal characteristics (e.g., personal habits and the height). In order to tackle the problems above, we propose a hierarchical posture representation (HPR). A posture is composed of several skeletal points. Each skeletal point can perform a unique operation and provide a certain amount of information. Thus, an action can be recognized based on the relationships and the information contribution levels of all skeletal points. In HPR, each skeletal point is first represented by its interaction features with other skeletal points. The interaction features are independent of the type of posture. Then, the information contribution level of each skeletal point is estimated based on its interaction features. An end-to-end adaptive action recognition network (AARN) is proposed to accomplish the three processes (estimating the information contribution level, mining the relationships among skeletal points, and recognizing actions). The proposed algorithm shows promising accuracy on two databases. Our work does not only solve the problem above but also introduce a novel action modeling method.