A novel hierarchical interaction model and HITS map for action recognition in static images

This paper proposes a novel fully automatic method to model the low-level human and object interactions for action recognition in the static images. Specifically, we exploit both the superpixels and the grid patches of an image to construct a hierarchical interaction graph and then develop an HITS map learning algorithm to learn the human-object interactions for recognizing the human actions. The major contributions of the paper are three-fold. First, a novel two-layer hierarchical interaction graph based on the superpixels and the grid patches is presented to model the low-level human-object interactions. Second, the novel HITS map, which is derived by the weighted HITS algorithm on the hierarchical interaction graph, assigns heavy weights to the important superpixels and grid patches that reveal more meaningful interactions. Third, the novel weighted image representation is derived from the learned HITS map for action recognition. Extensive experimental results show the feasibility of the proposed method using three representative datasets, namely, the Willow Action dataset, the UIUC Sports Event dataset and the CMU Sports dataset. In particular, the proposed method is able to (i) automatically model the humanobject interactions without extensive manual annotations or numerous error-prone detections, and (ii) improve upon other popular methods in terms of action recognition performance.