Adaptive Hard Example Mining for Image Captioning

Abstract

Reinforcement Learning (RL) based methods optimize evaluation metric directly in image captioning task. In these methods, metric scores of captions are regard as rewards for examples. However, existing methods suffer from inferior performances on hard examples. In this paper, we propose an adaptive hard example mining method with additional supervised training for image captioning. Beam search algorithm is leveraged to estimate score expectation for each example. Examples whose caption scores are lower than expectation are selected automatically. For the selected hard examples, we propose an additional reward policy for high-scoring captions to force model learning from them. The proposed method is hyper-parameter free without tuning. Experimental results on MSCOCO dataset validate effectiveness of the proposed method.