Multimodal Continual Learning Using Online Dictionary Updating

Abstract

This article studies the continual learning task from the perspective of multimodal fusion. In the multimodal fusion problem, the unified representations of heterogeneous modalities are continuously updated, while the classifier is also constantly refreshed for each multimodal learning task. To tackle this point, we establish a continual learning framework for multimodal learning and design an effective online dictionary updating method. Finally, we experimentally verify a complex material identification task and obtain promising results.