Text Classification Model Based on BERT-Capsule with Integrated Deep Learning

Abstract

Current text classification methods based on traditional capsule network models cannot properly reflect the importance of different words in a text sequence, and cannot effectively extract multi-level semantic features in text. To address the shortcomings of the traditional capsule network model, a text classification model based on BERT-Capsule integrated deep learning is proposed, which takes advantage of BERT s bi-directional encoding of text features and the improved routing mechanism of the capsule network as the basis. The model not only extracts the contexts information of text more comprehensively, but also learns the local word features and global semantic features of text to ensure the stability of the result classification. Finally, the BERT model before integration, the capsule network model and the BERT-Capsule model after integration are evaluated by instances on IMDB, AG News, and Reuters-21578 datasets. The experimental results demonstrate that the BERT-Capsule integrated model outperforms the BERT model before integration and the capsule network model in terms of accuracy of text classification, and the generalization error is lower, and the model can be used for text classification.