An Adaptive Device-Aware Model Optimization Framework

Abstract

Deep learning technology has been widely developed in all walks of life, especially in the medical research field. Recently, the deep neural network model has become a deeper and better direction, and followed by the problem of computing resources. The feasibility of a large neural network model can be evaluated by its suitability to sophisticated medical devices. With this basis, we propose an adaptive model optimization framework (AMOF). Compared to reported model compression techniques, we focus on the correlation between channels. AMOF cannot only output an accurate compression ratio, but also search for the optimal pruning channel. Specifically, evolutionary algorithms were introduced on the basis of reinforcement learning. Due to the complexity of a neural network, we propose a co-evolutionary algorithm, so as to guarantee the simultaneous evolution of multiple populations and finally output the optimal cutting channel. Notably, AMOF, combining reinforcement learning and evolutionary algorithm, can ensure the accuracy of this model applied under the full compression condition. The effectiveness of AMOF was proved by a large number of experimental tests. For example, on the CIFAR-10, the ResNet56 channel after our frame trimming was reduced by 30%; and the accuracy remained at 89.27%. Compared to the reinforcement learning compression method alone, AMOF can increase by 3.5 percentage points in the ResNet20 model.