A DNN Optimization Framework with Unlabeled Data for Efficient and Accurate Reconfigurable Hardware Inference

Abstract

Open-source deep-learning frameworks are prevalent in designing, training, and deploying deep neural networks (DNNs) on general-purpose computing devices, such as CPU, GPU, and DSP. However, for custom-designed reconfigurable hardware accelerators, there is no existing universal framework, capable of optimizing DNN deployment configuration and guiding the hardware design with specific accuracy and efficiency requirements. In the paper, we proposed a cross- platform framework, which can convert deep-learning models from popular open-source frameworks to intermediate representation and optimize weight/activation dynamic ranges and quantization strategy to achieve better efficiency and accuracy based on a baseline reference design of hardware accelerator. With a few unlabeled data, the proposed framework can analyze the statistical inference information, compare different bit-width impacts, and optimize network structure. We further illustrate the detailed experiment results using the framework, showing mAP and top-1 accuracy loss is less than 1.5% and 1.2% with 12-bit and 8-bit activation-constrained quantization schemes respectively for object detection and image classification.