Performance Evaluation of OpenCL-Enabled Inter-FPGA Optical Link Communication Framework CIRCUS and SMI

Abstract

In recent years, Field Programmable Gate Array (FPGAs) have attracted much attention as accelerators in the research area of HighPerformance Computing (HPC). One of the strong features of current FPGA devices is their ability to achieve high-bandwidth communication performance with direct optical links to construct multi-FPGA platforms as well as their adjustability. However, FPGA programming is not easily performed on user applications. By more user-friendly programming environments, FPGAs can be applied to various HPC applications on multi-FPGA platforms. Of the several studies aimed at realizing high-level synthesis to utilize the FPGA communication feature, we focus on two systems: Communication Integrated Recongurable CompUting System (CIRCUS) and Streaming Message Interface (SMI) which are available on an Intel FPGA with direct optical links with a peak performance of 40 ∼ 100 Gbps. In both systems, a user can access the optical link in OpenCL kernels where high-level programming for HPC applications is possible. In this paper, we introduce them for practical cases and compare their implementations and performance in real systems. In conclusion, we evaluated that the CIRCUS system for single point-to-point communication achieves a bandwidth of up to 90 Gbps with a 100-Gbps optical link using OpenCL code. It is 2.7 times faster than the SMI system implemented on the same platform, and we also confirmed that the broadcast data transfer among four FPGAs using CIRCUS is up to 31 Gbps of bandwidth which is 5.3 times faster compared to that achieved using SMI. In addition, we determined the main cause of the performance bottleneck on SMI when it is applied to a 100-Gbps platform and compared it with the CIRCUS implementation.