Richard Thavot

Methods to explore design space for MPEG RMC codec specifications

The recent MPEG Reconfigurable Media Coding (RMC) standard aims at defining media processing specifications (e.g. video codecs) in a form that abstracts from the implementation platform, but at the same time is an appropriate starting point for implementation on specific targets. To this end, the RMC framework has standardized both an asynchronous dataflow model of computation and an associated specification language. Either are providing the formalism and the theoretical foundation for multimedia specifications. Even though these specifications are abstract and platform-independent the new approach of developing implementations from such initial specifications presents obvious advantages over the approaches based on classical sequential specifications. The advantages appear particularly appealing when targeting the current and emerging homogeneous and heterogeneous manycore or multicore processing platforms. These highly parallel computing machines are gradually replacing single-core processors, particularly when the system design aims at reducing power dissipation or at increasing throughput. However, a straightforward mapping of an abstract dataflow specification onto a concurrent and heterogeneous platform does often not produce an efficient result. Before an abstract specification can be translated into an efficient implementation in software and hardware, the dataflow networks need to be partitioned and then mapped to individual processing elements. Moreover, system performance requirements need to be accounted for in the design optimization process. This paper discusses the state of the art of the combinatorial problems that need to be faced at this design space exploration step. Some recent developments and experimental results for image and video coding applications are illustrated. Both well-known and novel heuristics for problems such as mapping, scheduling and buffer minimization are investigated in the specific context of exploring the design space of dataflow program implementations.

High-level dataflow design of signal processing systems for reconfigurable and multicore heterogeneous platforms

The potential computational power of today multicore processors has drastically improved compared to the single processor architecture. Since the trend of increasing the processor frequency is almost over, the competition for increased performance has moved on the number of cores. Consequently, the fundamental feature of system designs and their associated design flows and tools need to change, so that, to support the scalable parallelism and the design portability. The same feature can be exploited to design reconfigurable hardware, such as FPGAs, which leads to rethink the mapping of sequential algorithms to HDL. The sequential programming paradigm, widely used for programming single processor systems, does not naturally provide explicit or implicit forms of scalable parallelism. Conversely, dataflow programming is an approach that naturally provides parallelism and the potential to unify SW and HDL designs on heterogeneous platforms. This study describes a dataflow-based design methodology aiming at a unified co-design and co-synthesis of heterogeneous systems. Experimental results on the implementation of a JPEG codec and a MPEG 4 SP decoder on heterogeneous platforms demonstrate the flexibility and capabilities of this design approach.

Secure computing with the MPEG RVC framework

Recently, ISO/IEC standardized a dataflow-programming framework called Reconfigurable Video Coding (RVC) for the specification of video codecs. The RVC framework aims at providing the specification of a system at a high abstraction level so that the functionality (or behavior) of the system become independent of implementation details. The idea is to specify a system so that only intrinsic features of the algorithms are explicitly expressed, whereas implementation choices can then be made only once specific target platforms have been chosen. With this system design approach, one abstract design can be used to automatically create implementations towards multiple target platforms. In this paper, we report our investigations on applying the methodology standardized by the MPEG RVC framework to develop secure computing in the domains of cryptography and multimedia security, leading to the conclusion that the RVC framework can successfully be applied as a general-purpose framework to other fields beyond multimedia coding. This paper also highlights the challenges we faced in conducting our study, and how our study helped the RVC and the secure computing communities benefited from each other. Our investigations started with the development of a Crypto Tools Library (CTL) based on RVC, which covers a number of widely used ciphers and cryptographic hash functions such as AES, Triple DES, ARC4 and SHA-2. Performance benchmarking results on the RVC-based AES and SHA-2 implementations in both C and Java revealed that the automatically generated implementations can achieve a comparable performance to some manually written reference implementations. We also demonstrated that the RVC framework can easily produce implementations with multi-core support without any change to the RVC code. A security protocol for mutual authentication was also implemented to demonstrate how one can build heterogeneous systems easily with RVC. By combining CTL with Video Tool Library (a standard library defined by the RVC standard), a non-standard RVC-based H.264/AVC encoder and a non-standard RVC-based JPEG codec, we further demonstrated the benefits of using RVC to develop different kinds of multimedia security applications, which include joint multimedia encryption-compression schemes, digital watermarking and image steganography in JPEG compressed domain. Our study has shown that RVC can be used as a general-purpose implementation-independent development framework for diverse data-driven applications with different complexities.

Motion Estimation Accelerator with User Search Strategy in an RVC Context

Motion estimation represents a key module in video compression. The RVC context requires proposing a flexible solution for motion estimation. According to the nature of the application, a full search is sometimes not suitable, hence, alternative fast/reduced solutions should be considered. This paper proposes a model and implementation of a flexible motion estimation engine, which can be configured to support any user-defined search strategy. Typically, the computational requirements of the search strategy can be traded with the RD-performance of the obtained video encoder. A CAL dataflow description of the accelerator is proposed so that it can be easily handled in the RVC context. An automatic translation of the proposed CAL module to HDL is performed, and a comparison between the generated HDL with handwritten HDL code of the same model is performed. This helps to evaluate the influences of the CAL model refinements.

Hardware/software co-design of dataflow programs for reconfigurable hardware and multi-core platforms

The possibility of specifying both software and hardware components from a unified high-level description of an application is a very attractive design approach. However, despite the efforts spent for implementing such an approach using general purpose programming languages, it has not yet shown to be viable and efficient for complex designs. One of the reasons is that the sequential programming model does not naturally provide explicit and scalable parallelism and composability properties that effectively permits to build portable applications that can be efficiently mapped on different kind of heterogeneous platforms. Conversely dataflow programming is an approach that naturally provides explicit parallel programs with composability properties. This paper presents a methodology for the hardware/software co-design that enables, by direct synthesis of both hardware descriptions (HDL), software components (C/C++) and mutual interfaces, to generate an implementation of the application from an unique dataflow program, running onto heterogeneous architectures composed by reconfigurable hardware and multi-core processors. Experimental results based on the implementation of a JPEG codec onto an heterogeneous platform are also provided to show the capabilities and flexibility of the implementation approach.

Methodology for the hardware/software co-design of dataflow programs

New generations of multi-core processors and reconfigurable hardware platforms are expected to provide a dramatic increase of processing capabilities. However, one obstacle for exploiting all the promises of such new platforms is the legacy of current applications and the development methodologies used, which is deeply rooted in a sequential way of thinking. A paradigm shift is necessary at all levels of application development to yield portable and efficient implementations, capable of exploiting the full potential of such platforms. Dataflow programming is an alternative approach that address the problem of providing portable and scalable parallel applications. Dataflow programming is able to explicitly expose the intrinsic parallelism of applications. This paper presents a hardware/software co-design methodology that starting from a unique dataflow program enables, by the direct synthesis of both hardware (HDL) and software components (C/C++), to map a signal processing application onto heterogeneous systems architectures composed by reconfigurable hardware and multi-core processors. Experimental results based on the implementation of the MPEG-4 Simple Profile decoder onto an heterogeneous platform are also provided to show the capabilities and flexibility of the approach.

An efficient hardware implementation of Diamond Search motion estimation using CAL dataflow language

Motion estimation represents a key module in video compression. The Reconfigurable Video Coding context (RVC) requires proposing a flexible solution for motion estimation. The motion estimation performance should be modified to fit with the user or the environments constraints. Depending on the required performances fixed by the application, a full search is sometimes not suitable, hence, alternative fast/reduced solutions should be considered. In this paper, an efficient Diamond Search motion estimation, described in RVC-CAL actor language, is introduced. Starting from a high level description based CAL language, an automatic translation of the proposed CAL module to HDL is performed. This is meant to give designers an edge over its market competitors by substantially reducing the Time to Market (TTM). A comparison between the generated HDL with a previous work supporting user search strategy and handwritten code is also provided. The results show substantial improvement in term of operating frequency and silicon area to previous work based CAL description.