Tiago A. da Fonseca

Macroblock sampling and mode ranking for complexity scalability in mobile H.264 video coding

We propose a framework for complexity scalability in H.264. The prediction is constrained so that only a subset of prediction modes are tested. The test subset is found by ranking the most “popular” modes (those the are most often picked as best) and selecting the modes that maximize their expected occurrence frequency given a complexity constraint. Ranking is performed by selecting a small set of macroblocks (the sampling population), for which all modes are tested. The remaining macroblocks only test the available dominant modes. The modes in the sampling population of a frame are used to process the next frame. Results are shown to verify the performance of the proposed method, which reveals sizeable complexity savings at small penalties.

Complexity-constrained H.264 HD video coding through mode ranking

Video compression is a computation-intensive task and real-time implementation of state-of-the-art video standards is a challenge. In this paper we propose a complexity scalable framework to H.264/AVC prediction module since the “intra” and “inter” prediction steps are responsible for almost all computation complexity. The idea is to employ a subset of prediction modes instead of testing all modes recommended by H.264/AVC standard. Only the most “popular” prediction modes are ranked and tested until reaching a complexity budget. The ranking is done by sampling macroblocks of previous frames. Results show a negligible the quality loss while achieving large computational savings.

Energy-constrained real-time H.264/AVC video coding

Energy consumption has become a leading design constraint for computing devices in order to defray electric bills for individuals and businesses. Over the past years, digital video communication technologies have demanded higher computing power availability and, therefore, higher energy expenditure. In order to meet the challenge to provide software-based video encoding solutions, we adopted an open source software implementation of an H.264 video encoder, the x264 encoder, and optimized its prediction stage in the energy sense (E). Thus, besides looking for the coding options which lead to the best coded representation in terms of rate and distortion (RD), we constrain the process to fit within a certain energy budget. i.e., an RDE optimization. We considered energy as the time integration of the real demanded electric power for a given system. We present an RDE-optimized framework which allows for software-based real-time video compression, meeting the desired targets of electrical consumption, hence, controlling carbon emissions. We show results of energy-constrained compression wherein one can save as much as 35% of the energy with small impact on RD performance.

Complexity-scalable H.264/AVC in an IPP-based video encoder

Real-time high-definition video encoding is a computation-hungry task that challenges software-based solutions. For that, in this work we adopted an Intel software implementation of an H.264 video encoder and optimized its prediction stage in the complexity sense (C). Thus, besides looking for the coding options which lead to the best coded representation in terms of rate and distortion, we constrain the process to fit within a certain time budget. We present an RDC-optimized framework which allows for real-time HD video compression.

Complexity-constrained rate-distortion optimization for h.264/avc video coding

In order to enable real-time software-based video encoding, in this work we optimized the prediction stage of an H.264 video encoder, in the complexity sense. Thus, besides looking for the coding options which lead to the best coded representation in terms of rate and distortion (RD), we constrain to a complexity (C) budget. We present a complexity optimized framework (RDC-optimized) which allows for real-time video compression and that does not make use of frame-skipping to comply to the desired encoding speed. We developed our framework around an open source software implementation of the H.264/AVC, the the ×264 encoder. Results show that tight complexity control is attainable in practice, with very little loss in RD performance.

Video compression complexity reduction with adaptive down-sampling

Encoding video sequences is a computation-demanding task in high-performance codecs. Optimizing this stage may result in a substantial encoding speed-up. In this paper, we propose a faster approach to encode sequences with the H.264/AVC codec, using mixed-resolution. By having some of the frames down-sampled, the overall computation is reduced, without greatly affecting the rate-distortion performance, which remains evaluated using the sequences original resolution. In addition to this approach, a method which regards only the most frequent prediction modes of a frame is incorporated, so that the combined complexity reduction can be significant with negligible performance losses.

A novel methodology for quality assessment of voxelized point clouds

Recent trends in multimedia technologies indicate a significant growth of interest for new imaging modalities that aim to provide immersive experiences by increasing the engagement of the user with the content. Among other solutions, point clouds denote an alternative 3D content representation that allows visualization of static or dynamic scenes in a more immersive way. As in many imaging applications, the visual quality of a point cloud content is of crucial importance, as it directly affects the user experience. Despite the recent efforts from the scientific community, subjective and objective quality assessment for this type of visual data representation remains an open problem. In this paper, we propose a new, alternative framework for quality assessment of point clouds. In particular, we develop a rendering software, which performs real-time voxelization and projection of the 3D point clouds onto 2D planes, while allowing interaction between the user and the projected views. These projected images are then employed by two-dimensional objective quality metrics, in order to predict the perceptual quality of the displayed stimuli. Benchmarking results, using subjective ratings that were obtained through experiments in two test laboratories, show that our framework provides high predictive power and outperforms the state of the art in objective quality assessment of point cloud imaging.