Ben Shirley

Object-based audio for interactive football broadcast

An end-to-end AV broadcast system providing an immersive, interactive experience for live events is the development aim for the EU FP7 funded project, FascinatE. The project has developed real time audio object event detection and localisation, scene modelling and processing methods for multimedia data, which will allow users to navigate the event by creating their own unique user-defined scene. As part of the first implementation of the system a test shoot was carried out capturing a live Premier League football game and methods have been developed to detect, analyse, extract and localise salient audio events from a range of sensors and represent them within an audio scene in order to allow free navigation within the scene. Within this context, this paper describes a procedure for the detection, extraction and localisation of ball-kicks and whistle-blows from the pitch-side microphones used in the broadcast of football and describes a potential audio streaming format for an object-based broadcast.

Towards a format-agnostic approach for production, delivery and rendering of immersive media

The media industry is currently being pulled in the often-opposing directions of increased realism (high resolution, stereoscopic, large screen) and personalization (selection and control of content, availability on many devices). We investigate the feasibility of an end-to-end format-agnostic approach to support both these trends. In this paper, different aspects of a format-agnostic capture, production, delivery and rendering system are discussed. At the capture stage, the concept of layered scene representation is introduced, including panoramic video and 3D audio capture. At the analysis stage, a virtual director component is discussed that allows for automatic execution of cinematographic principles, using feature tracking and saliency detection. At the delivery stage, resolution-independent audiovisual transport mechanisms for both managed and unmanaged networks are treated. In the rendering stage, a rendering process that includes the manipulation of audiovisual content to match the connected display and loudspeaker properties is introduced. Different parts of the complete system are revisited demonstrating the requirements and the potential of this advanced concept.

Media Production, Delivery and Interaction for Platform Independent Systems: Format-Agnostic Media

The underlying audio and video processing technology that is discussed in the book relates to areas such as 3D object extraction, audio event detection; 3D sound rendering and face detection, gesture analysis and tracking using video and depth information. The book will give an insight into current trends and developments of future media production, delivery and reproduction. Consideration of the complete production, processing and distribution chain will allow for a full picture to be presented to the reader. Production developments covered will include integrated workflows developed by researchers and industry practitioners as well as capture of ultra-high resolution panoramic video and 3D object based audio across a range of programme genres. Distribution developments will include script based format agnostic network delivery to a full range of devices from large scale public panoramic displays with wavefield synthesis and ambisonic audio reproduction to ’small screen’ mobile devices. Key developments at the consumer end of the chain apply to both passive and interactive viewing modes and will incorporate user interfaces such as gesture recognition and ‘second screen’ devices to allow manipulation of the audio visual content.

Demo paper: Audio object extraction for live sports broadcast

Recent interest in object-based audio systems for cinema opens interesting possibilities to extend the reach of an object-based approach to television broadcast of live events. For events where microphones may be placed close to the source of each sound (e.g. live music) this is relatively straightforward, however for sports events audio objects must be derived from multiple distant microphones. This presents significant challenges in extracting discrete audio objects with coordinate locations from a live event in real time. The research presented here demonstrates the extraction of audio objects from a UK Premier League football (soccer) game using only standard pitch-side microphone positions. Ball-kicks and whistle-blows are extracted in real time with small latency and location information is derived that is used to position the object using 3D coordinates. The resultant sound scene description allows rendering across any reproduction system, including with-height systems. The techniques are equally applicable to other sports with revised object identification templates.

Doctoral Consortium Chairs' Welcome

Welcome from the TVX 2017 Doctoral Consortium Chairs! The ACM TVX 2017 Doctoral Consortium continues its tradition of being an outstanding forum for presentation of doctoral research, results and an excellent opportunity to receive feedback from both industry and academic experts, and from other doctoral students working in research related to interactive experiences for television and online video. The TVX Doctoral Consortium also provides an excellent opportunity to network with your peers and with researchers and industry practitioners and to build connections that can help you in your doctoral study, and further in your career. The TVX 2017 Doctoral Consortium gives researchers and practitioners a unique opportunity to share their perspectives with others interested in the various aspects of interactive media. The world of broadcast and online media has been changing at an unprecedented rate over recent years; 360 video is commonplace on social media platforms used by most of us, VR and AR are gaining traction fast, powerful mobile devices are facilitating new applications in mobile interactive media, broadcasters and broadcast technology providers are beginning to see interactive technology innovation as key to expanding their market. These are exciting, and challenging, times to be working in interactive media research! We would encourage attendees to attend the keynote and invited talk presentations. These valuable and insightful talks can and will guide us to a better understanding of the future of interactive media. Putting together the Doctoral Consortium for TVX 2017 was a team effort. We first thank the authors for providing the content of the program. We are grateful to all those who worked very hard in reviewing papers. We look forward to welcoming you to the conference, hope you will find this program interesting and thought-provoking and that TVX 2017 will provide you with a valuable opportunity to share ideas with other researchers and practitioners from institutions around the world.

In-Programme Personalization for Broadcast: IPP4B

The IPP4B workshop assembled a group of researchers from academia and industry -- BBC R&D, Ericsson and MOG Technologies to discuss the state of the art and together envisage future directions for in programme personalisation in broadcasting. The workshop comprised one invited keynote, two invited presentations together with a paper and discussion sessions.