GLEAM -- An Illumination Estimation Framework for Real-time Photorealistic Augmented Reality on Mobile Devices (demo)

Abstract

Mixed reality mobile platforms attempt to co-locate virtual scenes with physical environments, towards creating immersive user experiences. However, to create visual harmony between virtual and physical spaces, the virtual scene must be accurately illuminated with realistic lighting that matches the physical environment. To this end, we design GLEAM, a framework that provides robust illumination estimation in real-time by integrating physical light-probe estimation with current mobile AR systems. We present a demo implementation of GLEAM by means of an AR application that estimates environmental illumination and renders the scene with real-time illumination updates. We demonstrate the efficacy of GLEAM s estimation against a current commercial status quo solution, Apple s ARKit, with the same application.