Learning to estimate indoor illumination

Authors: Gardner, Marc-André
Advisor: Lalonde, Jean-FrançoisGagné, Christian
Abstract: Producing images mixing real and virtual elements in a realistic fashion requires knowing the illumination conditions. Indeed, rendering engines need this lighting information to adjust the appearance of the objects in such a way that they visually blend in the surrounding scene. As such, any mismatch can break the illusion and reveal the presence of these virtual inserted objects. At a time when computer generated graphics are barely out of the infamous uncanny valley, obtaining accurate lighting conditions is thus a crucial part of many artistic pipelines. There exist approaches to measure a scene illumination, but they rely on specialized hardware, require careful calibration, and cannot be applied a posteriori, for instance when the picture we want to work with is already taken. In this thesis, we show how indoor illumination estimation can be framed as an end-to-end learning problem, and how a deep neural network can reliably estimate lighting information using a single, limited field-of-view, low dynamic range image (as a regular camera would produce). More specifically, we introduce two learning models for this task: 1) a method regressing an entire high dynamic range (HDR) panorama from a single image, and 2) a method estimating illumination in the form of a reduced set of lighting parameters. We also extend the latter to support an arbitrary number of images as input, in addition to the single image case. We provide detailed justifications and performance analysis for each of these methods, in addition to qualitative results demonstrating the effectiveness of our approaches for common artistic tasks and pipelines. The work presented in this thesis has several important and practical applications. Graphics domains such as special effects, augmented and virtual reality, and image editing immediately come to mind, but the field of potential applications is far vaster. From architecture (with the production or realistic mock-ups) to piloting and driving simulators (which would benefit from a more realistic illumination), from better personal entertainment to more interactive and intuitive approaches in education, the potential applications are virtually limitless. Overall, everything somehow linked to imaging and lighting can potentially be improved using the techniques we present in this thesis, which underlines the importance of the problem tackled in this work.
Document Type: Thèse de doctorat
Issue Date: 2020
Open Access Date: 23 November 2020
Permalink: http://hdl.handle.net/20.500.11794/67302
Grantor: Université Laval
Collection:Thèses et mémoires

Files in this item:
Description SizeFormat 
36050.pdf216.81 MBAdobe PDFThumbnail
All documents in CorpusUL are protected by Copyright Act of Canada.