Congrats to Peiyao Guo, Zhiyuan Pu! Keep up the good work!
Title: Low-light Color Imaging via Dual CameraAcquisition
Abstract: As existing low-light color imaging suffers from the unrealistic color representation or blurry texture with a single camera setup, we are motivated to devise a dual camera system using a high spatial resolution (HSR) monochrome camera and another low spatial resolution (LSR) color camera for synthesizing the high-quality color image under low-light illumination conditions. The key problem is how to efficiently learn and fuse cross-camera information for improved presentation in such heterogeneous setup with domain gap (e.g., color vs. monochrome, HSR vs. LSR). We have divided the end-to-end pipeline into three consecutive modularized sub-tasks, including the reference-based exposure compensation, reference-based colorization and reference-based super-resolution, to alleviate domain gap and capture inter-camera dynamics between hybrid inputs. In each step, we leverage the powerful deep neural network (DNN) to respectively transfer and enhance the illumination, spectral and spatial granularity in a data-driven way. Each module is first trained separately, and then jointly fine-tuned for robust and reliable performance. Experimental results have shown that our work provides the leading performance in synthetic content from popular test datasets when compared to existing algorithms, and offers appealing color reconstruction using real captured scenes from industrial cameras or smartphone cameras, for low-light color imaging application.
Title: Robust High Dynamic Range (HDR) Imaging with Complex Motion and Parallax
Abstract: High dynamic range (HDR) imaging is widely used in consumer photography, computer game rendering, autonomous driving, and surveillance systems. Reconstructing ghosting-free HDR images of dynamic scenes from a set of multi-exposure images is a challenging task, especially with large object motion, disparity, and occlusions, leading to visible artifacts using existing methods. In this paper, we propose a pyramidal alignment and masked merging network (PAMnet) that learns to synthesis HDR images from input low dynamic range (LDR) images in an end-to-end manner. Instead of aligning under/overexposed images to the reference view directly in pixel-domain, we apply deformable convolutions across multiscale features for pyramidal alignment. Aligned features offer more flexibility to refine the inevitable misalignment for subsequent merging network without reconstructing the aligned image explicitly. To make full use of aligned features, we use dilated dense residual blocks with squeeze-and-excitation (SE) attention. Such attention effectively helps to remove redundant information and suppress misaligned features. Additional mask-based weighting is further employed to refine the HDR reconstruction, which offers better image quality and sharp local details. Experiments demonstrate that the proposed PAMnet can produce ghosting-free HDR results in the presence of large disparity and motion. We present extensive comparative studies using several popular datasets to demonstrate superior quality comapred to the state-of-the-art algorithms.