Glass reflection removal using co-saliency based image alignment and low-rank matrix completion in gradient domain

Abstract

The images taken through glass often capture a target transmitted scene as well as undesired reflected scenes. In this paper, we propose a novel reflection removal algorithm using multiple glass images taken from slightly different camera positions. We first find co-saliency maps for input multiple glass images based on the center prior assumption, and then align multiple images reliably with respect to the transmitted scene by selecting feature points with high co-saliency values. The gradients of the transmission images are consistent while the gradients of the reflection images are varying across the aligned multiple glass images. Based on this observation, we compute gradient reliability such that the pixels belonging to consistent salient edges of the transmission image are assigned high reliability values. We restore the gradients of the transmission images and suppress the gradients of the reflection images by formulating a low-rank matrix completion problem in gradient domain. Finally, we reconstruct desired transmission images from the restored transmission gradients. Experimental results show that the proposed algorithm removes the reflection artifacts from glass images faithfully and outperforms the existing methods on challenging glass images with diverse characteristics.

Experimental Results

Comparison of the reflection removal algorithms. (a) Reference glass images. The reconstructed transmission images by using (b) RS [5], (c) SID [26], (d) ERC [25], (e) LMC [28], and (f) the proposed algorithm, respectively.

Publication

Byeong-Ju Han and Jae-Young Sim, “Glass reflection removal using co-saliency based image alignment and low-rank matrix completion in gradient domain,” IEEE Transactions on Image Processing, vol. 27, no. 10, pp. 4873-4888, Oct. 2018.