Reflection Removal

  • Reflection removal for 2D images

The images taken through glass often capture a target transmitted scene as well as undesired reflected scenes. In this paper, we propose an optimization problem to remove reflection automatically from multiple glass images taken at slightly different camera locations. We first warp the multiple glass images to a reference image, where the gradients are consistent in the transmission images while the gradients are varying across the reflection images. Based on this observation, we compute a gradient reliability such that the pixels belonging to the salient edges of the transmission image are assigned high reliability, but that of the reflection images are assigned low values. Then we suppress the gradients of the reflection images and recover the gradients of the transmission images only, by solving the proposed optimization problem in gradient domain. We reconstruct an original transmission image using the resulting optimal gradient map. Experimental results show that the proposed algorithm removes the reflection artifacts from glass images faithfully and outperforms the existing algorithms.

  • Reflection removal using deep learning

An image captured through a glass plane usually contains both of a target transmitted scene behind the glass plane and a reflected scene in front of the glass plane. We propose a semantic context based network to remove reflection artifacts from a single glass image. We first investigate a non-linear intensity mapping relationship for glass images to synthesize more realistic training sets. Then we devise an efficient reflection removal network using multi-scale generators and an interpreter, where the semantic context of the transmission image is adopted as a high-level cue for the interpreter to guide the generators. We also provide a new test data set of real glass images including the ground truth transmission and reflection images. Experiments are performed on four test data sets and we show that the proposed algorithm decomposes an input glass image into a transmission image and a reflection image more faithfully compared with the four existing state-of-the-art methods.

  • Publications
[1] Byeong-Ju Han and Jae-Young Sim, “Reflection removal using low-rank matrix completion,” in Proc. IEEE CVPR, Honolulu, USA, July 2017. [more]
[2] Byeong-Ju Han and Jae-Young Sim, “Glass reflection removal using co-saliency based image alignment and low-rank matrix completion in gradient domain,” IEEE Trans. Image Process., vol. 27, no. 10, pp. 4873-4888, Oct. 2018. [more]
[3] Byeong-Ju Han and Jae-Young Sim, “Single image reflection removal using non-linearly synthesized glass images and semantic context,” IEEE Access, vol. 7, no. 1, pp. 170796-170806, Nov. 2019. [more]