Saliency Detection

• Saliency detection of LiDAR scene

Large-scale 3D point clouds have been actively used in many applications with the advent of capturing devices. In this work, we propose a novel saliency detection algorithm for large-scale colored 3D point clouds which capture real-world scenes. We first voxelize an input point cloud, and then partition voxels into a supervoxel which corresponds to a clusters at the lowest level. We construct the supervoxel cluster hierarchy iteratively, where a high level cluster includes low level clusters which exhibit similar features to each other. We also estimate the saliency at each cluster by computing the distinctness of geometric and color features based on center-surround contrast. By averaging the multiscale saliency maps obtained at different levels of clusters, we obtain final saliency distribution. Experimental results demonstrate that the proposed algorithm extracts globally and locally salient regions from large-scale colored 3D point clouds faithfully by employing the geometric and photometric features together.

• Video saliency

For videos, there are two kinds of information, spatial and temporal information, which is applicable to detect video saliency. The contribution of each information, however, is not fixed but varying with the characteristic of input videos. Therefore, deciding the contribution is very important to detect proper video saliency. In order to decide the contribution, we adopt compactness prior. We compute the compactness of both spatial and temporal information, and combine them into one unified information. After that, we ascertain local background and foreground based on residual effects of human eyes. Final saliency is decided by the optimized ratio of feature distances from local foreground and background. Experimental results demonstrate that our proposed algorithm detects video saliency precisely and reliably.

• Image saliency

In this paper, we propose a graph-based multiscale saliency-detection algorithm by modeling eye movements as a random walk on a graph. The proposed algorithm first extracts intensity, color, and compactness features from an input image. It then constructs a fully connected graph by employing image blocks as the nodes. It assigns a high edge weight if the two connected nodes have dissimilar intensity and color features and if the ending node is more compact than the starting node. Then, the proposed algorithm computes the stationary distribution of the Markov chain on the graph as the saliency map. However, the performance of the saliency detection depends on the relative block size in an image. To provide a more reliable saliency map, we develop a coarse-to-fine refinement technique for multiscale saliency maps based on the random walk with restart (RWR). Specifically, we use the saliency map at a coarse scale as the restarting distribution of RWR at a fine scale. Experimental results demonstrate that the proposed algorithm detects visual saliency precisely and reliably. Moreover, the proposed algorithm can be efficiently used in the applications of proto-object extraction and image retargeting.

• 3D Mesh saliency

We propose a unified detection algorithm of view-independent and view-dependent saliency for 3D mesh models. While the conventional techniques use the irregular meshes, we adopt the semi-regular meshes to overcome the drawback of irregular connectivity for saliency computation. We employ the angular deviation of normal vectors between neighboring faces as geometric curvature features, which are evaluated at hierarchically structured triangle faces. We construct a fully-connected graph at each level of semi-regular mesh, where the face patches serve as graph nodes. At the base mesh level, we estimate the saliency as the stationary distribution of random walk. At the higher level meshes, we take the maximum value between the stationary distribution of random walk at the current level and an upsampled saliency map from the previous coarser scale. Moreover, we also propose a view-dependent saliency detection method which employs the visibility feature in addition to the geometric features to estimate the saliency with respect to a selected viewpoint. Experimental results demonstrate that the proposed saliency detection algorithm captures global conspicuous regions reliably and detects locally detailed geometric features faithfully, compared with the conventional techniques.

• Publications