Discriminative Co-Saliency and Background Mining Transformer for
Co-Salient Object Detection
Discriminative Co-Saliency and Background Mining Transformer for Co-Salient Object Detection
Long Li1 Junwei Han1 Ni Zhang1 Nian Liu2† Salman Khan2,3 Hisham Cholakkal2 Rao Muhammad Anwer2 Fahad Shahbaz Khan2,4

Abstract
Most previous co-salient object detection works mainly focus on extractingco-salient cues via mining the consistency relations across images whileignoring explicit exploration of background regions. In this paper, we proposea Discriminative co-saliency and background Mining Transformer framework (DMT)based on several economical multi-grained correlation modules to explicitlymine both co-saliency and background information and effectively model theirdiscrimination. Specifically, we first propose a region-to-region correlationmodule for introducing inter-image relations to pixel-wise segmentationfeatures while maintaining computational efficiency. Then, we use two types ofpre-defined tokens to mine co-saliency and background information via ourproposed contrast-induced pixel-to-token correlation and co-saliencytoken-to-token correlation modules. We also design a token-guided featurerefinement module to enhance the discriminability of the segmentation featuresunder the guidance of the learned tokens. We perform iterative mutual promotionfor the segmentation feature extraction and token construction. Experimentalresults on three benchmark datasets demonstrate the effectiveness of ourproposed method. The source code is available at:https://github.com/dragonlee258079/DMT.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| co-salient-object-detection-on-coca | DMT | MAE: 0.108 Mean F-measure: 0.590 S-measure: 0.725 max E-measure: 0.800 max F-measure: 0.619 mean E-measure: 0.753 |
| co-salient-object-detection-on-cosal2015 | DMT | MAE: 0.045 S-measure: 0.897 max E-measure: 0.936 max F-measure: 0.905 mean E-measure: 0.922 mean F-measure: 0.883 |
| co-salient-object-detection-on-cosod3k | DMT | MAE: 0.063 S-measure: 0.851 max E-measure: 0.895 max F-measure: 0.835 mean E-measure: 0.881 mean F-measure: 0.815 |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.