【ICCV 2021】少样本语义分割的元类别记忆学习 （MM-Net）
-
转载： latte拿铁（知乎）
本文提出了一种新颖的基于元类记忆的少样本分割方法（MM-Net），其中引入了一组可学习的记忆嵌入，以在基类训练期间记忆元类别信息，并在推理期间转移到新类别。
Learning Meta-class Memory for Few-Shot Semantic Segmentation
Zhonghua Wu, Xiangxi Shi, Guosheng lin, Jianfei Cai
Currently, the state-of-the-art methods treat few-shot semantic segmentation task as a conditional foreground-background segmentation problem, assuming each class is independent. In this paper, we introduce the concept of meta-class, which is the meta information (e.g. certain middle-level features) shareable among all classes. To explicitly learn meta-class representations in few-shot segmentation task, we propose a novel Meta-class Memory based few-shot segmentation method (MM-Net), where we introduce a set of learnable memory embeddings to memorize the meta-class information during the base class training and transfer to novel classes during the inference stage. Moreover, for the kk-shot scenario, we propose a novel image quality measurement module to select images from the set of support images. A high-quality class prototype could be obtained with the weighted sum of support image features based on the quality measure. Experiments on both PASCAL-5i5i and COCO dataset shows that our proposed method is able to achieve state-of-the-art results in both 1-shot and 5-shot settings. Particularly, our proposed MM-Net achieves 37.5% mIoU on the COCO dataset in 1-shot setting, which is 5.1% higher than the previous state-of-the-art.
目前，最先进的方法将少样本语义分割任务视为假设每个类都是独立的条件前景-背景分割问题。在本文中，我们介绍了元类别的概念，它是所有类之间可共享的元信息（例如某些中间层特征）。为了在少镜头分割任务中明确学习元类别表示，我们提出了一种新颖的基于元类记忆的少镜头分割方法（MM-Net），其中我们在基类训练期间引入了一组可学习的内存嵌入来记忆元类别信息，并在推理阶段转移到新类别。此外，对于 k-shot 场景，我们提出了一种新颖的图像质量测量模块来从支持图像集中选择图像。基于质量度量的支持图像特征的加权和可以得到一个高质量的类别原型。在 PASCAL-5i 和 COCO 数据集上的实验表明，我们提出的方法能够在 1-shot 和 5-shot 设置中实现最先进的结果。特别是，我们提出的 MM-Net 在 1-shot 设置中在 COCO 数据集上实现了 37.5% mIoU，比之前的最新技术高 5.1%。
Comments: ICCV 2021 Subjects: Computer Vision and Pattern Recognition (cs.CV) Cite as: arXiv:2108.02958 [cs.CV] 
