Abstract
Incremental few-shot semantic segmentation (IFSS) expands segmentation capacity of the trained model to segment new class images with few samples. However, semantic meanings may shift from background to object class or vice versa dur ing incremental learning. Moreover, new-class samples of ten lack representative attribute features when the new class greatly differs from the pre-learned old class. In this paper, we propose a causal framework to discuss the cause of semantic shift and incompleteness in IFSS, and we deconfound the revealed causal effects from two aspects. First, we propose a Causal Intervention Module (CIM) to resist semantic shift. CIM progressively and adaptively updates prototypes of old class, and removes the confounder in an intervention manner. Second, a Prototype Refinement Module (PRM) is proposed to complete the missing semantics. In PRM, knowledge gained from the episode learning scheme assists in fusing fea tures of new-class and old-class prototypes. Experiments on both PASCAL-VOC 2012 and ADE20k benchmarks demon strate the outstanding performance of our method.
Yirui Wu
Young Professor, CCF Senior Member
My research interests include Computer Vision, Artifical Intelligence, Multimedia Computing and Intelligent Water Conservancy.
Yuhang Xia
M.E. student
