The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences
Download
Publications Copernicus
Download
Citation
Articles | Volume XLIII-B2-2022
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2022, 329–334, 2022
https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-329-2022
Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B2-2022, 329–334, 2022
https://doi.org/10.5194/isprs-archives-XLIII-B2-2022-329-2022
 
30 May 2022
30 May 2022

A TWO-STAGE APPROACH FOR RARE CLASS SEGMENTATION IN LARGE-SCALE URBAN POINT CLOUDS

X. Zhang1, R. Xue1,2, and U. Soergel1 X. Zhang et al.
  • 1Institute for Photogrammetry, University of Stuttgart, 70174 Stuttgart, Germany
  • 2National Lab of Radar Signal Processing, Xidian University, 710071 Xi’an, China

Keywords: Deep Learning, Transformer, Two-Stage Approach, Rare Classes, Imbalanced Classes, Semantic Segmentation, Urban Point Clouds

Abstract. Although deep learning has greatly improved the semantic segmentation accuracy of point clouds, the segmentation of rare classes in large-scale urban scenes has not been targeted in available methods. This paper proposes a two-stage segmentation framework with automated workflows for imbalanced rare classes based on general semantic segmentation. The proposed approach includes two stages: general semantic segmentation and object-based refined semantic segmentation. Firstly, general segmentation networks are utilized to segment general large objects. Secondly, refined semantic segmentation is conducted by an automated workflow: 3D clustering and bounding box (BBox) generation are applied to the point cloud of rare fine-grained objects during the training, followed by object detection to extract fine-grained objects. Afterwards, as the constraints, the extracted BBoxes further refine the segmentation results. Our approach is evaluated on the Hessigheim High-Resolution 3D Point Cloud (H3D) Benchmark and obtains state-of-the-art 89.35% overall accuracy and outstanding 75.70% mean F1-Score. Furthermore, rare classes Vehicle and Chimney achieve breakthroughs from zero to 63.63% and 52.00% in F1-score, respectively.