Detection & Segmentation Reproduction (Low Baselines)

[MikeApple79]

@xiaoaoran, as been pointed out in this issue, your method is comparing baselines that are much lower than the current progress. This leads to questionable evaluations regarding your reporting:

• In Table 4 of the paper, the reported NDS scores for PointPillar, SECOND and CenterNet are 54.9, 58.6 and 59.6; however, the reported results in the OpenPCDet repository are actually 58.23, 62.29 and 66.48.
• In Table 1 of the paper, your baselines are MinkNet (55.9% mIoU) and SPVCNN (58.0% mIoU); however, the pure Cylinder3D model achieves 65.9% mIoU on the SemanticKITTI validation set. The newest works 2DPASS and PVKD achieve even higher scores.
• Your comparisons on the low-score baselines are not convincing at all. The real effectiveness of your method should be validated with proper baseline implementations, not on the "easy" scores that are deliberately lowed for demonstration purposes.

[xiaoaoran]

For object detection, we only used 2 GPUs for training due to resource constrain, but the comparison between 'baseline' and 'baseline+polarmix' is fair.

I am glad to remind that SPVCNN with polarmix achieves 66.2 mIoU, slightly surpassing the SOTA Cylinder3D at 65.9% mIoU, while with a much smaller training time and fewer computation resources (we only used one 2080Ti).

The 2DPASS is released after our submission. It also used 2D images for assistance while we focus on the single modal. PVKD is another excellent paper with knowledge distillation while our design and implementation are totally different.

Besides, please note this is a paper for data augmentation. It is common to test effectiveness in classical models [1,2,3], let alone SPVCNN with polarmix achieves SOTA performance. We test different baselines and compared polarmix with other data augmentation methods including the SOTA augmentation method Mix3D in point cloud segmentation.

Again, we would clarify that all our evaluations for data augmentation are fair.

[1] PointMixup: Augmentation for Point Clouds. ECCV2020.
[2] Point Cloud Augmentation with Weighted Local Transformations. ICCV2021.
[3] PatchAugment: Local Neighborhood Augmentation in Point Cloud Classification. ICCV2021.

Thanks for your comments.