[Fix] Fix nan loss in RotatedIoULoss when using AmpOptimizer

[jamiechoi1995]

Fix #888

After this fix, the loss is normal

06/26 10:28:07 - mmengine - INFO - Epoch(train)   [1][  50/1000]  lr: 6.1323e-06  eta: 1 day, 2:56:08  time: 0.9702  data_time: 0.1629  memory: 30334  loss: 2.0993  loss_cls: 0.8395  loss_bbox: 1.2598
06/26 10:28:45 - mmengine - INFO - Epoch(train)   [1][ 100/1000]  lr: 1.2389e-05  eta: 1 day, 0:03:42  time: 0.7640  data_time: 0.0112  memory: 29992  loss: 1.9670  loss_cls: 0.7536  loss_bbox: 1.2134
06/26 10:29:24 - mmengine - INFO - Epoch(train)   [1][ 150/1000]  lr: 1.8645e-05  eta: 23:06:32  time: 0.7654  data_time: 0.0124  memory: 30087  loss: 1.9570  loss_cls: 0.7950  loss_bbox: 1.1620
06/26 10:30:05 - mmengine - INFO - Epoch(train)   [1][ 200/1000]  lr: 2.4901e-05  eta: 23:07:00  time: 0.8360  data_time: 0.0772  memory: 30005  loss: 1.8406  loss_cls: 0.7878  loss_bbox: 1.0528
06/26 10:30:50 - mmengine - INFO - Epoch(train)   [1][ 250/1000]  lr: 3.1157e-05  eta: 23:25:05  time: 0.8904  data_time: 0.0912  memory: 30039  loss: 1.8048  loss_cls: 0.8258  loss_bbox: 0.9790
06/26 10:31:28 - mmengine - INFO - Epoch(train)   [1][ 300/1000]  lr: 3.7413e-05  eta: 23:00:47  time: 0.7599  data_time: 0.0084  memory: 30106  loss: 1.7235  loss_cls: 0.7824  loss_bbox: 0.9411
06/26 10:32:07 - mmengine - INFO - Epoch(train)   [1][ 350/1000]  lr: 4.3669e-05  eta: 22:46:50  time: 0.7751  data_time: 0.0066  memory: 30022  loss: 1.6806  loss_cls: 0.7826  loss_bbox: 0.8980

[liuyanyi]

Thanks for pr, is there any difference in mAP and training speed after using fp16?

[jamiechoi1995]

Thanks for pr, is there any difference in mAP and training speed after using fp16?

Hi Yanyi, the mAP and training speed are nearly the same, but enabling Amp training can set a larger batch size which can reduce the training time.