Abnormal Reconstruction Results for 6D Rotation in VQ-VAE

[einnullnull]

Hello, I have been trying to use your open-source VQ-VAE for trajectory reconstruction. While the reconstruction results for position and gripper width are satisfactory, I encountered significant issues with the 6D rotation reconstruction.

Here are the details of my experiments:

1. Using Your Example Data and Normalizer:

   • I normalized the action chunk using your provided example data and the normalizer.pt file.
   • The encoder part works as expected, and the generated action tokens match the stored results.
   • However, the reconstructed 6D rotation values are abnormal, ranging from hundreds to thousands, which is far from the expected range.

2. Using My Own UMI Dataset:

   • I also tested with my own UMI dataset, but the 6D rotation reconstruction still shows the same abnormal results.

Additionally, I noticed that the normalizer.pt file you provided does not scale the 6D rotation values.

Could you please help me identify where the problem might be? Is there something I missed in the normalization process, or is there a known issue with the 6D rotation reconstruction in the current implementation?