backward simulation not working? #81
Description
When we try, the backward simulation doesn't work.
If we turn it off, we can get a working model.
But whenever we train with backward simulation the training doesn't converge.
I saw in CauseVid the wandb their author pasted: https://wandb.ai/yin2/causvid/runs/8axhn7wn?nw=nwuseryin2, the backward simulation is also disabled. Is it really working?
Of maybe we should only start to enable backward simulation after some number of steps?
Anybody actually got back simulation working?