exclude_from_weight_decay added

[sadimanna]

No description provided.

[sadimanna]

This modified LARS code was tested on CIFAR-10 with ResNet-50 backbone. Linear Evaluation Test Accuracy is ~85% after 100 epochs.

[Ektagavas]

This modified LARS code was tested on CIFAR-10 with ResNet-50 backbone. Linear Evaluation Test Accuracy is ~85% after 100 epochs.

Hey, I am not able to achieve the linear evaluation accuracy on CIFAR10 with Resnet-50 reported in paper with this repo code. After modifying the LARS code, you are getting around 85%, so may I please know what hyper-parameters (like batchsize, LR, temperature) did you use? Also, 100 epochs is for Contrastive training or linear evaluation?

Thank you.

[sadimanna]

@Ektagavas Hi, I used a batch size of 64, LR 0.2, and temperature 0.5. 100 epochs for contrastive learning. Also, I used a pre-trained encoder. The results may differ if you train an encoder from scratch.

[Ektagavas]

@Ektagavas Hi, I used a batch size of 64, LR 0.2, and temperature 0.5. 100 epochs for contrastive learning. Also, I used a pre-trained encoder. The results may differ if you train an encoder from scratch.

@sadimanna Thanks for your quick response and for sharing the details! Can you pls also share from where you got your pre-trained encoder?

Thank you.

[sadimanna]

@Ektagavas I used torchvision.models.resnet50(pretrained = True)

[Ektagavas]

@Ektagavas I used torchvision.models.resnet50(pretrained = True)

Oh I see, the PyTorch's pre-trained model. Cool, thanks! :)