PEFT for Multiple Choice?

[jacob-morrison]

Hi! Is it possible to use PEFT for multiple choice models? I found this issue: #384 that never got any traction, and I'm wondering if it's currently possible to use the PEFT package with multiple choice models? It's not in the task enum, and I'm not sure how much work it would be to add that functionality if it doesn't already work out of the box.

[bohyunshin]

I think below pr might help you.
#263

[nbroad1881]

@pacman100 , how come the classifier weights don't get saved when using AutoModelForMultipleChoice and PeftForSequenceClassification?

https://colab.research.google.com/drive/19t895gxM_RTXlWJNQkGXFAWXAoS7BTA3?usp=sharing

[BenjaminBossan]

Hi Nicholas, we can look into that. From the notebook, it was a bit hard for me to understand what is expected and what is unexpected. Could you please give a quick explanation where the issue is?

[nbroad1881]

@BenjaminBossan ,

When you use AutoModelForSequenceClassification or AutoModelForMultipleChoice, the classifier weights are newly initialized. When you save and reload the peft model, you get different output scores for AutoModelForMultipleChoice but the same scores for AutoModelForSequenceClassification. Those newly initialized classifier weights are the reason.

I was trying to show that the parameters that have requires_grad are different, and that likely seems to be a clue as to why it isn't working.

I thought that if I added classifier to modules_to_save it would work, but it doesn't seem like it.

[BenjaminBossan]

Okay, I did some digging and I think I found the reason for your findings @nbroad1881:

First of all, the modules_to_save thing was a red herring. Yes, some weights related to modules_to_save are not loaded, but those don't matter[0]. The ModulesToSaveWrapper keeps a reference to the original module, which is the one that differs. But when forward is called, it uses different weights (model.classifier.modules_to_save["default"].weight) and those are loaded correctly.

So if this is not the issue, what is? The actual cause for the different outputs is that another module has diverging weights. That module is roberta.pooler. This module is also randomly initialized, as can be seen here:

torch.allclose(
    AutoModelForMultipleChoice.from_pretrained(name).roberta.pooler.dense.weight,
    AutoModelForMultipleChoice.from_pretrained(name).roberta.pooler.dense.weight,
)
# returns False

In fact, we're even warned about it:

Some weights of RobertaForMultipleChoice were not initialized from the model checkpoint at roberta-base and are newly initialized: ['roberta.pooler.dense.bias', 'classifier.bias', 'roberta.pooler.dense.weight', 'classifier.weight']

After adding "pooler" to the modules_to_save, I manage to get the exact same output. If you could confirm that this solves your problem, that would be great.

As an overall lesson, I think we can take away that all the modules that transformers warns us about, need to be added to modules_to_save.

[0] It is still a little bit of an issue I think, because, when the adapters are deactivated, we would expect the same output as from the base model, but that wouldn't be the case here. But that's unrelated.

[nbroad1881]

Ah I forgot about the pooler because regular AutoModelForSequenceClassification doesn't use it. Mystery solved! I have gotten so used to ignoring those warnings and this time it came back to bite me. Thank you very much for investigating

tldr;

If you are using peft with AutoModelForMultipleChoice, make sure to have this in the config

modules_to_save=["classifier", "pooler"]

[BenjaminBossan]

I have gotten so used to ignoring those warnings and this time it came back to bite me

Same. The solution was in plain sight the whole time :) Ideally, we could add an extra warning on the PEFT side but it's not possible (AFAICT) to retrieve that information from the loaded transformers model.