Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| prune_heads (`dict[int, list[int]]`, *optional*, defaults to `{}`): | ||
| Pruned heads of the model. The keys are the selected layer indices and the associated values, the list of | ||
| heads to prune in said layer. | ||
|
|
||
| For instance `{1: [0, 2], 2: [2, 3]}` will prune heads 0 and 2 on layer 1 and heads 2 and 3 on layer 2. |
There was a problem hiding this comment.
this won't break the loading if a saved config has this attribute?
There was a problem hiding this comment.
ok, we have kwargs so it won't break.
But this makes the things not transparent somehow, i.e. the saved config have it but not used at all during loading.
There was a problem hiding this comment.
We can load arbitrary serialized kwargs, so it shouldn't :)
|
[For maintainers] Suggested jobs to run (before merge) run-slow: albert, align, altclip, audio_spectrogram_transformer, beit, bert, bert_generation, blip, blip_2, bridgetower, bros, camembert, canine, chinese_clip, clap, clvp |
Cyrilvallez
left a comment
Cyrilvallez
left a comment
There was a problem hiding this comment.
Very very nice! Good riddance! 🧹🤗
* remove _prune_heads * remove prune_heads * finalize the purge * remove another patterns
4 participants
What does this PR do?
Spiritual successor to #41076 -- removes
prune_headsand related code/tests/docs.prune_headswas a feature that we stopped including in models a while ago. For instance, thetest_head_pruning_save_load_from_config_inittest passes in 28 model classes, but is skipped in 444.