🎉 Add Multitask Prompt Tuning#400
Conversation
|
Check out this pull request on See visual diffs & provide feedback on Jupyter Notebooks. Powered by ReviewNB |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. |
|
@pacman100 |
|
Also, a working notebook has been added @pacman100 🤗 |
Fix dict index in embedding retrieval
pacman100
left a comment
pacman100
left a comment
There was a problem hiding this comment.
Thank you so much @mayank31398 for adding MultiTask Prompt Tuning 🔥! Could you please resolve the conflicts and run make style and make quality to resolve the quality issues. Post that we can go ahead and merge this PR 🤗.
|
I just noticed that there isn't any addition to the docs for MPT. Could you please add some documentation there? Or are we fine with adding it in a separate PR? |
pacman100
left a comment
pacman100
left a comment
There was a problem hiding this comment.
Hello @mayank31398, Thank you for iterating! I believe the PR can be merged once the conflicts are resolved as there is an example showing usage and related tests. Thank you for all the work and for your kind patience 🤗
|
Hey @pacman100 let me add some comments in the code. |
|
@pacman100 I have addressed the suggestions. |
|
@pacman100 @younesbelkada lets merge this? |
|
Hey, guys still waiting for an update :) |
younesbelkada
left a comment
younesbelkada
left a comment
There was a problem hiding this comment.
Thanks a lot for your patience @mayank31398 and sorry for the delay , I left two tiny comments, apart from that I trust other maintainers review. Can you please revert the changes in the unrelated files?
Thanks!
|
@younesbelkada addressed the comments |
|
@mayank31398 Just to let you know: We'll probably soon merge #807 which adds a new folder structure to the tuners. If you want to, you can already adjust your code to use the new structure. Otherwise, it's also fine as is, then we'll do the refactoring later. |
I prefer leaving as is for now. |
pacman100
left a comment
pacman100
left a comment
There was a problem hiding this comment.
Thank you @mayank31398 for iterating multiple times and for your patience, LGTM! 🔥🚀✨
|
@mayank31398 Why can't I locate the distillation process of the source prompt in the code? Am I missing something? 😭 |
|
@junzhang-zj yeah, it doesn't have that. |
|
@mayank31398 Thank you for your answer, which addressed my concern about carelessness! |
* mpt * fix save * fix save * add jupyter notebook * add jupyter notebook * add jupyter notebook * drop shuffling * drop classify_dataset * drop classify_dataset * fix keys * fix keys * add comments * use EXACT_SOURCE_TASK in the example * formatting * Fix dict index in embedding retrieval * run style and quality * run style and quality * run style and quality * style * final fix * style * comment out failing tests * fix generation tests * fix style and save test * all testcases * fix import * add license header * reformat * fix encoder-decoder models * fix tests running multiple times * fix paper name for IA3 and add MPT paper * Trigger CI * address the recommended changes * reformat * address suggestions * address suggestions * revert reformatting * revert reformatting --------- Co-authored-by: Alex-Brooks <Alex.Brooks@ibm.com>
* fixed rl training args added steps argument and break to respect max training epochs added more PPOConfig args to script args removed llama tokenizer hacks removed extra args in dataset changed to llamatokenizer from autotokenizer black + isort * black and flake8 * style, quality, and switch back to AutoTokenizer
7 participants
Adds code for the paper: https://arxiv.org/abs/2303.02861