Remove hard-coded pad token id in distilbert and albert#3965
Merged
julien-c merged 1 commit intohuggingface:masterfrom May 12, 2020
monologg:get_pad_token_id_from_config
Merged
Remove hard-coded pad token id in distilbert and albert#3965julien-c merged 1 commit intohuggingface:masterfrom monologg:get_pad_token_id_from_config
julien-c merged 1 commit intohuggingface:masterfrom
monologg:get_pad_token_id_from_config
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3965 +/- ##
==========================================
- Coverage 78.45% 78.44% -0.02%
==========================================
Files 111 111
Lines 18518 18518
==========================================
- Hits 14528 14526 -2
- Misses 3990 3992 +2
Continue to review full report at Codecov.
|
Member
|
LGTM |
Contributor
Author
|
Hi:) Can you please check this PR? This one makes issue on Korean BERT. (which use I hope this PR will be applied on the next version of transformers library:) |
VictorSanh
approved these changes
May 7, 2020
Contributor
|
lgtm! |
Contributor
Author
|
Can you merge this PR? Thank you so much:) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
As the config adds
pad_token_idattribute,padding_idxset the value ofconfig.pad_token_idin BertEmbedding. ( PR #3793 )But it seems that not only the config of
Bert, but also that ofDistilBertandAlberthaspad_token_id. (Distilbert config, Albert config)But in Embedding class of Distilbert and Albert, it seems that
padding_idxis still hard-coded. So I've fixed those parts.