add capability to output intermediate hidden states#451
Merged
sarahwie merged 12 commits intoallenai:mainfrom Feb 16, 2024
Merged
add capability to output intermediate hidden states#451sarahwie merged 12 commits intoallenai:mainfrom
sarahwie merged 12 commits intoallenai:mainfrom
Conversation
Contributor
|
Have you tested this @sarahwie ? Looks close to what I would expect but I don't use the OLMo repo much :) |
Contributor
|
Hi @sarahwie this is great! Do you mind also adding the use of the two flags to the HF wrapper here: https://github.com/allenai/OLMo/blob/main/hf_olmo/modeling_olmo.py#L48 |
AkshitaB
approved these changes
Feb 14, 2024
Contributor
Author
|
I've tested this locally but haven't re-built the pip package, let me try that to make sure it still works as expected @natolambert |
Co-authored-by: 玄钛 <hanxudong.hxd@antgroup.com> Co-authored-by: Pete <epwalsh10@gmail.com> Co-authored-by: epwalsh <petew@allenai.org>
Contributor
Author
|
Temporarily added code to throw an error if Tested this, and everything is worked as expected. Should I merge? |
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a functionality that HF model forward calls have.
A very similar update can be done to add the
output_attentionsfunctionality, though I haven't written it yet.Note that HF code "appends" hidden states to a tuple object, but I'm using a list as I think it's better practice. Hence the type casting in the function output to return a tuple, in order to match expected output in HF's convention.