TypeError: forward() got an unexpected keyword argument 'cache_position'

[sanderland]

🐛 Describe the bug

When using generate, the above error is raised. In the source I can indeed see that forward() does not take such an argument or ignore unexpected extra args.

Versions

Python 3.9.2
ai2-olmo==0.2.5
transformers==4.39.1
torch==2.2.1
(etc)

[Ronho]

Hi! I experienced this issue as well. As I am not familiar with anything of this, please take my words with a grain of salt.

Debugging this showed that cache_position is passed to prepare_inputs_for_generation as part of the kwargs. The cache_position has been added to the model_kwargs in this commit.

Adding cache_position to the forward method of OLMoForCausalLM does the trick but I am not sure how it is intended to work/be used.

[fkcoder]

transformers==4.38.2 works for me.

[Ronho]

Yes, it should - the mentioned commit was added in version 4.39.0.

[2015aroras]

@Ronho is spot on with the cause and version. I didn't see this issue but I fixed this in #534