`AutoModel.from_pretrained(local_files_only=True)` changes the hash inside the `refs/main`

[kouroshHakha]

System Info

transformers 4.38.1

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

1. Download the model locally but put it under a hash that is not the same as the one from the latest in the hub
2. Run this. Interestingly if you instantiate the tokenizer first and the model it works fine. The culprit is that the is_peft_available() branch of the logic will look for adapter_config.json file and it won't find it, so it will ask hub and in that process it overrides the refs/main file.

from transformers import AutoModel, AutoTokenizer
model = AutoModel.from_pretrained(
    model_id, 
    local_files_only=True,
)

tokenizer = AutoTokenizer.from_pretrained(
    model_id, 
    local_files_only=True,
)

Expected behavior

The code above to work, more importantly AutoModel.from_pretrained() to not change the content of refs/main file.

[ArthurZucker]

cc @younesbelkada and @Rocketknight1

[Rocketknight1]

I totally lost track of this one - will take it now!

[Rocketknight1]

Hi @kouroshHakha, just to speed things up a bit, can you give me a specific model ID and commit hash to use in the reproduction script that triggers this bug?

[kouroshHakha]

I don't recall exactly. But if you take mistralai/Mistral-7B-Instruct-v0.1 for example and then add the files under a different snapshot commit (both in the directory name and in the main file) then you can repro the issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[LysandreJik]

Maybe also of interest to @Wauplin

[Rocketknight1]

I'm still planning to investigate this when I get a chance too!

[Wauplin]

I feel that this is a misunderstanding of how the cache-system works. When downloading from main, it's normal that the refs/main file in the cache gets updated with the latest known commit for that ref. I'm not sure to understand what's the issue here.

[Wauplin]

then add the files under a different snapshot commit

Just for context, files in the cache dir (e.g. in .../snapshots/ should not be modified manually. It is supposed to be a read-only folder otherwise the cache system is corrupted.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.