"Connection error, and we cannot find the requested files in the cached path." ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

[zakidotai]

I am trying to execute this command after installing all the required modules and I ran into this error:
NOTE : We are running this on HPC cluster.

python -c "from transformers import pipeline; print(pipeline('sentiment-analysis')('I love you'))"
Traceback (most recent call last):
File "", line 1, in
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/pipelines/init.py", line 340, in pipeline
framework = framework or get_framework(model)
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/pipelines/base.py", line 66, in get_framework
model = AutoModel.from_pretrained(model, revision=revision)
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/models/auto/modeling_auto.py", line 724, in from_pretrained
pretrained_model_name_or_path, return_unused_kwargs=True, **kwargs
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/models/auto/configuration_auto.py", line 360, in from_pretrained
config_dict, _ = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/configuration_utils.py", line 420, in get_config_dict
use_auth_token=use_auth_token,
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/file_utils.py", line 1056, in cached_path
local_files_only=local_files_only,
File "/home/civil/phd/cez198233/anaconda3/envs/lang2/lib/python3.7/site-packages/transformers/file_utils.py", line 1235, in get_from_cache
"Connection error, and we cannot find the requested files in the cached path."
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

#CONDA LIST OUTPUT
conda list

packages in environment at /home/civil/phd/cez198233/anaconda3/envs/lang2:

Name Version Build Channel

_libgcc_mutex 0.1 main
blas 1.0 mkl
brotlipy 0.7.0 py37hb5d75c8_1001 conda-forge
ca-certificates 2020.12.5 ha878542_0 conda-forge
certifi 2020.12.5 py37h89c1867_1 conda-forge
cffi 1.14.4 py37h261ae71_0
chardet 4.0.0 py37h89c1867_1 conda-forge
click 7.1.2 pyh9f0ad1d_0 conda-forge
cryptography 2.9.2 py37hb09aad4_0 conda-forge
cudatoolkit 10.0.130 0
dataclasses 0.7 pyhb2cacf7_7 conda-forge
filelock 3.0.12 pyh9f0ad1d_0 conda-forge
freetype 2.10.4 h5ab3b9f_0
gperftools 2.7 h767d802_2 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
importlib-metadata 3.4.0 py37h89c1867_0 conda-forge
intel-openmp 2020.2 254
joblib 1.0.0 pyhd8ed1ab_0 conda-forge
jpeg 9b h024ee3a_2
lcms2 2.11 h396b838_0
ld_impl_linux-64 2.33.1 h53a641e_7
libedit 3.1.20191231 h14c3975_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libpng 1.6.37 hbc83047_0
libstdcxx-ng 9.1.0 hdf63c60_0
libtiff 4.1.0 h2733197_1
lz4-c 1.9.3 h2531618_0
mkl 2020.2 256
mkl-service 2.3.0 py37he8ac12f_0
mkl_fft 1.2.0 py37h23d657b_0
mkl_random 1.1.1 py37h0573a6f_0
ncurses 6.2 he6710b0_1
ninja 1.10.2 py37hff7bd54_0
numpy 1.19.2 py37h54aff64_0
numpy-base 1.19.2 py37hfa32c7d_0
olefile 0.46 py37_0
openssl 1.1.1i h27cfd23_0
packaging 20.9 pyh44b312d_0 conda-forge
perl 5.32.0 h36c2ea0_0 conda-forge
pillow 8.1.0 py37he98fc37_0
pip 20.3.3 py37h06a4308_0
pycparser 2.20 py_2
pyopenssl 19.1.0 py37_0 conda-forge
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pysocks 1.7.1 py37h89c1867_3 conda-forge
python 3.7.9 h7579374_0
python_abi 3.7 1_cp37m conda-forge
pytorch 1.1.0 py3.7_cuda10.0.130_cudnn7.5.1_0 pytorch
readline 8.1 h27cfd23_0
regex 2020.11.13 py37h4abf009_0 conda-forge
requests 2.25.1 pyhd3deb0d_0 conda-forge
sacremoses 0.0.43 pyh9f0ad1d_0 conda-forge
sentencepiece 0.1.92 py37h99015e2_0 conda-forge
setuptools 52.0.0 py37h06a4308_0
six 1.15.0 py37h06a4308_0
sqlite 3.33.0 h62c20be_0
tk 8.6.10 hbc83047_0
tokenizers 0.9.4 py37h17e0dd7_1 conda-forge
torchvision 0.3.0 py37_cu10.0.130_1 pytorch
tqdm 4.56.0 pyhd8ed1ab_0 conda-forge
transformers 4.2.2 pyhd8ed1ab_0 conda-forge
typing_extensions 3.7.4.3 py_0 conda-forge
urllib3 1.26.3 pyhd8ed1ab_0 conda-forge
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h7b6447c_0
zipp 3.4.0 py_0 conda-forge
zlib 1.2.11 h7b6447c_3
zstd 1.4.5 h9ceee32_0

conda info --all output

conda info --all

 active environment : lang2
active env location : /home/civil/phd/cez198233/anaconda3/envs/lang2
        shell level : 1
   user config file : /home/civil/phd/cez198233/.condarc

populated config files : /home/civil/phd/cez198233/.condarc
conda version : 4.8.3
conda-build version : 3.18.11
python version : 3.7.6.final.0
virtual packages : __glibc=2.17
base environment : /home/civil/phd/cez198233/anaconda3 (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /home/civil/phd/cez198233/anaconda3/pkgs
/home/civil/phd/cez198233/.conda/pkgs
envs directories : /home/civil/phd/cez198233/anaconda3/envs
/home/civil/phd/cez198233/.conda/envs
platform : linux-64
user-agent : conda/4.8.3 requests/2.22.0 CPython/3.7.6 Linux/3.10.0-957.el7.x86_64 centos/7.6.1810 glibc/2.17
UID:GID : 86941:11302
netrc file : None
offline mode : False

conda environments:

base /home/civil/phd/cez198233/anaconda3
9pytorch /home/civil/phd/cez198233/anaconda3/envs/9pytorch
lang2 * /home/civil/phd/cez198233/anaconda3/envs/lang2
tf-gpu /home/civil/phd/cez198233/anaconda3/envs/tf-gpu

sys.version: 3.7.6 (default, Jan 8 2020, 19:59:22)
...
sys.prefix: /home/civil/phd/cez198233/anaconda3
sys.executable: /home/civil/phd/cez198233/anaconda3/bin/python
conda location: /home/civil/phd/cez198233/anaconda3/lib/python3.7/site-packages/conda
conda-build: /home/civil/phd/cez198233/anaconda3/bin/conda-build
conda-convert: /home/civil/phd/cez198233/anaconda3/bin/conda-convert
conda-debug: /home/civil/phd/cez198233/anaconda3/bin/conda-debug
conda-develop: /home/civil/phd/cez198233/anaconda3/bin/conda-develop
conda-env: /home/civil/phd/cez198233/anaconda3/bin/conda-env
conda-index: /home/civil/phd/cez198233/anaconda3/bin/conda-index
conda-inspect: /home/civil/phd/cez198233/anaconda3/bin/conda-inspect
conda-metapackage: /home/civil/phd/cez198233/anaconda3/bin/conda-metapackage
conda-render: /home/civil/phd/cez198233/anaconda3/bin/conda-render
conda-server: /home/civil/phd/cez198233/anaconda3/bin/conda-server
conda-skeleton: /home/civil/phd/cez198233/anaconda3/bin/conda-skeleton
conda-verify: /home/civil/phd/cez198233/anaconda3/bin/conda-verify
user site dirs: ~/.local/lib/python3.6
~/.local/lib/python3.7

CIO_TEST:
CONDA_DEFAULT_ENV: lang2
CONDA_EXE: /home/civil/phd/cez198233/anaconda3/bin/conda
CONDA_PREFIX: /home/civil/phd/cez198233/anaconda3/envs/lang2
CONDA_PROMPT_MODIFIER: (lang2)
CONDA_PYTHON_EXE: /home/civil/phd/cez198233/anaconda3/bin/python
CONDA_ROOT: /home/civil/phd/cez198233/anaconda3
CONDA_SHLVL: 1
HTTPS_PROXY:
HTTP_PROXY:
MANPATH: /usr/share/Modules/3.2.10/share/man::/opt/pbs/19.2.4/share/man
MODULEPATH: /home/soft/modules
PATH: /home/civil/phd/cez198233/anaconda3/bin:/home/civil/phd/cez198233/anaconda3/envs/lang2/bin:/home/civil/phd/cez198233/anaconda3/bin:/home/civil/phd/cez198233/anaconda3/bin:/home/civil/phd/cez198233/anaconda3/condabin:/opt/am/bin:/opt/am/sbin:/opt/pbs/default/bin:/opt/pbs/default/sbin:/usr/share/Modules/3.2.10/bin:/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/opt/ibutils/bin:/root/bin:/opt/pbs/19.2.4/bin:/home/civil/phd/cez198233/bin
REQUESTS_CA_BUNDLE:
SSL_CERT_FILE:
ftp_proxy:
http_proxy:
https_proxy:

[LysandreJik]

Hello! Can you try installing the brand new v4.3.0 to see if it resolves your issue?

[yzy1121]

same problem here and my transformers is v4.3.0, still not working

[LysandreJik]

Could you try to load the model/tokenizer and specify the local_files_only=True kwarg to the from_pretrained method, before passing them to the pipeline directly?

i.e., instead of:

pipeline('sentiment-analysis')('I love you')

try:

from transformers import pipeline, AutoModelForSequenceClassification, AutoTokenizer

model = AutoModelForSequenceClassification.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english", local_files_only=True)
tokenizer = AutoTokenizer.from_pretrained("distilbert-base-uncased-finetuned-sst-2-english", local_files_only=True)

pipeline('sentiment-analysis', model=model, tokenizer=tokenizer)('I love you')

[zakidotai]

Thanks @LysandreJik.

Tried your suggestions.
Not working. Tried by keeping local_files_only both True and False while loading the model. It did not work.

[iron-hope-shop]

@PrinceMohdZaki were you able to find a solution for this error?

[julien-c]

Can anyone having those kind of issues, please try #10235, and let us know if it provides more insight into the cause (networking issue, proxy error, etc.)?

Thanks!

[zakidotai]

@PrinceMohdZaki were you able to find a solution for this error?

Yeah. Finally we resolved it by exporting the https_proxy same as http_proxy as shown here : https://stackoverflow.com/questions/56628194/sslerror-installing-with-pip/56628419

[himanshisyadav]

I am facing the same issue when trying to do spacy.load. Is there an obvious solution to that?

I am following this tutorial to build a custom NER pipeline on an HPC cluster where the compute nodes do not have access to the internet.

Here's the log:

python3 -m spacy train data/config.cfg --paths.train ./train.spacy --paths.dev ./valid.spacy --output ./models/output --gpu-id 0
ℹ Saving to output directory: models/output
ℹ Using GPU: 0

=========================== Initializing pipeline ===========================
[2022-11-18 15:12:08,973] [INFO] Set up nlp object from config
[2022-11-18 15:12:08,982] [INFO] Pipeline: ['transformer', 'ner']
[2022-11-18 15:12:08,984] [INFO] Created vocabulary
[2022-11-18 15:12:08,986] [INFO] Finished initializing nlp object
Traceback (most recent call last):
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy/__main__.py", line 4, in <module>
    setup_cli()
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy/cli/_util.py", line 71, in setup_cli
    command(prog_name=COMMAND)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/typer/main.py", line 500, in wrapper
    return callback(**use_params)  # type: ignore
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy/cli/train.py", line 45, in train_cli
    train(config_path, output_path, use_gpu=use_gpu, overrides=overrides)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy/cli/train.py", line 72, in train
    nlp = init_nlp(config, use_gpu=use_gpu)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy/training/initialize.py", line 84, in init_nlp
    nlp.initialize(lambda: train_corpus(nlp), sgd=optimizer)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy/language.py", line 1317, in initialize
    proc.initialize(get_examples, nlp=self, **p_settings)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy_transformers/pipeline_component.py", line 355, in initialize
    self.model.initialize(X=docs)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/thinc/model.py", line 299, in initialize
    self.init(self, X=X, Y=Y)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy_transformers/layers/transformer_model.py", line 131, in init
    hf_model = huggingface_from_pretrained(name, tok_cfg, trf_cfg)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/spacy_transformers/layers/transformer_model.py", line 251, in huggingface_from_pretrained
    tokenizer = AutoTokenizer.from_pretrained(str_path, **tok_config)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 471, in from_pretrained
    tokenizer_config = get_tokenizer_config(pretrained_model_name_or_path, **kwargs)
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py", line 332, in get_tokenizer_config
    resolved_config_file = get_file_from_repo(
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/transformers/file_utils.py", line 2310, in get_file_from_repo
    resolved_file = cached_path(
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/transformers/file_utils.py", line 1921, in cached_path
    output_path = get_from_cache(
  File "/home/hyadav/.conda/envs/spacy_venv/lib/python3.10/site-packages/transformers/file_utils.py", line 2177, in get_from_cache
    raise ValueError(
ValueError: Connection error, and we cannot find the requested files in the cached path. Please try again or make sure your Internet connection is on.

[zakidotai]

First you need to run internet on the hpc. Read your hpc's documentation or ask system administrator about it.

[himanshisyadav]

First you need to run internet on the hpc. Read your hpc's documentation or ask system administrator about it.

We have access to the internet on the login nodes but not on the compute nodes. So, I can download everything on the login nodes I need before I finally start computations on the compute nodes.

[zakidotai]

Follow the same protocol to run internet on the compute nodes. If you are running the script by submitting a job on compute nodes, insert the commands to run internet in the job script before the python command. If proxies are also required to be exported, you can either export them before the python command or export proxies within the code using os module.

…

On Sat, 19 Nov 2022, 00:27 Himanshi Yadav, ***@***.***> wrote: First you need to run internet on the hpc. Read your hpc's documentation or ask system administrator about it. We have access to the internet on the login nodes but not on the compute nodes. So, I can download everything on the login nodes I need before I finally start computations on the compute nodes. — Reply to this email directly, view it on GitHub <#10067 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFATXOFCQS2LFR2I2KNVXN3WI7YFFANCNFSM4XI2ROIA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[himanshisyadav]

Follow the same protocol to run internet on the compute nodes. If you are running the script by submitting a job on compute nodes, insert the commands to run internet in the job script before the python command. If proxies are also required to be exported, you can either export them before the python command or export proxies within the code using os module.
…
On Sat, 19 Nov 2022, 00:27 Himanshi Yadav, @.> wrote: First you need to run internet on the hpc. Read your hpc's documentation or ask system administrator about it. We have access to the internet on the login nodes but not on the compute nodes. So, I can download everything on the login nodes I need before I finally start computations on the compute nodes. — Reply to this email directly, view it on GitHub <#10067 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFATXOFCQS2LFR2I2KNVXN3WI7YFFANCNFSM4XI2ROIA . You are receiving this because you modified the open/close state.Message ID: @.>

There is no protocol to run the internet on the compute nodes, you can not use internet on the compute nodes.