Cannot get it to work on M2 Mac

[ruze00]

Can anybody help me get txtai working on my M2 Mac?

I created a brand new conda environment, then pip install txtai[all]. Then I did brew install libomp and export OMP_NUM_THREADS=1 but no joy.

It crashes in the 01_Introducing_txtai.ipynb notebook at the line:

embeddings = Embeddings(path="sentence-transformers/nli-mpnet-base-v2")

There is no console output which might help narrow down the issue.

Does anybody have any more hints to get it working? This is/should be a clean environment and it doesn't work out the box.

[ruze00]

Doing a bit of debugging, it fails in transformers.modeling_utils._load_pretrained_model() at line 3603:

@classmethod
def _load_pretrained_model(
    cls,
    model,
    state_dict,
    loaded_keys,
    resolved_archive_file,
    pretrained_model_name_or_path,
    ignore_mismatched_sizes=False,
    sharded_metadata=None,
    _fast_init=True,
    low_cpu_mem_usage=False,
    device_map=None,
    offload_folder=None,
    offload_state_dict=None,
    dtype=None,
    hf_quantizer=None,
    keep_in_fp32_modules=None,
):
    is_safetensors = False

at the first line is_safetensors = False.

cls is transformers.models.mpnet.modeling_mpnet.MPNetModel.

libraries transformers==4.38.2, torch==2.2.1

Let me know what other info I can collect.

[ruze00]

Ok, setting KMP_DUPLICATE_LIB_OK=True works as a workaround. It seems to be a Pytorch issue not txtai.

Although, apparently the potential exists for incorrect results.

[davidmezzetti]

Thank you for opening and documenting how you fixed this! It will hopefully help someone, someday down the line.

[wjbmattingly]

@davidmezzetti you were right! This just saved me a lot of headache!! I'm trying to test my class on several machines and it broke on a Mac and could not debug this issue in a Notebook. Thank you soooo much!