Skip to content

An issue during evaluating MathVista #75

@simplelifetime

Description

@simplelifetime

I don't know how to solve it, seem to be a problem with huggingface evaluate module.
Traceback (most recent call last):
File "/home/aka/anaconda3/envs/lmms/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/akaanaconda3/envs/lmms/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/aka/lmms-eval/lmms_eval/main.py", line 24, in
from lmms_eval import evaluator, utils
File "/home/aka/lmms-eval/lmms_eval/evaluator.py", line 17, in
import lmms_eval.api.metrics
File "/home/aka/lmms-eval/lmms_eval/api/metrics.py", line 146, in
exact_match = evaluate.load("exact_match")
File "/home/aka/anaconda3/envs/lmms/lib/python3.10/site-packages/evaluate/loading.py", line 751, in load
evaluation_cls = import_main_class(evaluation_module.module_path)
File "/home/aka/anaconda3/envs/lmms/lib/python3.10/site-packages/evaluate/loading.py", line 76, in import_main_class
module = importlib.import_module(module_path)
File "/home/aka/anaconda3/envs/lmms/lib/python3.10/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/home/aka/.cache/huggingface/modules/evaluate_modules/metrics/evaluate-metric--exact_match/009c8b5313309ea5b135d526433d5ee76508ba1554cbe88310a30f85bb57ec88/exact_match.py", line 16
}
^
SyntaxError: closing parenthesis '}' does not match opening parenthesis '(' on line 14
E0507 14:41:36.009469 139918226703296 torch/distributed/elastic/multiprocessing/api.py:826] failed (exitcode: 1) local_rank: 0 (pid: 36925) of binary: /home/aka/anaconda3/envs/lmms/bin/python

Metadata

Metadata

Assignees

No one assigned

    Labels

    staleNo activity for an extended period

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions