Skip to content

Add 'gguf' to project dependencies#12046

Merged
hnyls2002 merged 2 commits intosgl-project:mainfrom
Muqi1029:patch-3
Oct 24, 2025
Merged

Add 'gguf' to project dependencies#12046
hnyls2002 merged 2 commits intosgl-project:mainfrom
Muqi1029:patch-3

Conversation

@Muqi1029
Copy link
Contributor

@Muqi1029 Muqi1029 commented Oct 24, 2025

Motivation

After installing sglang:v0.5.4

/root/.envs/dev/lib/python3.12/site-packages/torch/cuda/__init__.py:63: FutureWarning: The pynvml package is 
deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this t
o the maintainers of the package that installed pynvml for you.
  import pynvml  # type: ignore[import]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/projects/dev_project/sglang/python/sglang/launch_server.py", line 11, in <module>
    server_args = prepare_server_args(sys.argv[1:])
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/projects/dev_project/sglang/python/sglang/srt/server_args.py", line 3689, in prepare_server_arg
s
    return ServerArgs.from_cli_args(raw_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/projects/dev_project/sglang/python/sglang/srt/server_args.py", line 3339, in from_cli_args
    return cls(**{attr: getattr(args, attr) for attr in attrs})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<string>", line 268, in __init__
  File "/root/projects/dev_project/sglang/python/sglang/srt/server_args.py", line 544, in __post_init__
    self._handle_gpu_memory_settings(gpu_mem)
  File "/root/projects/dev_project/sglang/python/sglang/srt/server_args.py", line 805, in _handle_gpu_memory_
settings
    from sglang.srt.configs.model_config import ModelConfig
  File "/root/projects/dev_project/sglang/python/sglang/srt/configs/model_config.py", line 26, in <module>
    from sglang.srt.layers.quantization import QUANTIZATION_METHODS
  File "/root/projects/dev_project/sglang/python/sglang/srt/layers/quantization/__init__.py", line 45, in <mo
dule>
    from sglang.srt.layers.quantization.gguf import GGUFConfig
  File "/root/projects/dev_project/sglang/python/sglang/srt/layers/quantization/gguf.py", line 9, in <module>
    import gguf
ModuleNotFoundError: No module named 'gguf'

Modifications

Accuracy Tests

Benchmarking and Profiling

Checklist

@gemini-code-assist
Copy link
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@hnyls2002 hnyls2002 merged commit b04cd3d into sgl-project:main Oct 24, 2025
38 of 41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments