Support THUDM/GLM-4-0414 (GLM-Z1) Glm4ForCausalLM architecture.

[solrex]

Motivation

THUDM/GLM-4-0414 series model uses a new architecture “Glm4ForCausalLM” which is not supported by SGLang. This PR adds support of "Glm4ForCausalLM".

Modifications

• Updated transformers to version 4.51.3
• Added a new modeling file 'glm4.py'

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[solrex]

This PR can load the GLM-Z1-*-0414 models, but there is a high probability that the model's output will enter a cyclic generation and fail to stop. Don't know why. Any suggestion or code review are welcome.

[merrymercy]

Can you fix the conflicts?

[solrex]

Can you fix the conflicts?

Done.

[zhaochenyang20]

Motivation

THUDM/GLM-4-0414 series model uses a new architecture “Glm4ForCausalLM” which is not supported by SGLang. This PR adds support of "Glm4ForCausalLM".

Modifications

• Updated transformers to version 4.51.3
• Added a new modeling file 'glm4.py'

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

could you provide the benchmark score of GLM4 in your implementation?

[solrex]

Motivation

THUDM/GLM-4-0414 series model uses a new architecture “Glm4ForCausalLM” which is not supported by SGLang. This PR adds support of "Glm4ForCausalLM".

Modifications

• Updated transformers to version 4.51.3
• Added a new modeling file 'glm4.py'

could you provide the benchmark score of GLM4 in your implementation?

@zhaochenyang20 GLM-4-Z1 is a reasoning model. During evaluation, its <think> output is often too lengthy, which affects the test score. I haven’t found a good solution yet—do you have any suggestions?

BTW: THUDM team has tested it according to this ISSUE zai-org/GLM-4#750.

[zhaochenyang20]

let me ask member from zhipu

[blairyeung]

Could someone debug this and merge it? Appreciate it! The GLM 4 0414 is an amazon model.

[Muqi1029]

Hi,

I've been testing GLM-4-9B-0414 model support in SGLang. I found that when launching the server with python -m sglang.launch_server --model-path THUDM/GLM-4-9B-0414 --tp-size 1, it works as expected.

However, increasing --tp-size to 2 causes the following UnicodeDecodeError:

File "/root/muqi/mlsys/sglang/python/sglang/srt/hf_transformers_utils.py", line 224, in get_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/muqi/.base/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1013, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.base/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2025, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
File "/root/.base/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2101, in _from_pretrained
    chat_templates["default"] = chat_template_handle.read()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 43: ordinal not in range(128)

This error occurs because the transformers library, when attempting to read the chat_template.jinja file, does not explicitly set an encoding. In torch.distributed settings, child processes often fail to inherit LANG, LC_ALL, or LC_CTYPE environment variables. Consequently, open() defaults to the ASCII encoding, which cannot decode non-ASCII bytes present in the chat_template.jinja file (like 0xe5 in the GLM-4 tokenizer).

To address this, I've submitted a huggingface/transformers#38553 that explicitly sets encoding="utf-8" when opening such files. This ensures consistent and reliable file reading across different environments. I have manually set it locally, it works successfully.

[solrex]

Hi,

I've been testing GLM-4-9B-0414 model support in SGLang. I found that when launching the server with python -m sglang.launch_server --model-path THUDM/GLM-4-9B-0414 --tp-size 1, it works as expected.

However, increasing --tp-size to 2 causes the following UnicodeDecodeError:

File "/root/muqi/mlsys/sglang/python/sglang/srt/hf_transformers_utils.py", line 224, in get_tokenizer
    tokenizer = AutoTokenizer.from_pretrained(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/muqi/.base/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1013, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.base/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2025, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
File "/root/.base/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2101, in _from_pretrained
    chat_templates["default"] = chat_template_handle.read()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/share/uv/python/cpython-3.12.9-linux-x86_64-gnu/lib/python3.12/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 43: ordinal not in range(128)

This error occurs because the transformers library, when attempting to read the chat_template.jinja file, does not explicitly set an encoding. In torch.distributed settings, child processes often fail to inherit LANG, LC_ALL, or LC_CTYPE environment variables. Consequently, open() defaults to the ASCII encoding, which cannot decode non-ASCII bytes present in the chat_template.jinja file (like 0xe5 in the GLM-4 tokenizer).

To address this, I've submitted a huggingface/transformers#38553 that explicitly sets encoding="utf-8" when opening such files. This ensures consistent and reliable file reading across different environments. I have manually set it locally, it works successfully.

Yes, I encountered this issue as well, and I resolved it by setting all locale environment variables to en_US.UTF-8. Your fix is more convenient.

[FlamingoPg]

Looks good. Is this PR ready to merge?

[solrex]

Looks good. Is this PR ready to merge?

Yes.

[sixsixcoder]

@zhaochenyang20 run sglang gsm8k benchmark evaluation results use GLM-4-0414

[zhaochenyang20]

@zhaochenyang20 run sglang benchmark evaluation results use GLM-4-0414

thanks. will merge this