Add Jet-Nemotron

[futrime]

Motivation

To add support for Jet-Nemotron.

Modifications

• Added Jet-Nemotron implementation.
• Registered Jet-Nemotron as hybrid GDN attention model.
• Added Jet-Nemotron configuration.

Accuracy Tests

Model	GSM8K	MMLU
Jet-Nemotron-2B	0.762	0.622
Jet-Nemotron-4B	0.792	0.675

Benchmarking and Profiling

$ python3 -m sglang.bench_serving --backend sglang --num-prompt 100
============ Serving Benchmark Result ============
Backend:                                 sglang    
Traffic request rate:                    inf       
Max request concurrency:                 not set   
Successful requests:                     100       
Benchmark duration (s):                  36.61     
Total input tokens:                      33839     
Total input text tokens:                 33839     
Total input vision tokens:               0         
Total generated tokens:                  21640     
Total generated tokens (retokenized):    12768     
Request throughput (req/s):              2.73      
Input token throughput (tok/s):          924.20    
Output token throughput (tok/s):         591.03    
Total token throughput (tok/s):          1515.23   
Concurrency:                             39.95     
----------------End-to-End Latency----------------
Mean E2E Latency (ms):                   14626.36  
Median E2E Latency (ms):                 13954.04  
---------------Time to First Token----------------
Mean TTFT (ms):                          1397.39   
Median TTFT (ms):                        1266.72   
P99 TTFT (ms):                           2969.48   
---------------Inter-Token Latency----------------
Mean ITL (ms):                           61.49     
Median ITL (ms):                         65.26     
P95 ITL (ms):                            93.59     
P99 ITL (ms):                            205.48    
Max ITL (ms):                            1660.42   
==================================================

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[zhaochenyang20]

Nice done!

[zhaochenyang20]

I tends not to change this 😂

[zhaochenyang20]

Only two CI left. Let's wait and see.

[zhaochenyang20]

congrats!