[NPU][1/N] NPU basic functions refactor and new modelslim quant type

[iforgetmyname]

Motivation

Due to the underlying structural difference between gpgpus and npus, we have introduced a lot of is_npu branches in current repository from previous commits. Though literarlly it helps the out-of-box experience for our end-users and matches our rapid development pace, this way of orignizing codes breaks readability and of cource maintainability of the whole sglang project. We believe this is not a long-term solution and a healthy and robust way of continously maintaining multi-hardware support, such that starting with this pr, we are trying to refactoring npu-related codes into a specific folder that hides hardware differences and only exposes simplified interfaces that can be called by different models.

Modifications

• Tigger transfer_to_npu when engine is starting that mocks all supported torch.cuda calls to torch.npu
• • https://github.com/sgl-project/sglang/pull/13359/files#diff-1f1b431bcaea00cda2a62efe5d5f702f8bc0445982c1ec0b44d31d025f8faf68R72
• Introduce a helper function npu_format_cast that wraps data format casting on npu and condition checks for npu-only, this data format casting helps 5% performance improvement on npu
• Refactor out npu-related TokenToKVPool, PagedTokenToKVPoolAllocator, top_k forward and ascend attention backend
• Introduce a new quant type modelslim that supports npu-specific W8A8, W4A8-MoE, W4A16-MoE mixed quant methods

Accuracy Tests

CI should cover all.

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[sglang-bot]

help us clean the code

if is_cuda:
  ...
elif is_hip:
  ...
elif is_npu:
   ...
else:

[iforgetmyname]

/tag-and-rerun-ci

[iforgetmyname]

/tag-and-rerun-ci