Eager Compiler for Torch Compile

[Oasis-Git]

Motivation

To support piecewise cuda graph, previous compile backend sglang applies torch default compile backend as Inductor pattern compiler. While this compiler introduce better performance for native torch operations, it introduces a lot of additional issues especially with sgl kernels and cuda implementation of custom ops. Thus in the pr we hope to switch the compiler mode back to eager in which torch only capture fx.graph but not compile it further. It will introduce some benefits such as:

1. Better Support For Kernels/Ops: Developers will skip many compile issues before using piecewise cuda graph. From the perspective of performance with eager mode compiler we can natively support original kernel design and custom ops implementation.
2. Zero Compile Cost: The compilation takes much time for warmup and with eager mode compiler this process can be skipped.

Modifications

Introduction of Eager Compiler Implementation and corresponding server args support

Accuracy Tests

Benchmarking and Profiling

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.

[gemini-code-assist]

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!