[Feature] Support asynchronous dynamic lora loading/unloading #8162
Description
Checklist
- 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- 2. Please use English, otherwise it will be closed.
Motivation
Dynamic loading/unloading of lora adaptors has been supported in #7446.
However, the server cannot handle requests during loading/unloading, until this process has been finished.
In production environments, we expect that loading/unloading of lora can be executed asynchronously, so the server can keep handling requests.
cc @lifuhuang @slin1237 @Ying1123