Support multi-adpater concurrent inferencing

[whybeyoung]

Feature request

Hey all:

Very glad to see that PR-263 , but i try it and find that it use set_adpater method to switch lora-adapter , it means that we can only have concurrency with 1 in the same time. i see that lora layer can be inference without merged to origin base model. So can we support an way that permit user dynamicaly combine base_model and lora_adapter, and user can do concurrent call to base_model and lora_adapter

Thanks.

Motivation

This way can save many money............

Your contribution

NA

[whybeyoung]

oh , i find it .. #804

[BenjaminBossan]

#873 added support for multiple active adapters at once, if that's what you mean. The PR is merged but there hasn't been a release yet, so you would have to install from main to test the feature.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.