Skip to content

Comments

Add local kv cache store for vllm#2063

Open
vegetableysm wants to merge 2 commits intov6d-io:mainfrom
vegetableysm:kvs
Open

Add local kv cache store for vllm#2063
vegetableysm wants to merge 2 commits intov6d-io:mainfrom
vegetableysm:kvs

Conversation

@vegetableysm
Copy link
Collaborator

What do these changes do?

  • Add local kv cache store for vllm
  • Support store cache by VLLMBlock Object in v6d
  • Support swap kv cache to POSIX filesystem

Related issue number

Fixes #issue number

Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
Signed-off-by: vegetableysm <yuanshumin.ysm@alibaba-inc.com>
@github-actions
Copy link
Contributor

/cc @sighingnow, this issus/pr has had no activity for a long time, please help to review the status and assign people to work on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant