Skip to content

Make NIXL backend as an implementation of base backend. #5380

Closed
jokerwyt wants to merge 21 commits intosgl-project:mainfrom
jokerwyt:nixl-backend
Closed

Make NIXL backend as an implementation of base backend. #5380
jokerwyt wants to merge 21 commits intosgl-project:mainfrom
jokerwyt:nixl-backend

Conversation

@jokerwyt
Copy link
Contributor

@jokerwyt jokerwyt commented Apr 14, 2025

Extended #5006

Not ready yet. Things to re-consider:

  • Handshake and bootstrap workflow for xPyD, TP>8.
  • What extra server args do we need? And how to align with other backend?
  • What extra request field do we need? Remember to pass them through in all API endpoints. And how to align with other backend?
  • Fix mini_lb on /v1/chat/completions.
  • Remove all debug print.

Motivation

Modifications

Checklist

@jokerwyt jokerwyt closed this Apr 15, 2025
@jokerwyt jokerwyt deleted the nixl-backend branch April 15, 2025 07:53
@jokerwyt
Copy link
Contributor Author

The diff is too large to merge. I will take care of it after PD interface is stable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants