[router] bindings for go

[whybeyoung]

A high-level Go SDK for interacting with SGLang gRPC API, designed with an OpenAI-style API for familiarity and ease of use.

CC @slin1237

[gemini-code-assist]

Summary of Changes

Hello @whybeyoung, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request delivers a brand new Go SDK for the SGLang gRPC API. The primary goal is to provide Go developers with a familiar and easy-to-use interface, mirroring the popular OpenAI SDK. It enables seamless interaction with SGLang's capabilities, including real-time streaming and advanced chat functionalities, by integrating a high-performance Rust FFI layer. The SDK is thoroughly tested and well-documented to facilitate adoption and development.

Highlights

• New Go SDK: Introduces a high-level Go SDK for interacting with the SGLang gRPC API, designed with an OpenAI-style API for familiarity and ease of use.
• Core Functionality via Rust FFI: The SDK leverages a Rust Foreign Function Interface (FFI) for its core logic, ensuring performance and robust integration with the SGLang router.
• Chat Completion Features: Supports both streaming and non-streaming chat completions, including advanced features like tool calling and comprehensive request/response types.
• Comprehensive Testing and Documentation: Includes extensive unit and integration tests, benchmarks, and detailed documentation for all public types and functions, along with practical examples.
• Build and Development Tools: Adds a Makefile for streamlined building of the Rust FFI library and Go development tasks, simplifying the setup and usage process.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces Go bindings for the SGLang gRPC API, which is a significant and well-structured contribution. The new bindings include the Go client library, Rust FFI layer, build files, examples, and comprehensive documentation. My review focuses on improving the developer experience, enhancing the robustness of the Go client, fixing issues in tests, and cleaning up the underlying Rust FFI code. Key areas for improvement include making the build process more conventional, correctly handling context cancellation in the Go client, and improving the accuracy of tests and documentation.