[Router-GO] implement a Go SGLang Router - OpenAI Compatible API Server

sgl-model-gateway/bindings/golang/Makefile

@@ -1,10 +1,10 @@
-# Makefile for sglang-router golang bindings
+# Makefile for sgl-model-gateway golang bindings
 # This builds the Rust FFI library and provides convenience targets for Go development
 
 # Configuration
 CARGO_BUILD_DIR ?= $(shell pwd)/target
 BUILD_MODE ?= release
-LIB_NAME = libsglang_router_rs
+LIB_NAME = libsgl_model_gateway_go
 
 # Detect OS
 UNAME_S := $(shell uname -s)
@@ -30,7 +30,7 @@ PYTHON_LDFLAGS := $(shell python3-config --ldflags --embed 2>/dev/null || python
 
 # CGO flags - use exported lib directory if available, otherwise build directory
 LIB_DIR := $(if $(wildcard $(LIB_EXPORT_PATH)),$(LIB_EXPORT_DIR),$(LIB_BUILD_DIR))
-export CGO_LDFLAGS = -L$(LIB_DIR) -lsglang_router_rs $(PYTHON_LDFLAGS) -ldl
+export CGO_LDFLAGS = -L$(LIB_DIR) -lsgl_model_gateway_go $(PYTHON_LDFLAGS) -ldl
 export $(LD_LIBRARY_PATH_VAR) := $(LIB_DIR):$($(LD_LIBRARY_PATH_VAR))
 
 .PHONY: all build build-dev lib lib-clean clean test examples help run-simple run-streaming check-lib

sgl-model-gateway/bindings/golang/README.md

@@ -40,15 +40,61 @@ A high-level Go SDK for interacting with SGLang gRPC API, designed with an OpenA
 go get github.com/sglang/sglang-go-grpc-sdk
 ```
 
+### Sync Dependencies
+
+```bash
+cd sgl-model-gateway/bindings/golang
+go mod tidy
+```
+
 ### Build Requirements
 
-- Go 1.21 or later
-- Rust toolchain (for building the FFI library)
-- Python 3.x (for Python bindings in Rust FFI)
-- Tokio runtime for async operations
+- Go 1.21+, Rust toolchain, Python 3.x
 
 ## Quick Start
 
+### Benchmark
+
+Run the OpenAI-compatible server and benchmark:
+
+```bash
+# Set environment variables
+export SGL_TOKENIZER_PATH="/Users/yangyanbo/tokenizer"
+export SGL_GRPC_ENDPOINT="grpc://10.109.185.20:8001"
+
+# Run server
+cd examples/oai_server
+bash run.sh
+
+# Run E2E benchmark
+cd ../..
+make e2e E2E_MODEL=/work/models/qwencoder-3b E2E_TOKENIZER=/Users/yangyanbo/tokenizer E2E_INPUT_LEN=1024 E2E_OUTPUT_LEN=512
+```
+
+## Examples
+
+The SDK includes several examples in the `examples/` directory:
+
+- **simple**: Basic non-streaming chat completion example
+- **streaming**: Real-time streaming with performance metrics
+
+### Running Examples
+
+```bash
+# Run simple example
+cd bindings/golang/examples/simple
+bash run.sh
+
+# Run streaming example
+cd bindings/golang/examples/streaming
+bash run.sh
+
+# Or use Makefile from bindings/golang directory
+cd bindings/golang
+make run-simple
+make run-streaming
+```
+
 ### Basic Usage (Non-streaming)
 
 ```go
@@ -162,29 +208,7 @@ func float32Ptr(f float32) *float32 {
 }
 ```
 
-## Examples
-
-The SDK includes several examples in the `examples/` directory:
-
-- **simple**: Basic non-streaming chat completion example
-- **streaming**: Real-time streaming with performance metrics
-
-### Running Examples
-
-```bash
-# Run simple example
-cd bindings/golang/examples/simple
-bash run.sh
-
-# Run streaming example
-cd bindings/golang/examples/streaming
-bash run.sh
 
-# Or use Makefile from bindings/golang directory
-cd bindings/golang
-make run-simple
-make run-streaming
-```
 
 Examples automatically detect the server endpoint and tokenizer path via environment variables or defaults.
 
@@ -286,44 +310,21 @@ go tool cover -html=coverage.out -o coverage.html
 
 #### Unit Test Coverage
 
-- **Configuration validation** (`TestClientConfig`) - Validates ClientConfig requirements
-- **Type structures** - Verifying all struct types work correctly
-- **Response handling** - Testing response parsing and validation
-- **Concurrent operations** (`TestConcurrentClientOperations`) - Thread-safety verification
-- **Benchmarks** (`BenchmarkChatCompletionRequest`) - Performance measurement
-
-**Test Files**:
+- Configuration validation, type structures, response handling, concurrent operations, and benchmarks
 - `client_test.go` - 10 unit tests covering core functionality
-- Tests cover: config validation, message types, request validation, close operations, response types, streaming, tools, concurrency, and context cancellation
 
 ### Integration Tests
 
 Integration tests require a running SGLang server and test the full client-server interaction.
 
 #### Prerequisites
 
-1. Start an SGLang server:
-
-```bash
-# Using Python (requires sglang package installed)
-python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-hf
-
-# Or using pre-built Docker image
-docker run -p 20000:20000 lmsys/sglang:latest
-
-# Or build your own
-sglang launch_server --model-path <model_path>
-```
-
-2. Set required environment variables:
-
-```bash
-# Set the gRPC endpoint (default: grpc://localhost:20000)
-export SGL_GRPC_ENDPOINT=grpc://localhost:20000
-
-# Set the tokenizer path (required)
-export SGL_TOKENIZER_PATH=/path/to/tokenizer
-```
+1. Start SGLang server: `python -m sglang.launch_server --model-path <model_path>`
+2. Set environment variables:
+   ```bash
+   export SGL_GRPC_ENDPOINT=grpc://localhost:20000
+   export SGL_TOKENIZER_PATH=/path/to/tokenizer
+   ```
 
 #### Running Integration Tests
 
@@ -352,54 +353,13 @@ go test -tags=integration -race ./...
 
 ### Benchmarks
 
-Measure performance of SDK operations:
-
 ```bash
-# Run all benchmarks
 go test -bench=. -benchmem ./...
-
-# Run specific benchmark
-go test -bench=BenchmarkChatCompletionRequest -benchmem
-
-# Run for longer duration
-go test -bench=. -benchtime=10s ./...
-```
-
-Current benchmarks:
-- `BenchmarkChatCompletionRequest` - Measures request creation performance
-
-### CI/CD Integration
-
-Add to your GitHub Actions workflow:
-
-```yaml
-- name: Run Go tests
-  run: |
-    go test -race -cover ./...
-
-- name: Run integration tests (on main branch)
-  if: github.ref == 'refs/heads/main'
-  env:
-    SGL_GRPC_ENDPOINT: grpc://localhost:20000
-    SGL_TOKENIZER_PATH: /path/to/tokenizer
-  run: go test -tags=integration ./...
 ```
 
 ## Documentation
 
-### Code Documentation
-
-All public types and functions include comprehensive documentation:
-
-1. **Package-level documentation** in `client.go` with usage examples
-2. **Type documentation** for all structs with field descriptions
-3. **Function documentation** with:
-   - Purpose and behavior description
-   - Parameter documentation with types and constraints
-   - Return value documentation
-   - Error cases and handling
-   - Safety notes (for FFI functions)
-   - Usage examples
+All public types and functions include comprehensive documentation with usage examples.
 
 ### Key Documented Components
 
@@ -413,45 +373,19 @@ All public types and functions include comprehensive documentation:
 
 ### Viewing Documentation
 
-Generate and view HTML documentation:
-
 ```bash
-# Install godoc (if not already installed)
-go install golang.org/x/tools/cmd/godoc@latest
-
-# Generate and serve documentation
 godoc -http=:6060
-
 # Visit: http://localhost:6060/pkg/github.com/sglang/sglang-go-grpc-sdk/
 ```
 
 ## Development
 
-### Building
-
 ```bash
 cd bindings/golang
-
-# Build the Go bindings (compiles Rust FFI library)
-make build
-
-# Clean build
-make clean && make build
-```
-
-### Code Quality
-
-Ensure code quality before committing:
-
-```bash
-# Run Go vet (check for potential bugs)
-go vet ./...
-
-# Format code
-go fmt ./...
-
-# Run all tests with race detection
-go test -race ./...
+make build          # Build Go bindings
+go vet ./...        # Check code quality
+go fmt ./...        # Format code
+go test -race ./... # Run tests
 ```
 
 ### Project Structure
@@ -478,27 +412,23 @@ bindings/golang/
 
 ## Troubleshooting
 
-### Connection Errors
+### Missing Dependencies
 
-**Error**: `connection refused` or `failed to dial`
+Run `go mod tidy` to sync dependencies.
 
-**Solution**:
-1. Ensure SGLang server is running: `python -m sglang.launch_server`
-2. Check endpoint: `echo $SGL_GRPC_ENDPOINT`
-3. Verify port is not blocked: `nc -zv localhost 20000`
+### Connection Errors
 
-### Tokenizer Not Found
+Ensure SGLang server is running and check `SGL_GRPC_ENDPOINT`.
 
-**Error**: `tokenizer path not found` or `tokenizer configuration missing`
+### Tokenizer Not Found
 
-**Solution**:
-1. Set `SGL_TOKENIZER_PATH` environment variable
+Set `SGL_TOKENIZER_PATH` environment variable.
 2. Verify path contains required files: `ls $SGL_TOKENIZER_PATH`
 3. Files should include: `tokenizer.json`, `vocab.json`, `config.json`
 
 ### Build Failures
 
-**Error**: `library 'sglang_router_rs' not found`
+**Error**: `library 'sgl_model_gateway_go' not found`
 
 **Solution**:
 1. Rebuild Rust library: `cd sgl-model-gateway/bindings/golang && make build`