-
-
Notifications
You must be signed in to change notification settings - Fork 5
feature: velox server #248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
5ea001f
feature: velox server
rustatian 1361a93
chore: caching
rustatian aad95f4
chore: handle gitlab and default
rustatian ecbcbe9
chore: take into account force_rebuild
rustatian 35fc60f
feature: complete rework of how downloads/builder work
rustatian 5d4fb63
chore: update evicting mechanism of the cached data
rustatian bf0f1bb
chore: update bench, support all hosting platforms
rustatian decf5dd
chore: rename build_platform into the target_platform
rustatian 6315203
chore: more comments
rustatian 5c5ea99
Merge branch 'master' into feature/server
rustatian 5eb1374
chore: apply review fixes
rustatian 6f483a5
chore: resolve TODOs
rustatian c9c5027
chore: update templates
rustatian 908d28a
chore: code review fixes
rustatian b2a466d
chore: update benches
rustatian File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,219 @@ | ||
| # Velox - RoadRunner Build System | ||
|
|
||
| ## Project Overview | ||
|
|
||
| Velox is an automated build system for RoadRunner server and its plugins. It's part of the Spiral/RoadRunner ecosystem and provides tools for building, testing, and managing RoadRunner plugin builds. | ||
|
|
||
| **Key Technologies:** | ||
|
|
||
| - Go 1.24+ | ||
| - Protocol Buffers (protobuf) with buf | ||
| - GitHub/GitLab API integration | ||
| - gRPC and Connect | ||
| - Cobra CLI framework | ||
|
|
||
| ## Repository Structure | ||
|
|
||
| ``` | ||
| ├── api/ # Protocol buffer definitions | ||
| ├── builder/ # Core build logic | ||
| ├── cmd/vx/ # Main CLI entry point | ||
| ├── gen/ # Generated protobuf code | ||
| ├── github/ # GitHub API integration | ||
| ├── gitlab/ # GitLab API integration | ||
| ├── internal/cli/ # CLI command implementations | ||
| ├── v2/ # Version 2 refactored components | ||
| └── velox.toml # Configuration file | ||
| ``` | ||
|
|
||
| ## Common Commands | ||
|
|
||
| ### Building and Testing | ||
|
|
||
| ```bash | ||
| # Run tests | ||
| go test -v -race ./... | ||
| make test | ||
|
|
||
| # Regenerate protobuf code | ||
| make regenerate | ||
| # Or manually: | ||
| rm -rf ./gen && buf generate && buf format -w | ||
|
|
||
| # Build the velox binary | ||
| go build -o vx ./cmd/vx | ||
|
|
||
| # Run velox (requires config) | ||
| ./vx -c velox.toml build | ||
| ./vx -c velox.toml server -a 127.0.0.1:8080 | ||
| ``` | ||
|
|
||
| ### Development Tools | ||
|
|
||
| ```bash | ||
| # Format Go code | ||
| go fmt ./... | ||
|
|
||
| # Run linter (if available) | ||
| golangci-lint run | ||
|
|
||
| # Check dependencies | ||
| go mod tidy | ||
| go mod verify | ||
| ``` | ||
|
|
||
| ## Core Files and Components | ||
|
|
||
| ### Main Entry Points | ||
|
|
||
| - `cmd/vx/main.go` - CLI application entry point | ||
| - `internal/cli/root.go:18` - Root command setup with configuration | ||
|
|
||
| ### Build System | ||
|
|
||
| - `builder/builder.go` - Core build logic | ||
| - `builder/templates/` - Build templates for different versions | ||
| - `v2/builder/` - Refactored v2 build system | ||
|
|
||
| ### Configuration | ||
|
|
||
| - `config.go` - Configuration structure and validation | ||
| - `velox.toml` - Default configuration file format | ||
|
|
||
| ### API Integration | ||
|
|
||
| - `github/repo.go` - GitHub repository management | ||
| - `gitlab/repo.go` - GitLab repository management | ||
| - `api/` - Protocol buffer service definitions | ||
|
|
||
| ## Code Style Guidelines | ||
|
|
||
| ### Go Conventions | ||
|
|
||
| - Follow standard Go formatting (`gofmt`) | ||
| - Use meaningful variable names | ||
| - Prefer explicit error handling | ||
| - Use context.Context for cancellation | ||
| - Follow Go module structure with `github.com/roadrunner-server/velox/v2025` | ||
|
|
||
| ### Protocol Buffers | ||
|
|
||
| - Use buf for generation and formatting | ||
| - Service definitions in `api/service/v1/` | ||
| - Generate code with `buf generate` | ||
|
|
||
| ### Error Handling | ||
|
|
||
| - Use `github.com/pkg/errors` for error wrapping | ||
| - Always handle errors explicitly | ||
| - Use structured logging with zap | ||
|
|
||
| ## Testing Instructions | ||
|
|
||
| ```bash | ||
| # Run all tests with race detection | ||
| make test | ||
| # or | ||
| go test -v -race ./... | ||
|
|
||
| # Run specific package tests | ||
| go test -v ./builder/ | ||
| go test -v ./github/ | ||
|
|
||
| # Run with coverage | ||
| go test -cover ./... | ||
| ``` | ||
|
|
||
| ## Repository Etiquette | ||
|
|
||
| ### Branching | ||
|
|
||
| - Main branch: `master` | ||
| - Feature branches: `feature/description` | ||
| - Use conventional commit messages | ||
|
|
||
| ### Plugin Compatibility | ||
|
|
||
| ⚠️ **Important Plugin Guidelines:** | ||
|
|
||
| - Do not use plugin's `master` branch | ||
| - Use tags with the **same major version** | ||
| - Currently supported plugins version: `v5.x.x` | ||
| - Currently supported RR version: `>=v2024.x.x` | ||
|
|
||
| ### Commit Guidelines | ||
|
|
||
| - Use descriptive commit messages | ||
| - Reference issues when applicable | ||
| - Keep commits focused and atomic | ||
|
|
||
| ## Developer Environment Setup | ||
|
|
||
| ### Prerequisites | ||
|
|
||
| - Go 1.24+ (toolchain: go1.24.0) | ||
| - buf CLI for protocol buffer generation | ||
| - Git for version control | ||
|
|
||
| ### Setup Steps | ||
|
|
||
| 1. Clone repository | ||
| 2. Install dependencies: `go mod download` | ||
| 3. Generate protobuf code: `make regenerate` | ||
| 4. Run tests: `make test` | ||
| 5. Build: `go build ./cmd/vx` | ||
|
|
||
| ### Configuration | ||
|
|
||
| Create `velox.toml` based on project requirements. The CLI expects: | ||
|
|
||
| ```bash | ||
| ./vx -c velox.toml build # Build mode | ||
| ./vx -c velox.toml server # Server mode | ||
| ``` | ||
|
|
||
| ## Unexpected Project Behaviors | ||
|
|
||
| ### Version Management | ||
|
|
||
| - Project uses `v2025` module path | ||
| - Replace directive: `github.com/roadrunner-server/velox/v2025/gen => ./gen` | ||
| - Multiple template versions in `builder/templates/` | ||
|
|
||
| ### Build System | ||
|
|
||
| - Templates are versioned (V2, V2023, V2024, V2025) | ||
| - Build process involves GitHub/GitLab API calls | ||
| - Server mode provides build-as-a-service functionality | ||
|
|
||
| ### Protobuf Generation | ||
|
|
||
| - Uses buf instead of protoc directly | ||
| - Generated code goes into `gen/` directory | ||
| - Must regenerate after proto changes | ||
|
|
||
| ## Useful Development Patterns | ||
|
|
||
| ### Adding New Build Templates | ||
|
|
||
| 1. Create new template in `builder/templates/` | ||
| 2. Update `builder.go` to reference new template | ||
| 3. Add corresponding tests in `builder_test.go` | ||
|
|
||
| ### GitHub/GitLab Integration | ||
|
|
||
| - Use existing pool patterns in `github/pool.go` | ||
| - Implement repository interface consistently | ||
| - Handle rate limiting and authentication | ||
|
|
||
| ### CLI Commands | ||
|
|
||
| - Follow cobra patterns in `internal/cli/` | ||
| - Use persistent flags for common options | ||
| - Implement proper validation in `PersistentPreRunE` | ||
|
|
||
| ## Links and Documentation | ||
|
|
||
| - [RoadRunner Docs](https://docs.roadrunner.dev/customization/build) | ||
| - [Project Repository](https://github.com/roadrunner-server/velox) | ||
| - [Discord Community](https://discord.gg/TFeEmCs) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,6 +5,7 @@ | |
| *.so | ||
| *.dylib | ||
| .venv | ||
| .DS_Store | ||
|
|
||
| # Test binary, built with `go test -c` | ||
| *.test | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,2 +1,7 @@ | ||
| test: | ||
| go test -v -race ./... | ||
| go test -v -race ./... | ||
|
|
||
| regenerate: | ||
| rm -rf ./gen | ||
| buf generate | ||
| buf format -w |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package api.request.v1; | ||
|
|
||
| import "buf/validate/validate.proto"; | ||
|
|
||
| option go_package = "github.com/roadrunner-server/velox/v2025/gen/go/api/request/v1;requestV1"; | ||
|
|
||
| message Platform { | ||
| // GOOS | ||
| string os = 1; | ||
| // GOARCH | ||
| string arch = 2; | ||
| } | ||
|
|
||
| message BuildRequest { | ||
| // request_id is optional and needed to match requests with their responses on the client side | ||
| // must be a valid uuid | ||
| string request_id = 1 [ | ||
| (buf.validate.field).string.uuid = true, | ||
| (buf.validate.field).required = false | ||
| ]; | ||
| // rr version is required and must be a valid semantic version | ||
| string rr_version = 2 [ | ||
| (buf.validate.field).required = true, | ||
| (buf.validate.field).cel = { | ||
| id: "rr_version.format" | ||
| message: "rr_version must be a semantic version starting with 'v' (e.g., v2025.1.0), 'master', or a git commit SHA (7-40 hex characters)" | ||
| expression: "this.matches('^(v\\\\d+\\\\.\\\\d+\\\\.\\\\d+.*|master|[a-f0-9]{7,40})$')" | ||
| } | ||
| ]; | ||
| bool force_rebuild = 3 [(buf.validate.field).required = false]; | ||
| // target platform is required and must be a valid platform. Used to specify the platform for the build (host platform used by default) | ||
| Platform target_platform = 4 [(buf.validate.field).required = false]; | ||
| // plugins are required and represent the list of plugins to be used for the build | ||
| repeated Plugin plugins = 5 [(buf.validate.field).required = true]; | ||
| } | ||
|
|
||
| message Plugin { | ||
| // module name in a Go module format, for example: "github.com/roadrunner-server/velox" or "github.com/roadrunner-server/velox/v2" | ||
| string module_name = 1 [(buf.validate.field).required = true]; | ||
| // plugin github tag | ||
| string tag = 2 [(buf.validate.field).required = true]; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package api.response.v1; | ||
|
|
||
| option go_package = "github.com/roadrunner-server/velox/v2025/gen/go/api/response/v1;responseV1"; | ||
|
|
||
| message BuildResponse { | ||
| string path = 1; | ||
| string logs = 2; | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| syntax = "proto3"; | ||
|
|
||
| package api.service.v1; | ||
|
|
||
| import "api/request/v1/request.proto"; | ||
| import "api/response/v1/response.proto"; | ||
|
|
||
| option go_package = "github.com/roadrunner-server/velox/v2025/gen/go/api/service/v1;serviceV1"; | ||
|
|
||
| service BuildService { | ||
| rpc Build(api.request.v1.BuildRequest) returns (api.response.v1.BuildResponse); | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| version: v2 | ||
| plugins: | ||
| - remote: buf.build/protocolbuffers/go:v1.36.7 | ||
| out: gen/go | ||
| opt: paths=source_relative | ||
| - remote: buf.build/grpc/go:v1.5.1 | ||
| out: gen/go | ||
| opt: | ||
| - paths=source_relative | ||
| - require_unimplemented_servers=false | ||
| - remote: buf.build/bufbuild/connect-go:v1.10.0 | ||
| out: gen/go | ||
| opt: | ||
| - paths=source_relative |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| # Generated by buf. DO NOT EDIT. | ||
| version: v2 | ||
| deps: | ||
| - name: buf.build/bufbuild/protovalidate | ||
| commit: 6c6e0d3c608e4549802254a2eee81bc8 | ||
| digest: b5:a7ca081f38656fc0f5aaa685cc111d3342876723851b47ca6b80cbb810cbb2380f8c444115c495ada58fa1f85eff44e68dc54a445761c195acdb5e8d9af675b6 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| version: v2 | ||
| deps: | ||
| - buf.build/bufbuild/protovalidate:v0.14.2 | ||
|
|
||
| name: buf.build/roadrunner-server/velox | ||
| lint: | ||
| use: | ||
| - STANDARD | ||
| except: | ||
| - FIELD_NOT_REQUIRED | ||
| - PACKAGE_NO_IMPORT_CYCLE | ||
| breaking: | ||
| use: | ||
| - FILE | ||
| except: | ||
| - EXTENSION_NO_DELETE | ||
| - FIELD_SAME_DEFAULT | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.