Open
Conversation
9014b58 to
8f84581
Compare
f5d97e2 to
213483c
Compare
Revert benchmarks.yaml to main state (remove bare-runner toolchain fixes) and create a separate benchmarks-custom-runners.yaml targeting dedicated GCP self-hosted runners for consistent, reproducible benchmark results. Runner assignment: - runner-large: setup/build job (Rust compilation) - runner-medium: all benchmark jobs - runner-small: validation and aggregation Includes toolchain installations for bare runners (Rust for Python bindings, Maven for Java, clang/libclang for PHP). Triggered on pull_request and workflow_dispatch.
- Java: use setup-maven-4 action (Maven >= 3.9.11 required by enforcer) - Ruby: mkdir RUNNER_TOOL_CACHE before setup-ruby (permission denied) - C#: set DOTNET_INSTALL_DIR to writable path (permission denied) - PHP: search /usr/lib/llvm-*/lib/ for libclang (ldconfig misses it) - Third-party Python benchmarks: install build-essential (linker cc missing) - Set RUNNER_TOOL_CACHE at workflow level for all self-hosted runner jobs
…enchmarks - Ruby: sudo mkdir /opt/hostedtoolcache (ruby/setup-ruby uses runner context, not RUNNER_TOOL_CACHE env var) - Third-party Python benchmarks (docling, markitdown, pdfplumber, pymupdf4llm, unstructured, mineru): add rustup install for maturin builds triggered by uv run during benchmark execution
…y benchmarks - Replace rustup install with UV_NO_BUILD_PACKAGE=kreuzberg env var and --no-build-package kreuzberg flag on uv sync. The benchmark harness uses uv run which triggers building kreuzberg-py via maturin; this flag prevents that entirely. - Ruby: add clang/libclang install + LIBCLANG_PATH export (bindgen for rb-sys needs libclang, same issue as PHP)
2baaf2f to
d5c04f5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Details