feat: Add benchmarking harness with spot suite#10

Merged

ilblackdragon merged 13 commits intomainfrom

Feb 17, 2026

Commits on Feb 17, 2026

feat: Add benchmarking harness for agent evaluation

ilblackdragon
and
claude
committed
feat: Add spot benchmark suite for end-to-end agent verification

ilblackdragon
and
claude
committed
fix: address audit findings in benchmarks crate

ilblackdragon
and
claude
committed
fix: rewrite tasks.jsonl with scored results after scoring

ilblackdragon
and
claude
committed
feat: prefix benchmark runs with model name and commit hash

ilblackdragon
and
claude
committed
feat: add 8 memory benchmark scenarios to spot suite

ilblackdragon
and
claude
committed
chore: fmt channel.rs and gitignore bench-results

ilblackdragon
and
claude
committed
fix: address critical and high findings from PR review

ilblackdragon
and
claude
committed
fix: relax smoke-greeting assertion to not demand parrot greeting

ilblackdragon
and
claude
committed
feat: 100% spot baseline (GPT-5.2 @ 2c43b83, 21/21 pass)

ilblackdragon
and
claude
committed
fix: address remaining PR review comments

ilblackdragon
and
claude
committed
fix: reject absolute paths in SWE-bench and validate matrix config

ilblackdragon
and
claude
committed
fix: fail tasks on setup_task error and compute git hash once

ilblackdragon
and
claude
committed