Commit 1bec83e
committed
test(core): unpin evals while keeping integration tests pinned
- Removes hardcoded model from TestRig for unpinned evaluations.\n- Pins integration tests to gemini-2.5-pro via new GEMINI_TEST_TYPE=integration env var.\n- Moves shell efficiency evals to USUALLY_PASSES to prevent PR blocking while tracking Gemini 3.1 parent 4494f9e commit 1bec83e
3 files changed
Lines changed: 13 additions & 6 deletions
File tree
- evals
- integration-tests
- packages/test-utils/src
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
23 | | - | |
| 23 | + | |
24 | 24 | | |
25 | 25 | | |
26 | 26 | | |
| |||
49 | 49 | | |
50 | 50 | | |
51 | 51 | | |
52 | | - | |
| 52 | + | |
53 | 53 | | |
54 | 54 | | |
55 | 55 | | |
| |||
72 | 72 | | |
73 | 73 | | |
74 | 74 | | |
75 | | - | |
| 75 | + | |
76 | 76 | | |
77 | 77 | | |
78 | 78 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
23 | 26 | | |
24 | 27 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
408 | 408 | | |
409 | 409 | | |
410 | 410 | | |
411 | | - | |
412 | | - | |
413 | | - | |
| 411 | + | |
| 412 | + | |
| 413 | + | |
| 414 | + | |
| 415 | + | |
| 416 | + | |
| 417 | + | |
414 | 418 | | |
415 | 419 | | |
416 | 420 | | |
| |||
0 commit comments