feat(assertions): extend deterministic assertions#33
Open
quanticsoul4772 wants to merge 2 commits intoEvaliphy:mainfrom
Open
feat(assertions): extend deterministic assertions#33quanticsoul4772 wants to merge 2 commits intoEvaliphy:mainfrom
quanticsoul4772 wants to merge 2 commits intoEvaliphy:mainfrom
Conversation
Establishes the deterministicExpect() API and toContain reference implementation as described in issue Evaliphy#26. New module: packages/assertions/src/deterministic/ - types.ts — DeterministicAssertionResult interface {passed, score, reason, message, assertionName, type, durationMs} - DeterministicAssertionError.ts — extends Error with structured formatting showing caller message, assertion name, expected, received - expect.ts — deterministicExpect(value, message?) entry point returning DeterministicMatcherChain; toContain(expected) sync impl Key properties: - Fully synchronous — no async, no LLM client, no judge config - Throws DeterministicAssertionError on failure (caught by engine) - Result shape matches DeterministicAssertionResult contract - Named deterministicExpect to coexist alongside existing LLM expect() tests/deterministic.test.ts — 8 tests covering: - passing result shape and fields - durationMs present - optional message forwarding - throws DeterministicAssertionError on failure - error.result shape on failure - error message contains expected/received/caller-message - error.name === 'DeterministicAssertionError' Fixes Evaliphy#26 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds 7 new methods to DeterministicMatcherChain: - toNotContain(expected) — substring absence check - toContainAll(expected[]) — all substrings must be present - toContainAny(expected[]) — at least one substring must be present - toMatchRegex(pattern) — RegExp match - toHaveMinLength(min) — length >= min - toHaveMaxLength(max) — length <= max - toHaveLengthBetween(min, max) — min <= length <= max Each method follows the existing toContain pattern: returns DeterministicAssertionResult, throws DeterministicAssertionError on failure. Tests added for all 7 methods (pass + throw cases). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Someone is attempting to deploy a commit to the priyanshus's projects Team on Vercel. A member of the Team first needs to authorize it. |
Collaborator
|
Its depended on other PR #29 Keeping it on hold. |
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves #27. Extends
DeterministicMatcherChainwith 7 new methods following the pattern established in #29.toNotContain(expected)toContainAll(expected[])toContainAny(expected[])toMatchRegex(pattern)RegExptoHaveMinLength(min)toHaveMaxLength(max)toHaveLengthBetween(min, max)All methods return
DeterministicAssertionResultand throwDeterministicAssertionErroron failure.Test plan
pnpm run test)pnpm run lint)🤖 Generated with Claude Code