update detectors to reflect immutable attempt prompt#1415
update detectors to reflect immutable attempt prompt#1415leondz merged 3 commits intoNVIDIA:mainfrom
Conversation
|
|
This is no longer the case as of #1254, happy to discuss further. |
* remove usage of `Attempt.all_outputs` * update detectors to process only latest `Attempt.outputs` * update detectors to support `None` value outputs Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
8468014 to
e390e4b
Compare
|
outcome of discussion was that we should support different scopes of detection, controllable by will this PR limit |
|
|
Great, lgtm |
leondz
left a comment
There was a problem hiding this comment.
A few clarifications, no major blockers
* expand test for detector instance to ensure result order if preserved * denote testing detectors are not expected to return `None` * account for `0.0` treated as `False` * account for `None` in message text of productkey detector * return `None` for `None` in Json detector Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
* output `None` -> detector result `None` * output `Messsage(text=None)` -> detector result `None` Signed-off-by: Jeffrey Martin <jemartin@nvidia.com>
Attempt.promptis considered immutable once set as of #1254. This revision aligns expectation thatoutputsis only set once on an attempt after the generator provides inference results.Attempt.all_outputsAttempt.outputsNonevalue outputsAdditional optimization are envisioned and should be considered in a future iteration:
Attempt.all_outputsVerification
List the steps needed to make sure this thing works