Skip to content

bug fix: enable extended detectors, module-specified encoding payloads by default#1238

Merged
leondz merged 1 commit intoNVIDIA:mainfrom
leondz:bug/corecfg_20250528
Jun 2, 2025
Merged

bug fix: enable extended detectors, module-specified encoding payloads by default#1238
leondz merged 1 commit intoNVIDIA:mainfrom
leondz:bug/corecfg_20250528

Conversation

@leondz
Copy link
Collaborator

@leondz leondz commented May 28, 2025

garak.core.yaml had two undesired behaviors:

  1. extended detectors=false , should now be true
  2. probes.encoding config overrode the defaults in the encoding.EncodingMixin class which had been updated to cover more payloads by default

Verification

  • default run uses extended detectors; try python -m garak -m test -p grandma.Win11 - two detectors should run
  • encoding probes should have more than 30 inferences with generations=1; try python -m garak -m test -p encoding -g1 and check that scores are out of max _config.run._soft_prompt_probe_cap and not 30.

@leondz leondz added bug Something isn't working probes Content & activity of LLM probes labels May 28, 2025
@leondz leondz requested a review from jmartin-tech May 29, 2025 05:48
@leondz leondz merged commit f871108 into NVIDIA:main Jun 2, 2025
13 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators Jun 2, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

bug Something isn't working probes Content & activity of LLM probes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant