bugfix: reduce latent optimisation permutation explosion

[leondz]

requires #1152
resolves #1161

create snippets by sampling & using a set to avoid dupes, instead of sampling from all perms

Verification

List the steps needed to make sure this thing works

• garak -m test -p latentinjection.LatentWhoisSnippet,latentinjection.LatentWhoisSnippetFull
• garak -m test -p latentinjection.LatentInjectionFactSnippetEiffel,latentinjection.LatentInjectionFactSnippetLegal

[jmartin-tech]

Testing with single generation reports expected counts:

latentinjection.LatentInjectionFactSnippetEiffel                            base.TriggerListDetector: PASS  ok on  256/ 256
latentinjection.LatentInjectionFactSnippetLegal                             base.TriggerListDetector: PASS  ok on  256/ 256

latentinjection.LatentWhoisSnippet                                          base.TriggerListDetector: PASS  ok on  256/ 256
latentinjection.LatentWhoisSnippetFull                                      base.TriggerListDetector: PASS  ok on  640/ 640

[jmartin-tech]

This seems odd still, what does soft_probe_prompt_cap have to do with number of valid contexts?

Not a blocker, just seems like an odd hidden side-effect for soft_probe_prompt_cap.

[jmartin-tech]

Noted impact for setting soft_probe_prompt_cap > 16k

latentinjection.LatentWhoisSnippet                                          base.TriggerListDetector: PASS  ok on 4000/4000
latentinjection.LatentWhoisSnippetFull                                      base.TriggerListDetector: PASS  ok on 4000/4000

[jmartin-tech]

Testing showed some concerns in local model exec, I am not sure this is due to the change here however:

probes.latentinjection.LatentWhois:   7%|███▊                                                 | 12/168 [00:08<01:29,  1.75it/s]This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (1024). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1369: indexSelectSmallIndex: block: [3,0,0], thread: [96,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1369: indexSelectSmallIndex: block: [3,0,0], thread: [97,0,0] Assertion `srcIndex < srcSelectDimSize` failed.

Further investigation in progress.

[jmartin-tech]

Issue is confirmed to exist in released v0.10.3.1 when targeting gpt2:

python -m garak -m huggingface -n gpt2 -p latentinjection.LatentWhois -g 1
garak LLM vulnerability scanner v0.10.3.1 ( https://github.com/NVIDIA/garak ) at 2025-04-29T16:28:14.400861
📜 logging to /home/testing/.local/share/garak/garak.log
🦜 loading generator: Hugging Face 🤗 pipeline: gpt2
Device set to use cuda
📜 reporting to /home/testing/.local/share/garak/garak_runs/garak.a4d3d7a7-8c36-4b16-8247-4f19ac32a4d0.report.jsonl
🕵️  queue of probes: latentinjection.LatentWhois
probes.latentinjection.LatentWhois:  14%|███████▊                                               | 4/28 [00:00<00:04,  4.82it/s]This is a friendly reminder - the current text generation call will exceed the model's predefined maximum length (1024). Depending on the model, you may observe exceptions, performance degradation, or nothing at all.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1369: indexSelectSmallIndex: block: [3,0,0], thread: [96,0,0] Assertion `srcIndex < srcSelectDimSize` failed.
/pytorch/aten/src/ATen/native/cuda/Indexing.cu:1369: indexSelectSmallIndex: block: [3,0,0], thread: [97,0,0] Assertion `srcIndex < srcSelectDimSize` failed.

Just noting findings, this will not block merge.