Quick way to debug failing fuzz tests

[xarvh]

My workflow with fuzzy tests is something like

1. I write fuzzy tests
2. The fuzzy tests find some values that make the test fail
3. To reproduce the error exactly, I manually replace the fuzzy test parameters with the specific values found by the fuzzy tests
4. I run the offending code, usually with a lot of Debug.log to understand why it is failing
5. I get swamped in thousands of Debug.log lines because the fuzz test is still being executed hundreds of time, and it takes a lot more time than necessary

Right now the best way I found around this is to cut & paste the whole code of the Fuzzy test and turn it into a new, non-fuzzy test that runs with the parameters found by the fuzzy test, and use Test.only to skip the other tests.

I don't know if the above is the best approach. If it isn't, it would be nice to have a way to run a fuzzy test in non-fuzzy mode, ie with specific parameters, without having to fiddle too much with the code.

[mgold]

That accurately describes the current workflow. I agree that it's less than ideal (although also not terrible).

So what would the ideal be? Maybe upon a failing fuzz test, the test runner creates a temporary file with a unit test with the discovered inputs. It then runs only that test.

There would need to be some interaction to determine

• that you do indeed want to isolate a fuzz failure (perhaps when in --watch mode)
• which failure(s) to isolate (multiple fuzz tests, or perhaps a test has multiple simplest failing values)
• to go back to normal testing (should this happen when the test starts passing?)
• if you want to save these inputs as a regression test