enable complex arguments for "npm run render-goldens" command#4542
enable complex arguments for "npm run render-goldens" command#4542elalish merged 17 commits intogoogle:masterfrom bhouston:update-screenshots-cli
Conversation
…support for --missing-only, --quiet and --dry-run
elalish
left a comment
There was a problem hiding this comment.
This looks great, but can you update our README with the descriptions from your comment? I'm going to give you the benefit of the doubt that you've tested this yourself, since you're using it more than I am at the moment anyway.
|
@elalish I've updated the README for "npm run render-goldens." |
|
I've merged in the "npm run test" CLI update into this PR to make it simpler. I've also further updated the README to contain both sets of options in it. |
|
I've renamed "npm run update-screenshots" to "npm run render-goldens", it is much more accurate/descriptive, e.g. the script renders the images that go into the goldens folder. |
|
In the latest push I've made it so that the renderer you are testing for fidelity is configurable. It defaults to "model-viewer" (for full backwards compatibility) but you can specify any of the web-based renderers (filament, babylon, gltf-sample-viewer, three-gpu-renderer) and it works. |
elalish
left a comment
There was a problem hiding this comment.
Thank you, this looks like a huge improvement in usability!
| -r, --renderer | | Limit to specific renderers. This now allows you to specify multiple renderers in the whitelist, rather than only one. | ||
| -s, --scenario | | Limit to specific scenarios. This now also allows you to specify multiple scenarios in the whitelist. You can use a full name or a partial name of scenarios and it will match against all that contain that scenario substring. | ||
| -p, --port | 9040 | Port for web server. | ||
| -m, --missing-only | false | Only render if an output image is missing. Very useful when adding new tests. |
…#4542) * enable complex line arguments for update-renderer command, including support for --missing-only, --quiet and --dry-run * remove debug messages. * allow setting the port. * support --scenario being a subset of the scenario name and case insensitive * add cli for render fidelity tests, allow dry run, quiet modes. * better description * enable quite mode in CI. add missing types for yargs * improve readme for new update-screenshots cli. * update readme with new "npm run test" CLI options. * rename "npm run update-screenshots" to "npm run render-goldens", more accurate name. * make which renderer you fidelity testing a CLI parameter, defaults to "model-viewer" * be clear which renderers are supported, just web-based ones. * update readme with new renderer choice for fidelity test. * remove debug code.
This implements this idea: #4536
To run a subset of renders or scenarios do something like this:
Explanation of the new features:
--renderer, this now allows you to specify multiple renderers in the whitelist, rather than only one.
-- scenario, this now also allows you to specify multiple scenarios in the whitelist, rather than only one. It now also selects scenarios based on being a subset of the config.json scenario name, thus you can run all tests that contain say "transform" or "clearcoat".
--missing-only is very useful when I add new tests and I don't want to render all other tests just to get goldens for the new ones.
--dry-run, when I want to figure out which tests will run given that command line.
--quiet, I can run update-screenshots in the background while working without Chrome constantly popping up to disrupt my flow. Also gives about a 10% speed boost on my machine.
--port, allows me to run two of these at the same time, useful when I want to run two different renderers at the same time, or work on two separate branches simultaneously.