Golden file comparator can't be extended to define a custom behavior

[agacemi]

Description

Current implementation of _TestOptimizationAwareGoldenFileComparator doesn't allow extension mecanism in order to define a custom behavior like allowing tolerance of pixel difference when comparing files. Even a custom comparator is defined, it will be overriden by _TestOptimizationAwareGoldenFileComparator.

Expected Behavior

A custom comparator are also executed in addition of _TestOptimizationAwareGoldenFileComparator.

Additional Context

Extending the current file comparator meets a real need. We would like to tolerate a little pixel difference on CI. Since upgrading to Flutter 3.7, Golden, even when using the Ahem font, Golden tests are platform sensitive. Then when using very_good test (or flutter) on CI running on MacBook Pro, tests fail because golden masters are built on different architecture or machine type (e.g. M1/M2 with arm chip ).

This is a known issue on flutter. A workaround is allow pixel diff when comparing.:

• Golden tests occassionally failing with 0.00% diff eBay/flutter_glove_box#160
• Framework tests fail only on M1 machines flutter/flutter#111739

[carlosfiori]

Facing same problem here

[alestiago]

The latest CLI version (0.17.0) should solve this. Feel free to reopen if you experience issues.