-
Notifications
You must be signed in to change notification settings - Fork 18
Description
Issue identified in #936, comment reproduced below.
After the update we find that the result comparison still fails but only for a singular data point in one of the examples (log below). Digging a little deeper (looking at the plot) we find that the point of failure is at a place were the spectral value in question is very small (~-1.497), compared to the overall maximum of the spectrum (~7500). However, the spectra are compared locally (per wavelength) and the absolute tolerance for the comparison between results is scaled with the local value of the data, meaning that the tolerance for a point close to zero is much tighter than the tolerance for a point near the maximum. This is a point where the result comparison could be further refined, but for now we accept this as a PASS ✔️ and this PR may be merged.
FAILED .github/test_result_consistency.py::test_result_data_var_consistency[study_fluorescence-decay_associated_spectra]
E AssertionError: Result data_var data mismatch: 'decay_associated_spectra' in 'dataset1.nc'.
E With sum of absolute difference: 0.0039020775839004873 and shape: (49, 5)
E Mean difference: 1.592684728122648e-05
E
E assert False
E + where False = <function allclose.<locals>._allclose at 0x7f40599e5790>(<xarray.DataArray 'decay_associated_spectra' (spectral: 49, component: 5)>\narray([[-145.581345, 52.590453, 21.6330...7\n rate (component) float64 ...\n lifetime (component) float64 ...\nDimensions without coordinates: component, <xarray.DataArray 'decay_associated_spectra' (spectral: 49, component: 5)>\narray([[-145.581346, 52.590453, 21.6330...7\n rate (component) float64 ...\n lifetime (component) float64 ...\nDimensions without coordinates: component, atol=array([[1.73546487e-05, 6.26927054e-06, 2.57886253e-06, 2.17881660e-06,\n 6.32769907e-06],\n [1.50005268e-0... 5.91504505e-06],\n [5.92400043e-05, 4.45607905e-05, 4.22466039e-06, 1.01596965e-04,\n 6.42848991e-06]]), rtol=1e-05, print_fail=20)
.github/test_result_consistency.py:221: AssertionError
----------------------------- Captured stdout call -----------------------------
allclose first 1 failures:
(40, 1): -1.4969902223364386 -1.4969635695838823
=========================== short test summary info ============================
FAILED .github/test_result_consistency.py::test_result_data_var_consistency[study_fluorescence-decay_associated_spectra]Originally posted by @jsnel in #935 (comment)