Mimic `Cubature.INDIVIDUAL` behavior [Feature Req.]

[agerlach]

I started to put a PR together to allow for convergence testing based on the individual integrands, like is default in Cubature.jl. However, I quickly realized it may get more involved than I initially thought as the error estimate E is assumed to be scalar in cubrule and the box updates.

Do you have any thoughts/suggestions for adding this feature? Thanks.

[stevengj]

If you use an absolute tolerance, then you can just pass norm=x->norm(x,Inf) (i.e. use the L∞ norm). For relative tolerances, however, that's not equivalent to Cubature.INDIVIDUAL.

Can you explain more about your use-case? Do you have a bunch of integrals with completely unrelated magnitudes? Why not do the integrals separately?

[agerlach]

I want to do optimization w/ reverse mode AD over the quadrature. I have an adjoint written that does ∫f(x,p)dx → ∫(∂/∂p f(x,p))dx. Not having individual convergence criterion for each partial can lead to instabilities in the optimization.

[agerlach]

Why not do the integrals separately?

The integrand is expense to compute. It is of the form g(S(x)) where S is an ODE solve. If interested in more details see our paper https://arxiv.org/pdf/2008.08737.pdf

[stevengj]

I'm not sure I follow. For optimization, isn't it sufficient for the computed gradient (via quadrature) to be sufficiently close to the true gradient in some norm? i.e. isn't a combined error norm precisely what is needed?

[agerlach]

@stevengj You are absolutely correct. Thank you. I miss characterized the behavior I saw. On second thought, the issue may be with respect to using constant tolerances for the quadrature gradient computation in the optimization process. Does it seem reasonable to need to update these as the optimization converges?

[stevengj]

It depends on the optimization algorithm (and how tolerant it is of inexact gradients), but in general I would set the quadrature tolerance to something proportional to the convergence tolerance you want from the optimization.

If you want the optimum to a high accuracy (low tolerance), then it might be reasonable to optimize once using a high tolerance, and then "polish" the solution by running the again with a lower tolerance starting at the high-tolerance solution.