Discussed offline with @ReubenBond
This won't happen in steady state.
However let say there are very many long running grains, and reminders are setup for them. Then let say cluster is restarted completely due to some issues and if there are only a 1-3 pods after full cluster restart, LocalReminderService will send pretty much all the old reminders to these few pods. This can cause issues due to heavy load on those pods.
Discussed offline with @ReubenBond
This won't happen in steady state.
However let say there are very many long running grains, and reminders are setup for them. Then let say cluster is restarted completely due to some issues and if there are only a 1-3 pods after full cluster restart, LocalReminderService will send pretty much all the old reminders to these few pods. This can cause issues due to heavy load on those pods.