Scale the Reward Distribution contract

[kalabukdima]

We've hit the gas limit at ~900 workers, so a new contract has to be developed

[kalabukdima]

What's left:

• fetch state from the contract in case of interrupt
• switch stats collection to new backend
• fix API incompatibilities

[kalabukdima]

What's left:

• fetch state from the contract in case of interrupt
• switch stats collection to new backend (DK)
• fix API incompatibilities
• switch reward monitor ingress (DK)
• move to the real testnet instead of vnet
• debug the issue with targetCapacity call

[kalabukdima]

Another issue emerged: we can't get precisely the same reward amounts, so the Merkle root can't be reproduced. In this situation we get stuck because we can't restore the initial input that produced that root.
A solution would be to upload the reward amounts to S3 and send the link to that file to the contract. We could then fallback to that data if the calculations didn't lead to the same Merkle root.

What's left:

• fetch state from the contract in case of interrupt
• switch stats collection to new backend (DK)
• fix API incompatibilities
• figure out the distribution frequency (it's ~520 times more frequent than expected)
• switch reward monitor ingress (DK)
• store the distribution in S3 to allow recovery
• debug the issue with targetCapacity call
• deploy to the mainnet

[kalabukdima]

What's left:

• fetch state from the contract in case of interrupt
• switch stats collection to new backend (DK)
• fix API incompatibilities
• figure out the distribution frequency (it's ~520 times more frequent than expected)
• switch reward monitor ingress (DK)
• store the distribution in S3 to allow recovery
• debug the issue with targetCapacity call
• deploy to the mainnet

[kalabukdima]

Works in virtual (Tenderly) mainnet, collecting stats to a separate dashboard: https://grafana.infra.gc.subsquid.io/d/ed83c95c-6ee6-433c-b384-cfdd7149eb30/network-rewards?orgId=1&from=now-7d&to=now&timezone=utc&var-network=virtual_mainnet&var-includeJailed&var-period=30

Next steps:

• Switch reward monitor ingress (DK)
• Deploy to the real testnet (via Fordefi)
• Deploy to the real mainnet