Skip to content

[RelayMiner] Add comprehensive relay timing and gRPC call metrics#1703

Merged
red-0ne merged 10 commits intomainfrom
feat/relay-timing
Aug 4, 2025
Merged

[RelayMiner] Add comprehensive relay timing and gRPC call metrics#1703
red-0ne merged 10 commits intomainfrom
feat/relay-timing

Conversation

@red-0ne
Copy link
Copy Markdown
Contributor

@red-0ne red-0ne commented Jul 30, 2025 •
edited
Loading

Summary

Add detailed timing metrics for relay request/response preparation and gRPC call duration monitoring

Primary Changes:

  • Add MeteredClientConn wrapper for gRPC call duration tracking with component-based labeling
  • Implement relay request/response preparation duration metrics in proxy handler
  • Add context-based component identification for metrics (proxy, miner, sessions_manager)
  • Enhance error response handling with proper Content-Length and Connection headers

Secondary changes:

  • Add comprehensive logging with timing information throughout relay processing
  • Update supplier config to use metered gRPC client connection
  • Add detailed duration tracking for backend service calls and response preparation

Issue:

Performance monitoring and timing analysis for relay processing pipeline

Type of change

Select one or more from the following:

  • New feature, functionality or library
  • Bug fix
  • Code health or cleanup
  • Documentation
  • Other (specify)

Sanity Checklist

  • I have updated the GitHub Issue Metadata: assignees, reviewers, labels, project, iteration and milestone
  • For docs: make docusaurus_start
  • For small changes: make go_develop_and_test and make test_e2e
  • For major changes: devnet-test-e2e label to run E2E tests in CI
  • For migration changes: make test_e2e_oneshot
  • 'TODO's, configurations and other docs

🤖 Generated with Claude Code

@red-0ne @claude
feat: Add comprehensive relay timing and gRPC call metrics
5fe0dc3 
🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions github-actions bot added the consensus-breaking IMPORTANT! If the PR with this tag is merged, next release WILL HAVE TO BE an upgrade. label Jul 30, 2025
@red-0ne red-0ne changed the title feat: Add comprehensive relay timing and gRPC call metrics [RelayMiner] Add comprehensive relay timing and gRPC call metrics Jul 30, 2025
@red-0ne red-0ne self-assigned this Jul 30, 2025
@github-actions github-actions bot removed the consensus-breaking IMPORTANT! If the PR with this tag is merged, next release WILL HAVE TO BE an upgrade. label Jul 30, 2025
@red-0ne red-0ne added relayminer Changes related to the Relayminer miner Changes related to the Miner proxy Changes related to the Proxy scalability labels Jul 30, 2025
@red-0ne red-0ne added this to Shannon Jul 30, 2025
@github-project-automation github-project-automation bot moved this to 📋 Backlog in Shannon Jul 30, 2025
@red-0ne red-0ne moved this from 📋 Backlog to 👀 In review in Shannon Jul 30, 2025
@red-0ne red-0ne added this to the RelayMiner Stability milestone Jul 30, 2025
@red-0ne red-0ne requested a review from Olshansk July 30, 2025 17:44
Olshansk
Olshansk requested changes Jul 30, 2025
View reviewed changes
Copy link
Copy Markdown
Collaborator

@Olshansk Olshansk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@red-0ne Quick preliminary first review.

@red-0ne red-0ne requested a review from Olshansk August 1, 2025 14:03
@github-actions
Copy link
Copy Markdown

github-actions bot commented Aug 2, 2025

The CI will now also run the e2e tests on devnet, which increases the time it takes to complete all CI checks.

You may need to run make trigger_ci to submit an empty commit that'll trigger the tests.

GCP workloads (requires changing the namespace to 1703)
Grafana network dashboard for devnet-issue-1703

@github-actions github-actions bot added devnet push-image CI related - pushes images to ghcr.io labels Aug 2, 2025
@red-0ne red-0ne merged commit 577009a into main Aug 4, 2025
8 checks passed
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in Shannon Aug 4, 2025
bryanchriswhite added a commit that referenced this pull request Aug 15, 2025
@bryanchriswhite
Merge remote-tracking branch 'pokt/main' into chore/localnet/tilt
838841b 
* pokt/main: (34 commits)
  [GOVERNANCE][PARAMS][CUTTM] Updating global CUTTM per Foundation requirement (#1727)
  Update docusaurus/docs/4_develop/upgrades/4_upgrade_list.md w/ v0.1.28 MainNet block
  Updating bulk_params_main/tokenomics_params.json so proposer gets rewards again
  Update upgrade_tx_v0.1.28_main.json
  [Morse Recovery] Add recovery allowlist entries for issues #1718 and #1719 (#1723)
  [Upgrade]: Update v0.1.28 upgrade list docs (#1722)
  [Upgrade] v0.1.28 upgrade txs (#1721)
  [Upgrade] Add v0.1.28 upgrade plan (#1720)
  [CometBFT] Use Pocket's fork temporarily (#1717)
  Use use_cometbft_fork_v038.17 (#1714)
  [RelayMiner] Add session lifetime caching (#1713)
  Update recovery allowlist for recovery-8aug2025 (#1716)
  [Tokenomics] Validator proper decoding (#1710)
  [Session Rollovers] First attempt at solving session rollovers from RelayMiner's POV (#1705)
  [RelayMiner] Add comprehensive relay timing and gRPC call metrics (#1703)
  [Shared Module] Increase `ComputeUnitsPerRelayMax` (#1709)
  [RelayMiner] Improve non-2xx errors visibility, do not mine 5xx (#1695)
  [Governance] - Updated Params to match the ones in the forum  (#1707)
  [RelayMiner] Improve RelayMiner timeout handling and test reliability (#1691)
  [Morse Recovery] Updated Allowlist for Recovery (#1696)
  ...
bryanchriswhite added a commit that referenced this pull request Aug 15, 2025
@bryanchriswhite
Merge branch 'chore/localnet/tilt' into chore/ibc-axelar
f2ec1f5 
* chore/localnet/tilt: (36 commits)
  [GOVERNANCE][PARAMS][CUTTM] Updating global CUTTM per Foundation requirement (#1727)
  Update docusaurus/docs/4_develop/upgrades/4_upgrade_list.md w/ v0.1.28 MainNet block
  Updating bulk_params_main/tokenomics_params.json so proposer gets rewards again
  Update upgrade_tx_v0.1.28_main.json
  [Morse Recovery] Add recovery allowlist entries for issues #1718 and #1719 (#1723)
  [Upgrade]: Update v0.1.28 upgrade list docs (#1722)
  [Upgrade] v0.1.28 upgrade txs (#1721)
  [Upgrade] Add v0.1.28 upgrade plan (#1720)
  [CometBFT] Use Pocket's fork temporarily (#1717)
  Use use_cometbft_fork_v038.17 (#1714)
  [RelayMiner] Add session lifetime caching (#1713)
  Update recovery allowlist for recovery-8aug2025 (#1716)
  [Tokenomics] Validator proper decoding (#1710)
  [Session Rollovers] First attempt at solving session rollovers from RelayMiner's POV (#1705)
  [RelayMiner] Add comprehensive relay timing and gRPC call metrics (#1703)
  [Shared Module] Increase `ComputeUnitsPerRelayMax` (#1709)
  [RelayMiner] Improve non-2xx errors visibility, do not mine 5xx (#1695)
  [Governance] - Updated Params to match the ones in the forum  (#1707)
  [RelayMiner] Improve RelayMiner timeout handling and test reliability (#1691)
  [Morse Recovery] Updated Allowlist for Recovery (#1696)
  ...

# Conflicts:
#	tiltfiles/ibc.tilt
bryanchriswhite added a commit that referenced this pull request Aug 15, 2025
@bryanchriswhite
Merge branch 'refs/heads/chore/ibc-axelar' into docs/ibc/localnet
361cd50 
* refs/heads/chore/ibc-axelar: (36 commits)
  [GOVERNANCE][PARAMS][CUTTM] Updating global CUTTM per Foundation requirement (#1727)
  Update docusaurus/docs/4_develop/upgrades/4_upgrade_list.md w/ v0.1.28 MainNet block
  Updating bulk_params_main/tokenomics_params.json so proposer gets rewards again
  Update upgrade_tx_v0.1.28_main.json
  [Morse Recovery] Add recovery allowlist entries for issues #1718 and #1719 (#1723)
  [Upgrade]: Update v0.1.28 upgrade list docs (#1722)
  [Upgrade] v0.1.28 upgrade txs (#1721)
  [Upgrade] Add v0.1.28 upgrade plan (#1720)
  [CometBFT] Use Pocket's fork temporarily (#1717)
  Use use_cometbft_fork_v038.17 (#1714)
  [RelayMiner] Add session lifetime caching (#1713)
  Update recovery allowlist for recovery-8aug2025 (#1716)
  [Tokenomics] Validator proper decoding (#1710)
  [Session Rollovers] First attempt at solving session rollovers from RelayMiner's POV (#1705)
  [RelayMiner] Add comprehensive relay timing and gRPC call metrics (#1703)
  [Shared Module] Increase `ComputeUnitsPerRelayMax` (#1709)
  [RelayMiner] Improve non-2xx errors visibility, do not mine 5xx (#1695)
  [Governance] - Updated Params to match the ones in the forum  (#1707)
  [RelayMiner] Improve RelayMiner timeout handling and test reliability (#1691)
  [Morse Recovery] Updated Allowlist for Recovery (#1696)
  ...
bryanchriswhite added a commit that referenced this pull request Aug 15, 2025
@bryanchriswhite
Merge branch 'docs/ibc/localnet' into feat/ibc/packet-forward-middleware
76194c2 
* docs/ibc/localnet: (36 commits)
  [GOVERNANCE][PARAMS][CUTTM] Updating global CUTTM per Foundation requirement (#1727)
  Update docusaurus/docs/4_develop/upgrades/4_upgrade_list.md w/ v0.1.28 MainNet block
  Updating bulk_params_main/tokenomics_params.json so proposer gets rewards again
  Update upgrade_tx_v0.1.28_main.json
  [Morse Recovery] Add recovery allowlist entries for issues #1718 and #1719 (#1723)
  [Upgrade]: Update v0.1.28 upgrade list docs (#1722)
  [Upgrade] v0.1.28 upgrade txs (#1721)
  [Upgrade] Add v0.1.28 upgrade plan (#1720)
  [CometBFT] Use Pocket's fork temporarily (#1717)
  Use use_cometbft_fork_v038.17 (#1714)
  [RelayMiner] Add session lifetime caching (#1713)
  Update recovery allowlist for recovery-8aug2025 (#1716)
  [Tokenomics] Validator proper decoding (#1710)
  [Session Rollovers] First attempt at solving session rollovers from RelayMiner's POV (#1705)
  [RelayMiner] Add comprehensive relay timing and gRPC call metrics (#1703)
  [Shared Module] Increase `ComputeUnitsPerRelayMax` (#1709)
  [RelayMiner] Improve non-2xx errors visibility, do not mine 5xx (#1695)
  [Governance] - Updated Params to match the ones in the forum  (#1707)
  [RelayMiner] Improve RelayMiner timeout handling and test reliability (#1691)
  [Morse Recovery] Updated Allowlist for Recovery (#1696)
  ...
bryanchriswhite added a commit that referenced this pull request Aug 15, 2025
@bryanchriswhite
Merge branch 'feat/ibc/packet-forward-middleware' into chore/ibc-osmosis
b8bea0b 
* feat/ibc/packet-forward-middleware: (36 commits)
  [GOVERNANCE][PARAMS][CUTTM] Updating global CUTTM per Foundation requirement (#1727)
  Update docusaurus/docs/4_develop/upgrades/4_upgrade_list.md w/ v0.1.28 MainNet block
  Updating bulk_params_main/tokenomics_params.json so proposer gets rewards again
  Update upgrade_tx_v0.1.28_main.json
  [Morse Recovery] Add recovery allowlist entries for issues #1718 and #1719 (#1723)
  [Upgrade]: Update v0.1.28 upgrade list docs (#1722)
  [Upgrade] v0.1.28 upgrade txs (#1721)
  [Upgrade] Add v0.1.28 upgrade plan (#1720)
  [CometBFT] Use Pocket's fork temporarily (#1717)
  Use use_cometbft_fork_v038.17 (#1714)
  [RelayMiner] Add session lifetime caching (#1713)
  Update recovery allowlist for recovery-8aug2025 (#1716)
  [Tokenomics] Validator proper decoding (#1710)
  [Session Rollovers] First attempt at solving session rollovers from RelayMiner's POV (#1705)
  [RelayMiner] Add comprehensive relay timing and gRPC call metrics (#1703)
  [Shared Module] Increase `ComputeUnitsPerRelayMax` (#1709)
  [RelayMiner] Improve non-2xx errors visibility, do not mine 5xx (#1695)
  [Governance] - Updated Params to match the ones in the forum  (#1707)
  [RelayMiner] Improve RelayMiner timeout handling and test reliability (#1691)
  [Morse Recovery] Updated Allowlist for Recovery (#1696)
  ...

# Conflicts:
#	tiltfiles/ibc.tilt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Olshansk Olshansk Olshansk approved these changes

Assignees

@red-0ne red-0ne

Labels

devnet devnet-test-e2e miner Changes related to the Miner proxy Changes related to the Proxy push-image CI related - pushes images to ghcr.io relayminer Changes related to the Relayminer scalability

Projects

Shannon
Status: ✅ Done

Milestone

RelayMiner Stability

Development

Successfully merging this pull request may close these issues.

2 participants

@red-0ne @Olshansk