Skip to content

feat(telemetry): add specific PR, issue, and custom tracking IDs for GitHub Actions#21129

Merged
cocosheng-g merged 6 commits intogoogle-gemini:mainfrom
cocosheng-g:feat/add-github-action-telemetry-metrics
Mar 10, 2026
Merged

feat(telemetry): add specific PR, issue, and custom tracking IDs for GitHub Actions#21129
cocosheng-g merged 6 commits intogoogle-gemini:mainfrom
cocosheng-g:feat/add-github-action-telemetry-metrics

Conversation

@cocosheng-g
Copy link
Contributor

@cocosheng-g cocosheng-g commented Mar 4, 2026

Summary

This PR adds new telemetry attributes to the Gemini CLI to support granular usage metrics for GitHub Actions. These metrics will allow us to track unique Issues and PRs touched by the CLI, and distinguish between different workflow triggers (e.g., Automated vs. Scheduled triage).

Will fix #21130

Details

  • Added GEMINI_CLI_GH_EVENT_NAME to EventMetadataKey.
  • Added specific tracking IDs: GEMINI_CLI_GH_PR_NUMBER, GEMINI_CLI_GH_ISSUE_NUMBER, and GEMINI_CLI_GH_CUSTOM_TRACKING_ID replacing generic GH_EVENT_NUMBER.
  • Updated ClearcutLogger to capture these from environment variables (GITHUB_EVENT_NAME, GH_PR_NUMBER, GH_ISSUE_NUMBER, and GH_CUSTOM_TRACKING_ID).
  • The PR/Issue number is logged as a raw string ID (e.g., 123) to avoid fingerprinting PII, relying on the repository context to provide global uniqueness for backend queries.
  • Updated telemetry tests in clearcut-logger.test.ts to cover the new environment variables and metadata keys.
  • Updated telemetry documentation in docs/cli/telemetry.md.

Related Issues

Related to internal PM request for "Gemini CLI GitHub Action Usage Metric".

How to Validate

  1. Run the new tests: npm test -w @google/gemini-cli-core -- src/telemetry/clearcut-logger/clearcut-logger.test.ts
  2. Verify that when GITHUB_EVENT_NAME and specific IDs (GH_PR_NUMBER, etc.) are set in the environment, they appear in the Clearcut log metadata as expected.

Example events

Sample 1: api_request (Automated Issue Triage)
Triggered when the CLI sends a prompt to the model during an "Issue Opened" event.

{
  "event_name": "api_request",
  "console_type": "GEMINI_CLI",
  "application": 102,
  "event_metadata": [
    [
      { "gemini_cli_key": 20,  "value": "gemini-1.5-pro" },         // API_REQUEST_MODEL
      { "gemini_cli_key": 39,  "value": "GitHub" },                 // SURFACE
      { "gemini_cli_key": 130, "value": "gemini-triage" },          // GH_WORKFLOW_NAME
      { "gemini_cli_key": 132, "value": "a1b2c3d4..." },            // GH_REPO_HASH
      { "gemini_cli_key": 176, "value": "issues" },                 // GH_EVENT_NAME
      { "gemini_cli_key": 178, "value": "42" }                      // GH_ISSUE_NUMBER (Issue #42)
    ]
  ]
}

Sample 2: api_response (PR Review)
Triggered when the model returns a response for a Pull Request review.

{
  "event_name": "api_response",
  "console_type": "GEMINI_CLI",
  "application": 102,
  "event_metadata": [
    [
      { "gemini_cli_key": 21,  "value": "gemini-1.5-pro" },         // API_RESPONSE_MODEL
      { "gemini_cli_key": 25,  "value": "1240" },                   // INPUT_TOKEN_COUNT
      { "gemini_cli_key": 26,  "value": "450" },                    // OUTPUT_TOKEN_COUNT
      { "gemini_cli_key": 130, "value": "gemini-review" },          // GH_WORKFLOW_NAME
      { "gemini_cli_key": 176, "value": "pull_request" },           // GH_EVENT_NAME
      { "gemini_cli_key": 177, "value": "101" }                     // GH_PR_NUMBER (PR #101)
    ]
  ]
}

Sample 3: start_session (Scheduled Triage)
Triggered when the CLI starts up as part of a cron job (Scheduled).

{
  "event_name": "start_session",
  "console_type": "GEMINI_CLI",
  "application": 102,
  "event_metadata": [
    [
      { "gemini_cli_key": 1,   "value": "gemini-1.5-flash" },       // START_SESSION_MODEL
      { "gemini_cli_key": 130, "value": "gemini-issue-fixer" },     // GH_WORKFLOW_NAME
      { "gemini_cli_key": 176, "value": "schedule" },               // GH_EVENT_NAME
      { "gemini_cli_key": 179, "value": "101,102,103" }             // GH_CUSTOM_TRACKING_ID 
    ]
  ]
}

Pre-Merge Checklist

  • Updated relevant documentation and README (if needed)
  • Added/updated tests (if needed)
  • Noted breaking changes (if any)
  • Validated on required platforms/methods:
    • MacOS
    • Windows
    • Linux

@cocosheng-g cocosheng-g requested review from a team as code owners March 4, 2026 18:02
@gemini-code-assist

This comment was marked as outdated.

@cocosheng-g cocosheng-g self-assigned this Mar 4, 2026
gemini-code-assist[bot]

This comment was marked as outdated.

@gemini-cli gemini-cli bot added the area/enterprise Issues related to Telemetry, Policy, Quota / Licensing label Mar 4, 2026
@cocosheng-g cocosheng-g enabled auto-merge March 4, 2026 18:45
@cocosheng-g cocosheng-g requested a review from kschaab March 4, 2026 18:46
@cocosheng-g cocosheng-g changed the title feat(telemetry): add github action event name and hashed event number feat(telemetry): add github action event name and event number Mar 4, 2026
@cocosheng-g cocosheng-g changed the title feat(telemetry): add github action event name and event number feat(telemetry): add specific PR, issue, and custom tracking IDs for GitHub Actions Mar 6, 2026
@cocosheng-g cocosheng-g force-pushed the feat/add-github-action-telemetry-metrics branch from a8541db to 7a621a9 Compare March 6, 2026 17:55
…, issue, and custom tracking ids

- Replaced `GEMINI_CLI_GH_EVENT_NUMBER` with `GEMINI_CLI_GH_PR_NUMBER`, `GEMINI_CLI_GH_ISSUE_NUMBER`, and `GEMINI_CLI_GH_CUSTOM_TRACKING_ID` to allow tracking of specific GitHub Action contexts.
- Updated `ClearcutLogger` to extract `GH_PR_NUMBER`, `GH_ISSUE_NUMBER`, and `GH_CUSTOM_TRACKING_ID` from the environment.
- Updated telemetry unit tests to verify the new environment variables and metadata keys.
- Updated telemetry documentation in `docs/cli/telemetry.md` to reflect the new attributes.
@cocosheng-g cocosheng-g force-pushed the feat/add-github-action-telemetry-metrics branch from 7a621a9 to 8fcf4c2 Compare March 6, 2026 17:57
@cocosheng-g cocosheng-g added this pull request to the merge queue Mar 10, 2026
Merged via the queue into google-gemini:main with commit e91f86c Mar 10, 2026
26 of 27 checks passed
@cocosheng-g cocosheng-g deleted the feat/add-github-action-telemetry-metrics branch March 10, 2026 15:22
PrasannaPal21 pushed a commit to PrasannaPal21/gemini-cli that referenced this pull request Mar 10, 2026
cocosheng-g added a commit to google-github-actions/run-gemini-cli that referenced this pull request Mar 11, 2026
## Description

This PR implements the necessary infrastructure in the
\`run-gemini-cli\` GitHub Action to support the Gemini CLI usage metrics
(P0 requirements).

It aligns with the core CLI telemetry changes introduced in
[google-gemini/gemini-cli#21129](google-gemini/gemini-cli#21129).

### Changes Made:
* **Strongly-Typed Telemetry Tracking:** Replaced the generic
\`github_item_number\` with three specific inputs: \`github_pr_number\`,
\`github_issue_number\`, and \`github_custom_tracking_id\`. This ensures
that Pull Requests, Issues, and custom batch IDs are logged to distinct
fields, providing maximum accuracy for downstream analytics without
overlapping ID spaces.
* **Installation Telemetry:** Added \`GH_WORKFLOW_NAME\`,
\`GH_PR_NUMBER\`, \`GH_ISSUE_NUMBER\`, and \`GH_CUSTOM_TRACKING_ID\` to
the \`Install Gemini CLI\` step to ensure extension installations are
properly tracked.
* **OTEL Collector Enhancement:** Updated the
\`scripts/collector-gcp.yaml.template\` to include specific resource
attributes for \`github.pr.number\`, \`github.issue.number\`, and
\`github.custom_tracking.id\`.
* **Example Workflow Updates:** Updated all example workflows (Review,
Triage, Assistant, Scheduled Triage) to explicitly pass the correct IDs
to the action, improving clarity and ensuring accurate telemetry.

### Example Telemetry Scenarios

Because the CLI appends these context fields to the \`baseMetadata\` of
all events (API Request, Response, Error), downstream analytics can
perfectly distinguish and count unique issues vs. PRs based on the
specific fields.

#### Scenario A: Automated PR Review
When PR 42 triggers the \`gemini-review\` workflow, the CLI logs:
```json
{
  \"event_name\": \"api_request\",
  \"event_metadata\": [
    [
      { \"gemini_cli_key\": 130, \"value\": \"gemini-review\" },
      { \"gemini_cli_key\": 172, \"value\": \"pull_request\" },
      { \"gemini_cli_key\": 173, \"value\": \"42\" } // GH_PR_NUMBER
    ]
  ]
}
```
*Analysts can count \`DISTINCT gh_pr_number WHERE gh_workflow_name LIKE
'%review%'\` to satisfy the P0 PR metric.*

#### Scenario B: Automated Issue Triage
When Issue 88 triggers the \`gemini-triage\` workflow, the CLI logs:
```json
{
  \"event_name\": \"api_request\",
  \"event_metadata\": [
    [
      { \"gemini_cli_key\": 130, \"value\": \"gemini-triage\" },
      { \"gemini_cli_key\": 172, \"value\": \"issues\" },
      { \"gemini_cli_key\": 174, \"value\": \"88\" } // GH_ISSUE_NUMBER
    ]
  ]
}
```
*Analysts can count \`DISTINCT gh_issue_number WHERE gh_workflow_name
LIKE '%triage%' AND gh_event_name = 'issues'\` to satisfy the P0
Automated Issue metric.*

#### Scenario C: Scheduled Batch Triage
When a cron job runs and triages 3 issues (IDs 101, 102, 103) in a
single CLI invocation, the CLI logs:
```json
{
  \"event_name\": \"api_request\",
  \"event_metadata\": [
    [
      { \"gemini_cli_key\": 130, \"value\": \"gemini-scheduled-triage\" },
      { \"gemini_cli_key\": 172, \"value\": \"schedule\" },
      { \"gemini_cli_key\": 175, \"value\": \"101,102,103\" } // GH_CUSTOM_TRACKING_ID
    ]
  ]
}
```
*Analysts can split the \`gh_custom_tracking_id\` string by commas to
count exactly 3 unique issues processed during a scheduled invocation.*

## Related Issues
* Fixes P0 metrics requirements for GitHub Action usage.
* Depends on
[google-gemini/gemini-cli#21129](google-gemini/gemini-cli#21129)
kunal-10-cloud pushed a commit to kunal-10-cloud/gemini-cli that referenced this pull request Mar 12, 2026
liamhelmer pushed a commit to badal-io/gemini-cli that referenced this pull request Mar 12, 2026
DavidAPierce pushed a commit that referenced this pull request Mar 16, 2026
yashodipmore pushed a commit to yashodipmore/geemi-cli that referenced this pull request Mar 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/enterprise Issues related to Telemetry, Policy, Quota / Licensing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: Add granular GitHub Action usage metrics to telemetry

2 participants