Skip to content

Adding fetcher for plugin to pipeline mapping.#15970

Merged
ritwiksahani merged 1 commit intosimplified-upgradesfrom
simplified-upgrades-list-API
Jun 10, 2025
Merged

Adding fetcher for plugin to pipeline mapping.#15970
ritwiksahani merged 1 commit intosimplified-upgradesfrom
simplified-upgrades-list-API

Conversation

@ritwiksahani
Copy link
Contributor

@ritwiksahani ritwiksahani commented Jun 9, 2025

  • Added a metadata fetcher to fetch mapping from CDAP metadata.

Sample response from metadata API for a user plugin Trash and the pipeline DataFusionQuickstart

{
  "sort": "io.cdap.cdap.data2.metadata.dataset.SortInfo@74a24bc1",
  "offset": 0,
  "limit": 2147483647,
  "numCursors": 0,
  "total": 1,
  "results": [
    {
      "metadataEntity": {
        "details": {
          "namespace": "default",
          "artifact": "trash-plugin",
          "version": "1.2.0",
          "type": "batchsink",
          "plugin": "Trash"
        },
        "type": "plugin"
      },
      "metadata": {
        "SYSTEM": {
          "properties": {
            "default:DataFusionQuickstart": "1"
          },
          "tags": []
        }
      }
    }
  ],
  "cursors": [],
  "showHidden": false,
  "entityScope": [
    "USER"
  ]
}

Copy link
Contributor

@sidhdirenge sidhdirenge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a sample response from metadata api in the PR description?

@ritwiksahani
Copy link
Contributor Author

Should we add a sample response from metadata api in the PR description?

Added.

* Added a metadata fetcher to fetch mapping from CDAP metadata.

Sample response from metadata API for a user plugin `Trash` and the pipeline `DataFusionQuickstart`
```
{
  "sort": "io.cdap.cdap.data2.metadata.dataset.SortInfo@74a24bc1",
  "offset": 0,
  "limit": 2147483647,
  "numCursors": 0,
  "total": 1,
  "results": [
    {
      "metadataEntity": {
        "details": {
          "namespace": "default",
          "artifact": "trash-plugin",
          "version": "1.2.0",
          "type": "batchsink",
          "plugin": "Trash"
        },
        "type": "plugin"
      },
      "metadata": {
        "SYSTEM": {
          "properties": {
            "default:DataFusionQuickstart": "1"
          },
          "tags": []
        }
      }
    }
  ],
  "cursors": [],
  "showHidden": false,
  "entityScope": [
    "USER"
  ]
}
```
@ritwiksahani ritwiksahani force-pushed the simplified-upgrades-list-API branch from 20fb886 to ec0c17a Compare June 9, 2025 14:01
@ritwiksahani ritwiksahani merged commit 5c077de into simplified-upgrades Jun 10, 2025
1 check passed
@ritwiksahani ritwiksahani deleted the simplified-upgrades-list-API branch June 10, 2025 05:18
ritwiksahani added a commit that referenced this pull request Jun 20, 2025
Adding fetcher for plugin to pipeline mapping.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants