Skip to content

[FEATURE] Implement API Consistency Tests for Remaining Workflow Steps #908

@junweid62

Description

@junweid62

Is your feature request related to a problem?

Currently, only the agent step API consistency test has been completed. There are additional workflow steps that require similar consistency checks to ensure that the API specifications match the required enum parameters. Without these tests, we cannot guarantee that future changes won't break the consistency across other workflow steps.

What solution would you like?

Follow #900 to extend the API consistency test implementation for other workflow steps.

Steps:

  1. Use ApiSpecFetcher to fetch the OpenAPI specification:
    Leverage the ApiSpecFetcher utility introduced in Add ApiSpecFetcher for Fetching and Comparing API Specifications #900 to programmatically fetch the OpenAPI specification for each workflow step.

  2. Utilize the compareRequiredFields method:
    Use the compareRequiredFields method from ApiSpecFetcher to compare the required input fields from the API specification with the predefined enum values for each workflow step.

  3. Integrate ApiConsistencyTest into each step's unit tests:
    For each workflow step (e.g., registerAgent, registerModel), ensure an API consistency check is added in the respective unit tests to validate the correctness of required API parameters.

  4. Add a GitHub Action for daily workflow test execution:
    Once all steps are completed, set up a GitHub Action to trigger a daily run of the workflow tests to ensure API consistency over time.

Do you have any additional context?

The agent step API consistency test has already been implemented. The same approach can be applied to the remaining steps to ensure all workflow steps are covered. This issue is important to maintain consistency in our API as it evolves.

CheckList

ML-Commons Related

  • CreateConnectorStep

DeleteAgentStep(No request body)
DeleteModelStep(No request body)
DeployModelStep(No request body)

  • RegisterAgentStep
  • RegisterLocalCustomModelStep
  • RegisterLocalPretrainedModelStep
  • RegisterLocalSparseEncodingModelStep
  • RegisterModelGroupStep
  • RegisterRemoteModelStep
    UndeployModelStep(No request body)

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions