[Proposal] Writable Warm Status API Model

[e-emoto]

Is your feature request related to a problem? Please describe

The Status API in Writable Warm will be for listing the in-progress and failed index tierings. Since Writable Warm will eventually support other types of tierings for both dedicated and non-dedicated warm node clusters, the API needs to be extensible to cover those cases. The part of the design below describes the API model for the Status API, and the design for the rest of the API will be part of a follow-up task once some details about tiering metadata are figured out.

Describe the solution you'd like

API Model:

The API will use a source and target as input to filter which tierings are shown. It will validate that both inputs are valid tiers, and then use them to find any tierings that match the described type. The API should still work if only one of the source or target is given, and will find any tierings with that input, allowing for more flexible queries. In the default case if no source or target is given as input, the status API should return all in progress or failed tierings for the specified indices, regardless of the tiering change.

API Request:

GET /<indexNameOrPattern>/_tier?source=hot&target=warm
GET /<indexNameOrPattern>/_tier?status=ongoing
GET /<indexNameOrPattern>/_tier?verbose=true/false

The API would be a get request that has a few parameters. The index name in the path will be required, but can support using ‘_all’ or ‘*’ to get migrations from all indices that match the parameters. The API will also support comma separated index names.

API Parameters:

source = hot / warm (optional, no default)
target = hot / warm (optional, no default)

The values for the source and target parameters are the tiers, with source being the tier the index started in and target being the tier it is moving to.

status = failed / ongoing (optional, no default)

The values of the status parameter represent the state of the tiering. failed indicates that the tiering has failed and ongoing means the tiering process is in progress.

verbose = true / false (default false)

The verbose parameter determines whether the API response should include details like the shard relocation status, failure reason, and tiering start time.

API Response:

Success:

{
    "tiering_status" : [{
        "index" : "test1"
        "source": "hot",
        "target": "warm",
        "status" : "failed/ongoing",
    }]
}

With Verbose Flag:

{
    "tiering_status" : [{
        "index" : "test1"
        "source": "hot",
        "destination": "warm",
        "status" : "ongoing",
        "start_time" : "2024-06-27T00:00:00Z",
        "failure_time" : "2024-06-27T10:00:00Z",
        "duration" : "10:00:00",
        "shards" : {
                "total" : 10, 
                "successful" : 3, 
                "failed" : 2, 
                "ongoing" : 5,
            },
        "ongoing" : [{
                shard_id: 3, 
                node_id: "", 
                reason: ""
            }, 
            ... 
        ],
        "failed" : [{
                shard_id: 1, 
                node_id: "", 
                reason: ""
            }, 
            ... 
        ],
    }]
}

Failure:

{
    "error": {
        "root_cause": [
            {
                "type": "",
                "reason": "",
            }
        ],
    },
    "status": xxx
}

Example API Use Cases:

Get All Ongoing Tierings:

GET /_all/_tier?status=ongoing

{
    "tiering_status" : [{
        "index" : "test1"
        "source": "hot",
        "target": "warm",
        "status" : "ongoing",
    },{
        "index" : "test2"
        "source": "warm",
        "target": "hot",
        "status" : "ongoing",
    },
    
    ...
    
    ]
}

Get All Failed Hot To Warm Tierings:

GET /_all/_tier?source=hot&target=warm&status=failed

{
    "tiering_status" : [{
        "index" : "test3"
        "source": "hot",
        "target": "warm",
        "status" : "failed",
    },{
        "index" : "test4"
        "source": "hot",
        "target": "warm",
        "status" : "failed",
    },
    
    ...
    
    ]
}

Get Shard Details for a Specific Index Tiering:

GET /target_index/_tier?verbose=true

{
    "tiering_status" : [{
        "index" : "target_index"
        "source": "hot",
        "target": "warm",
        "status" : "ongoing",
        "start_time" : "2024-06-27T00:00:00Z",
        "failure_time" : "",
        "duration" : "",
        "shards" : {
                "total" : 10, 
                "successful" : 4, 
                "failed" : 0, 
                "ongoing" : 6,
            },
        "ongoing" : [{
                shard_id: 3, 
                node_id: "", 
                reason: ""
            }, 
            ... 
        ],
        "failed" : [],
    }]
}

Related component

Search:Remote Search

Describe alternatives you've considered

No response

Additional context

This issue is for getting feedback on the API structure, and will be followed up with a PR for the API spec and a low level design description.

Related issues:
#14679
#13294

[e-emoto]

@andrross @dblock @mch2 @ankitkala @rayshrey Any feedback on this would be greatly appreciated, thanks!