(msk-alpha): Add Kafka versions 2.8.2.tiered and support for tiered storage mode

[chrispidcock]

Describe the feature

AWS MSK has supported Tiered storage for almost a year now.
Support MSK Kafka versions

Add

• Enums for Kafka versions 3.5.1 and 2.8.2.tiered
• Support in Kafka construct for the CloudFormation StorageMode paramater

Tiered Storage

Using such feature requires both upgrading the cluster to 2.8.2.tiered version and enabling TIERED storage in the cluster properties (as per the official docs), but the latter is not yet possible via @aws-cdk_aws-msk-alpha.Cluster class.

Use Case

Our team is currently using AWS CDK for IaC for a MSK implementation. The lack of support for tiered storage is something we can hopefully add very soon.

Ideally, we can use define methods in the CDK library rather than applying work arounds to enable tiered storage.

Adding support for tiered storage to the CDK will simplify the process of:

• creating new MSK clusters.
• updating an existing cluster to use tiered storage.

Proposed Solution

• Add new versions to the KafkaVersion class
• Add support for the StorageMode parameter, already present in AWS::MSK::Cluster properties.
• Add validation to ensure cluster compatibility with selected storage mode, as per aws docs: Tiered storage
• Update tests for new versions and parameters
• Update snapshots as required for the StorageMode parameter

Other Information

Related Issues:

• Mention of adding support for Amazon MSK tiered storage version 2.8.2.tiered however due to the additional requirements, was left out.

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.91.0

Environment details (OS name and version, etc.)

Debian GNU/Linux (Docker image)

[pahud]

Awesome! Thank you for your PR.

[chrispidcock]

Hello, @pahud , or any maintainer of this project,

Our team is actively building an EDA streaming service, with aws MSK and aws-cdk as the core of the infrastructure.

I have raised (to my knowledge) a "maintainer review ready PR" #27560, and am wondering if there is anything remaining that I can do to continue its movement?

Thanks!