[Feature] Include the Apache DolphinScheduler on Amazon EKS solution into unified repository maintenance

[SEZ9]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

What is the outcome that you are trying to reach?

1. Apache DolphinScheduler is an Apache top-level project. It's a distributed and extensible open-source workflow orchestration platform with powerful visual DAG interfaces, offering convenience in visual organization. It has a broad global AWS customer base.
2. A more efficient architecture and performance. It natively supports the visual invocation of various AWS components, including data services like Redshift, Athena, Glue, DMS, and EMR.
3. By integrating with AWS EKS, Aurora database, and others, it provides a more reliable and scalable architecture. Of course, many customers are already using Apache DolphinScheduler on AWS EKS.

Describe the solution you would like

A Terraform script has been created by AWS SA to deploy Apache DolphinScheduler to EKS with a single click. It supports EFS persistence, remote Task log writing to S3, and Aurora Serverless database for metadata storage .

Describe alternatives you have considered

Currently, Airflow does not support visual task orchestration, and its integration with AWS native services is not as convenient as Apache DolphinScheduler. Therefore, integrating Apache DolphinScheduler is highly necessary.

Additional context

In the past, we have had close collaboration with the Apache DolphinScheduler open-source community, publishing multiple joint blogs, including technical best practices and customer case studies.

1. How to Integrate Apache DolphinScheduler with AWS EMR&Redshift
2. Using Apache DolphinScheduler for Amazon EMR Job Scheduling
3. Using Amazon EMR Serverless, Athena, and Dolphin Scheduler to build an on-cloud and off-cloud data synchronization solution
4. Deploy the serverless Apache DolphinScheduler task scheduling system on AWS