Figure out airflow 2 to 3 database migration path

[shelleydoljack]

From the documentation it is suggested to run airflow db clean on your database before migrating to airflow 3. Should we add a deploy task that does airflow db clean whenever we deploy or just add it as a task to be run whenever?

According to the docs we should also run airflow dags reserialize on our old airflow instance prior to upgrading to airflow 3.

In the airflow-init section for docker compose, we can add _AIRFLOW_DB_MIGRATE: 'true' https://github.com/sul-dlss/rialto-airflow/blob/e80a965de3631da3999d2fa33d40dbe64428adb9/compose.prod.yaml#L258 in order to apply the database changes for airflow 3 when upgrading.

[shelleydoljack]

We plan to run Airflow 3 with postgres 16. We want new database servers for stage and prod and plan to get a copy of the data from the old db servers to the new db servers and run the db clean, dags reserialize, and db migration.

[shelleydoljack]

New db servers requested: https://github.com/sul-dlss/operations-tasks/issues/4412