Make dag_version_id in TI non-nullable#50825
Merged
ephraimbuddy merged 39 commits intoapache:mainfrom Jul 8, 2025
Merged
Conversation
ephraimbuddy
requested review from
XD-DENG,
amoghrajesh,
ashb,
kaxil and
uranusjr
as code owners
boring-cyborg
bot
added
area:API
ephraimbuddy
force-pushed
the
make-ti-dag-version-id-non-nullable2
branch
from
20c6e75 to
08bfa28
Compare
ephraimbuddy
requested review from
hussein-awala and
jedcunningham
as code owners
ephraimbuddy
force-pushed
the
make-ti-dag-version-id-non-nullable2
branch
from
9d3fc74 to
63eb72b
Compare
ephraimbuddy
force-pushed
the
make-ti-dag-version-id-non-nullable2
branch
2 times, most recently
from
aad0a5f to
80439a1
Compare
uranusjr
reviewed
May 22, 2025
uranusjr
reviewed
May 22, 2025
ephraimbuddy
force-pushed
the
make-ti-dag-version-id-non-nullable2
branch
2 times, most recently
from
5409839 to
616e0a1
Compare
ephraimbuddy
requested review from
gopidesupavan,
jason810496 and
potiuk
as code owners
ephraimbuddy
force-pushed
the
make-ti-dag-version-id-non-nullable2
branch
from
616e0a1 to
070f7a3
Compare
ephraimbuddy
force-pushed
the
make-ti-dag-version-id-non-nullable2
branch
from
564ac88 to
f3c6444
Compare
HsiuChuanHsu
pushed a commit
to HsiuChuanHsu/airflow
that referenced
this pull request
Jul 10, 2025
* Make dag_version_id in TI non-nullable Enforces proper serialization of DAGs before creating DagRuns and TaskInstances. The changes include: - Making dag_version_id a required parameter in TaskInstance constructor - Migrating the dag_version_id column in TaskInstance to be non-nullable - Adding a check in _create_orm_dagrun to raise AirflowException if dag is not serialized - Updating tests to ensure DAGs are properly serialized before creating DagRuns This prevents running runs with missing serialized DAG versions. * fixup! Make dag_version_id in TI non-nullable * Fix tests for some providers * fixup! Fix tests for some providers * fixup! fixup! Fix tests for some providers * more test fixes * fixup! more test fixes * fixup! fixup! more test fixes * Update more tests * fixup! Update more tests * fixup! fixup! Update more tests * fixup! fixup! fixup! Update more tests * fixup! fixup! fixup! fixup! Update more tests * fixup! fixup! fixup! fixup! fixup! Update more tests * Add compat for MockedTaskInstance * fixup! Add compat for MockedTaskInstance * fixup! fixup! Add compat for MockedTaskInstance * fix system test * fixup! fix system test * A fix for kube integration tests * Update the expected TIs from test multiple versions number file * fix incorrect imports * fix time delta async test * Remove non working kube test fix * Fix task sdk test * Add cadwyn migration and fix some tests * fix some provider tests * Add data migration before migrating dag_version_id * Update test, TaskInstanceResponse and fix assert dag_version in task_command.py * Resolve conflicts * Fix test * Fix more tests * fixup! fixup! Fix more tests * Fix bad rebase * update fab www-hash * Fix conflicts * Fix MappedTaskInstancedata * add dag_version_id to a new test * Fix fab www hash
stephen-bracken
pushed a commit
to stephen-bracken/airflow
that referenced
this pull request
Jul 15, 2025
* Make dag_version_id in TI non-nullable Enforces proper serialization of DAGs before creating DagRuns and TaskInstances. The changes include: - Making dag_version_id a required parameter in TaskInstance constructor - Migrating the dag_version_id column in TaskInstance to be non-nullable - Adding a check in _create_orm_dagrun to raise AirflowException if dag is not serialized - Updating tests to ensure DAGs are properly serialized before creating DagRuns This prevents running runs with missing serialized DAG versions. * fixup! Make dag_version_id in TI non-nullable * Fix tests for some providers * fixup! Fix tests for some providers * fixup! fixup! Fix tests for some providers * more test fixes * fixup! more test fixes * fixup! fixup! more test fixes * Update more tests * fixup! Update more tests * fixup! fixup! Update more tests * fixup! fixup! fixup! Update more tests * fixup! fixup! fixup! fixup! Update more tests * fixup! fixup! fixup! fixup! fixup! Update more tests * Add compat for MockedTaskInstance * fixup! Add compat for MockedTaskInstance * fixup! fixup! Add compat for MockedTaskInstance * fix system test * fixup! fix system test * A fix for kube integration tests * Update the expected TIs from test multiple versions number file * fix incorrect imports * fix time delta async test * Remove non working kube test fix * Fix task sdk test * Add cadwyn migration and fix some tests * fix some provider tests * Add data migration before migrating dag_version_id * Update test, TaskInstanceResponse and fix assert dag_version in task_command.py * Resolve conflicts * Fix test * Fix more tests * fixup! fixup! Fix more tests * Fix bad rebase * update fab www-hash * Fix conflicts * Fix MappedTaskInstancedata * add dag_version_id to a new test * Fix fab www hash
kaxil
reviewed
Jul 23, 2025
Comment on lines
+280
to
+282
| dag = DagBag().get_dag("test_mapped_classic") | ||
| dag.sync_to_db() | ||
| SerializedDagModel.write_dag(dag, bundle_name="testing") |
Member
There was a problem hiding this comment.
This has introduced a bug. This was the clue where we had to change the test
ephraimbuddy
added a commit
to astronomer/airflow
that referenced
this pull request
Aug 11, 2025
This is alternative to apache#53820. Here we make the TI.dag_version_id nullable on the database level. it's still enforced in code
1 task
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Enforces serialization of DAGs before creating DagRuns and TaskInstances.
The changes include:
This prevents running runs with missing serialized DAG versions.