Remove sla_miss_callback from dag_kwargs for Airflow >= 3.1.0

[jroachgolf84]

I was notified of the following issue. This PR aims to fix that:

This might be expected, but I wanted to check if you're aware of it. I tried dag-factory 1.0.0 with the latest Airflow 3.1 nightly :airflow_spin:.

Even a simple DAG definition fails with:

TypeError: DAG.__init__() got an unexpected keyword argument 'sla_miss_callback'
The issue is that dagbuilder.py sets the parameter:
dag_kwargs["sla_miss_callback"] = dag_params.get("sla_miss_callback", None)

every time a DAG is instantiated. The parameter was deprecated in 3.0 and removed in 3.1. I assume this will be addressed once 3.1 is released, but I couldn't find an open issue yet, so I wanted to flag it.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 93.86%. Comparing base (b4ceb21) to head (e12ab5c).
⚠️ Report is 52 commits behind head on main.

Files with missing lines	Patch %	Lines
dagfactory/dagbuilder.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #586      +/-   ##
==========================================
- Coverage   93.93%   93.86%   -0.08%     
==========================================
  Files          13       13              
  Lines        1121     1124       +3     
==========================================
+ Hits         1053     1055       +2     
- Misses         68       69       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tatiana]

Thanks for fixing this, @jroachgolf84 ! I'll wait to merge when we are in a IST friendly time so @pankajastro can review it before we merge.

[pankajastro]

LGTM

[pankajkoti]

LGTM. But @pankajastro, I think we need a fix on how we evaluate and set dag_kwargs. If the user is not setting sla_miss_callback and other such params in the YAML, I believe we should not add it to the kwargs at all. If the parameters exist in the Airflow DAG init, let Airflow set the default values instead of us setting the defaults to None.

[pankajastro]

LGTM. But @pankajastro, I think we need a fix on how we evaluate and set dag_kwargs. If the user is not setting sla_miss_callback and other such params in the YAML, I believe we should not add it to the kwargs at all. If the parameters exist in the Airflow DAG init, let Airflow set the default values instead of us setting the defaults to None.

I agree that we should not add a default value in this lib for DAG params, but I feel this is out of scope for this PR. I have created an issue to track it: #587