Feat: add strict mode flag#719
Closed
corsettigyg wants to merge 42 commits into
Closed
Conversation
…ict mode propagates Agent-Logs-Url: https://github.com/corsettigyg/dag-factory/sessions/e0f692d9-6c32-43a1-9bc9-e13d8e1198a4 Co-authored-by: corsettigyg <155465603+corsettigyg@users.noreply.github.com>
…dant shutil import in test Agent-Logs-Url: https://github.com/corsettigyg/dag-factory/sessions/cddefdd5-a8c1-4a48-9e7d-d4c359a5b21e Co-authored-by: corsettigyg <155465603+corsettigyg@users.noreply.github.com>
… -> "for all", trailing whitespace) Agent-Logs-Url: https://github.com/corsettigyg/dag-factory/sessions/64f4c0ad-8e7b-403c-b9d1-bcfad67d66fd Co-authored-by: corsettigyg <155465603+corsettigyg@users.noreply.github.com>
corsettigyg
requested review from
pankajkoti and
tatiana
and removed request for
a team
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a strict_mode configuration flag to make DAG Factory surface YAML DAG build/parsing failures as exceptions (instead of silently skipping), improving CI/CD visibility and troubleshooting.
Changes:
- Introduces
dag_factory.strict_mode(Airflow config/env) and wires it into DAG generation to optionally raise on failures. - Updates DAG building to isolate failures per DAG (continue building others) while tracking failed DAG names.
- Expands test coverage and documentation for strict vs non-strict behavior, including directory-scan behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
dagfactory/dagfactory.py |
Tracks per-DAG failures and raises DagFactoryConfigException in strict mode; aggregates strict errors during directory scanning. |
dagfactory/settings.py |
Adds strict_mode setting read from Airflow config. |
tests/test_dagfactory.py |
Adds/updates tests for strict and non-strict behavior and adjusts for build_dags() return type change. |
docs/configuration/environment_variables.md |
Documents the new strict_mode setting and environment variable. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
corsettigyg
force-pushed
the
strict-mode
branch
from
2825419 to
715cdcd
Compare
pankajastro
reviewed
May 4, 2026
Contributor
There was a problem hiding this comment.
@corsettigyg, thanks for working on this. Do we need to make changes to the build_dags() and _generate_dags() functions? Since we encourage users to use the load_yaml_dags function for DAG authoring, couldn’t the strict check logic live only in that function? Also, we’re planning to release a new version in a couple of days, so if we can wrap up the review feedback, we can include it in the upcoming release.
|
|
||
| def build_dags(self) -> Dict[str, DAG]: | ||
| """Build DAGs using the config file.""" | ||
| def build_dags(self) -> Tuple[Dict[str, DAG], List[Tuple[str, Exception]]]: |
Contributor
There was a problem hiding this comment.
I would call it a breaking change (the return type change). We should document it in the changelog.
Contributor
Author
There was a problem hiding this comment.
it is inside a private method tho _DagFactory. I'd expect users to not import from private methods
Comment on lines
+244
to
+259
| try: | ||
| # Apply DAG-level default arguments from global_default_args to each dag_config, | ||
| # this is helpful because some arguments are not supported in default_args. | ||
| if isinstance(global_default_args, dict): | ||
| dag_config = {**dag_level_args, **dag_config} | ||
|
|
||
| dag_config["task_groups"] = dag_config.get("task_groups", {}) | ||
| dag_builder: DagBuilder = DagBuilder( | ||
| dag_name=dag_name, | ||
| dag_config=dag_config, | ||
| default_config=default_config, | ||
| yml_dag=self._serialise_config_md(dag_name, dag_config, default_config), | ||
| ) | ||
| dag: Dict[str, Union[str, DAG]] = dag_builder.build() | ||
| dags[dag["dag_id"]]: DAG = dag["dag"] | ||
| except Exception as exc: # pylint: disable=broad-except |
Contributor
There was a problem hiding this comment.
Because of the try/except block, if a user was directly using the method, it will no longer raise the error and will instead return it. We may need to document this or reduce its scope of how we are propagating strict mode error.
| # original exception chain available via __cause__ for debugging. | ||
| wrapped = DagFactoryConfigException(f"Failed to load dag config from '{config_file_abs_path}': {e}") | ||
| wrapped.__cause__ = e | ||
| strict_errors.append(wrapped) |
Contributor
There was a problem hiding this comment.
why are you appending error here without checking strict_errors?
| if settings.strict_mode and build_errors: | ||
| details = "; ".join(f"{name}: {exc}" for name, exc in build_errors) | ||
| # Chain the first exception so Airflow's UI shows its traceback as __cause__. | ||
| first_error = build_errors[0][1] |
Contributor
There was a problem hiding this comment.
why are you storing all error and raising only the first error's traceback
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 4.35.1 to 4.35.2. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/github/codeql-action/releases">github/codeql-action's releases</a>.</em></p> <blockquote> <h2>v4.35.2</h2> <ul> <li>The undocumented TRAP cache cleanup feature that could be enabled using the <code>CODEQL_ACTION_CLEANUP_TRAP_CACHES</code> environment variable is deprecated and will be removed in May 2026. If you are affected by this, we recommend disabling TRAP caching by passing the <code>trap-caching: false</code> input to the <code>init</code> Action. <a href="https://redirect.github.com/github/codeql-action/pull/3795">#3795</a></li> <li>The Git version 2.36.0 requirement for improved incremental analysis now only applies to repositories that contain submodules. <a href="https://redirect.github.com/github/codeql-action/pull/3789">#3789</a></li> <li>Python analysis on GHES no longer extracts the standard library, relying instead on models of the standard library. This should result in significantly faster extraction and analysis times, while the effect on alerts should be minimal. <a href="https://redirect.github.com/github/codeql-action/pull/3794">#3794</a></li> <li>Fixed a bug in the validation of OIDC configurations for private registries that was added in CodeQL Action 4.33.0 / 3.33.0. <a href="https://redirect.github.com/github/codeql-action/pull/3807">#3807</a></li> <li>Update default CodeQL bundle version to <a href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.2">2.25.2</a>. <a href="https://redirect.github.com/github/codeql-action/pull/3823">#3823</a></li> </ul> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/github/codeql-action/blob/main/CHANGELOG.md">github/codeql-action's changelog</a>.</em></p> <blockquote> <h1>CodeQL Action Changelog</h1> <p>See the <a href="https://github.com/github/codeql-action/releases">releases page</a> for the relevant changes to the CodeQL CLI and language packs.</p> <h2>[UNRELEASED]</h2> <p>No user facing changes.</p> <h2>4.35.2 - 15 Apr 2026</h2> <ul> <li>The undocumented TRAP cache cleanup feature that could be enabled using the <code>CODEQL_ACTION_CLEANUP_TRAP_CACHES</code> environment variable is deprecated and will be removed in May 2026. If you are affected by this, we recommend disabling TRAP caching by passing the <code>trap-caching: false</code> input to the <code>init</code> Action. <a href="https://redirect.github.com/github/codeql-action/pull/3795">#3795</a></li> <li>The Git version 2.36.0 requirement for improved incremental analysis now only applies to repositories that contain submodules. <a href="https://redirect.github.com/github/codeql-action/pull/3789">#3789</a></li> <li>Python analysis on GHES no longer extracts the standard library, relying instead on models of the standard library. This should result in significantly faster extraction and analysis times, while the effect on alerts should be minimal. <a href="https://redirect.github.com/github/codeql-action/pull/3794">#3794</a></li> <li>Fixed a bug in the validation of OIDC configurations for private registries that was added in CodeQL Action 4.33.0 / 3.33.0. <a href="https://redirect.github.com/github/codeql-action/pull/3807">#3807</a></li> <li>Update default CodeQL bundle version to <a href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.2">2.25.2</a>. <a href="https://redirect.github.com/github/codeql-action/pull/3823">#3823</a></li> </ul> <h2>4.35.1 - 27 Mar 2026</h2> <ul> <li>Fix incorrect minimum required Git version for <a href="https://redirect.github.com/github/roadmap/issues/1158">improved incremental analysis</a>: it should have been 2.36.0, not 2.11.0. <a href="https://redirect.github.com/github/codeql-action/pull/3781">#3781</a></li> </ul> <h2>4.35.0 - 27 Mar 2026</h2> <ul> <li>Reduced the minimum Git version required for <a href="https://redirect.github.com/github/roadmap/issues/1158">improved incremental analysis</a> from 2.38.0 to 2.11.0. <a href="https://redirect.github.com/github/codeql-action/pull/3767">#3767</a></li> <li>Update default CodeQL bundle version to <a href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.1">2.25.1</a>. <a href="https://redirect.github.com/github/codeql-action/pull/3773">#3773</a></li> </ul> <h2>4.34.1 - 20 Mar 2026</h2> <ul> <li>Downgrade default CodeQL bundle version to <a href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.24.3">2.24.3</a> due to issues with a small percentage of Actions and JavaScript analyses. <a href="https://redirect.github.com/github/codeql-action/pull/3762">#3762</a></li> </ul> <h2>4.34.0 - 20 Mar 2026</h2> <ul> <li>Added an experimental change which disables TRAP caching when <a href="https://redirect.github.com/github/roadmap/issues/1158">improved incremental analysis</a> is enabled, since improved incremental analysis supersedes TRAP caching. This will improve performance and reduce Actions cache usage. We expect to roll this change out to everyone in March. <a href="https://redirect.github.com/github/codeql-action/pull/3569">#3569</a></li> <li>We are rolling out improved incremental analysis to C/C++ analyses that use build mode <code>none</code>. We expect this rollout to be complete by the end of April 2026. <a href="https://redirect.github.com/github/codeql-action/pull/3584">#3584</a></li> <li>Update default CodeQL bundle version to <a href="https://github.com/github/codeql-action/releases/tag/codeql-bundle-v2.25.0">2.25.0</a>. <a href="https://redirect.github.com/github/codeql-action/pull/3585">#3585</a></li> </ul> <h2>4.33.0 - 16 Mar 2026</h2> <ul> <li> <p>Upcoming change: Starting April 2026, the CodeQL Action will skip collecting file coverage information on pull requests to improve analysis performance. File coverage information will still be computed on non-PR analyses. Pull request analyses will log a warning about this upcoming change. <a href="https://redirect.github.com/github/codeql-action/pull/3562">#3562</a></p> <p>To opt out of this change:</p> <ul> <li><strong>Repositories owned by an organization:</strong> Create a custom repository property with the name <code>github-codeql-file-coverage-on-prs</code> and the type "True/false", then set this property to <code>true</code> in the repository's settings. For more information, see <a href="https://docs.github.com/en/organizations/managing-organization-settings/managing-custom-properties-for-repositories-in-your-organization">Managing custom properties for repositories in your organization</a>. Alternatively, if you are using an advanced setup workflow, you can set the <code>CODEQL_ACTION_FILE_COVERAGE_ON_PRS</code> environment variable to <code>true</code> in your workflow.</li> <li><strong>User-owned repositories using default setup:</strong> Switch to an advanced setup workflow and set the <code>CODEQL_ACTION_FILE_COVERAGE_ON_PRS</code> environment variable to <code>true</code> in your workflow.</li> <li><strong>User-owned repositories using advanced setup:</strong> Set the <code>CODEQL_ACTION_FILE_COVERAGE_ON_PRS</code> environment variable to <code>true</code> in your workflow.</li> </ul> </li> <li> <p>Fixed <a href="https://redirect.github.com/github/codeql-action/issues/3555">a bug</a> which caused the CodeQL Action to fail loading repository properties if a "Multi select" repository property was configured for the repository. <a href="https://redirect.github.com/github/codeql-action/pull/3557">#3557</a></p> </li> <li> <p>The CodeQL Action now loads <a href="https://docs.github.com/en/organizations/managing-organization-settings/managing-custom-properties-for-repositories-in-your-organization">custom repository properties</a> on GitHub Enterprise Server, enabling the customization of features such as <code>github-codeql-disable-overlay</code> that was previously only available on GitHub.com. <a href="https://redirect.github.com/github/codeql-action/pull/3559">#3559</a></p> </li> <li> <p>Once <a href="https://docs.github.com/en/code-security/how-tos/secure-at-scale/configure-organization-security/manage-usage-and-access/giving-org-access-private-registries">private package registries</a> can be configured with OIDC-based authentication for organizations, the CodeQL Action will now be able to accept such configurations. <a href="https://redirect.github.com/github/codeql-action/pull/3563">#3563</a></p> </li> <li> <p>Fixed the retry mechanism for database uploads. Previously this would fail with the error "Response body object should not be disturbed or locked". <a href="https://redirect.github.com/github/codeql-action/pull/3564">#3564</a></p> </li> <li> <p>A warning is now emitted if the CodeQL Action detects a repository property whose name suggests that it relates to the CodeQL Action, but which is not one of the properties recognised by the current version of the CodeQL Action. <a href="https://redirect.github.com/github/codeql-action/pull/3570">#3570</a></p> </li> </ul> <h2>4.32.6 - 05 Mar 2026</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/github/codeql-action/commit/95e58e9a2cdfd71adc6e0353d5c52f41a045d225"><code>95e58e9</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/3824">#3824</a> from github/update-v4.35.2-d2e135a73</li> <li><a href="https://github.com/github/codeql-action/commit/6f31bfe060e817d81e938dbec767969d20031e25"><code>6f31bfe</code></a> Update changelog for v4.35.2</li> <li><a href="https://github.com/github/codeql-action/commit/d2e135a73a39154e3a231aeb49163c4661c5b8b1"><code>d2e135a</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/3823">#3823</a> from github/update-bundle/codeql-bundle-v2.25.2</li> <li><a href="https://github.com/github/codeql-action/commit/60abb65df09fcf213c398e064c8a80db1f15cdaf"><code>60abb65</code></a> Add changelog note</li> <li><a href="https://github.com/github/codeql-action/commit/5a0a562209255e956ad8aafcee303294e64eefa2"><code>5a0a562</code></a> Update default bundle to codeql-bundle-v2.25.2</li> <li><a href="https://github.com/github/codeql-action/commit/65216971a11ded447a6b76263d5a144519e5eee1"><code>6521697</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/3820">#3820</a> from github/dependabot/github_actions/dot-github/wor...</li> <li><a href="https://github.com/github/codeql-action/commit/3c45af2dd258e1623af1898da5c86545b514e028"><code>3c45af2</code></a> Merge pull request <a href="https://redirect.github.com/github/codeql-action/issues/3821">#3821</a> from github/dependabot/npm_and_yarn/npm-minor-345b93...</li> <li><a href="https://github.com/github/codeql-action/commit/f1c339364c12f922998186ed897e45e3b4ae8874"><code>f1c3393</code></a> Rebuild</li> <li><a href="https://github.com/github/codeql-action/commit/1024fc496c87e944a93e98d8cf2c09e2c7602a30"><code>1024fc4</code></a> Rebuild</li> <li><a href="https://github.com/github/codeql-action/commit/9dd4cfed96030ccdfe1af4daf7a7964322704fed"><code>9dd4cfe</code></a> Bump the npm-minor group across 1 directory with 6 updates</li> <li>Additional commits viewable in <a href="https://github.com/github/codeql-action/compare/c10b8064de6f491fea524254123dbe5e09572f13...95e58e9a2cdfd71adc6e0353d5c52f41a045d225">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.0.0 to 8.1.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/astral-sh/setup-uv/releases">astral-sh/setup-uv's releases</a>.</em></p> <blockquote> <h2>v8.1.0 🌈 New input <code>no-project</code></h2> <h2>Changes</h2> <p>This add the a new boolean input <code>no-project</code>. It only makes sense to use in combination with <code>activate-environment: true</code> and will append <code>--no project</code> to the <code>uv venv</code> call. This is for example useful <a href="https://redirect.github.com/astral-sh/setup-uv/issues/854">if you have a pyproject.toml file with parts unparseable by uv</a></p> <h2>🚀 Enhancements</h2> <ul> <li>Add input no-project in combination with activate-environment <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/856">#856</a>)</li> </ul> <h2>🧰 Maintenance</h2> <ul> <li>fix: grant contents:write to validate-release job <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/860">#860</a>)</li> <li>Add a release-gate step to the release workflow <a href="https://github.com/zanieb"><code>@zanieb</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/859">#859</a>)</li> <li>Draft commitish releases <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/858">#858</a>)</li> <li>Add action-types.yml to instructions <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/857">#857</a>)</li> <li>chore: update known checksums for 0.11.7 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/853">#853</a>)</li> <li>Refactor version resolving <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/852">#852</a>)</li> <li>chore: update known checksums for 0.11.6 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/850">#850</a>)</li> <li>chore: update known checksums for 0.11.5 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/845">#845</a>)</li> <li>chore: update known checksums for 0.11.4 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/843">#843</a>)</li> <li>Add a release workflow <a href="https://github.com/zanieb"><code>@zanieb</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/839">#839</a>)</li> <li>chore: update known checksums for 0.11.3 @<a href="https://github.com/apps/github-actions">github-actions[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/836">#836</a>)</li> </ul> <h2>📚 Documentation</h2> <ul> <li>Update ignore-nothing-to-cache documentation <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/833">#833</a>)</li> <li>Pin setup-uv docs to v8 <a href="https://github.com/eifinger"><code>@eifinger</code></a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/829">#829</a>)</li> </ul> <h2>⬆️ Dependency updates</h2> <ul> <li>chore(deps): bump release-drafter/release-drafter from 7.1.1 to 7.2.0 @<a href="https://github.com/apps/dependabot">dependabot[bot]</a> (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/855">#855</a>)</li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/astral-sh/setup-uv/commit/08807647e7069bb48b6ef5acd8ec9567f424441b"><code>0880764</code></a> fix: grant contents:write to validate-release job (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/860">#860</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/717d6aba0f15312f509f5c4999e34d71ecbab8a9"><code>717d6ab</code></a> Add a release-gate step to the release workflow (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/859">#859</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/5a911eb3a3983b5e650f2dad95c1ce698ca94378"><code>5a911eb</code></a> Draft commitish releases (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/858">#858</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/080c31e04cd7155b0ca676d08c7bc260a4476a23"><code>080c31e</code></a> Add action-types.yml to instructions (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/857">#857</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/b3e97d2ba1a1eed7e9d1f8456dd06c3b725bc3a6"><code>b3e97d2</code></a> Add input no-project in combination with activate-environment (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/856">#856</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/7dd591db9557f680290587fcc578372813b9ff64"><code>7dd591d</code></a> chore(deps): bump release-drafter/release-drafter from 7.1.1 to 7.2.0 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/855">#855</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/1541b7762698877904805605192ecd63d0e4787a"><code>1541b77</code></a> chore: update known checksums for 0.11.7 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/853">#853</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/cdfb2ee6dde255817c739680168ad81e184c4bfb"><code>cdfb2ee</code></a> Refactor version resolving (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/852">#852</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/cb84d12dc6a0d495b82fcae14fa4559b90698660"><code>cb84d12</code></a> chore: update known checksums for 0.11.6 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/850">#850</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/1912cc65f2e839707d7a16f2372f30b57d35fd80"><code>1912cc6</code></a> chore: update known checksums for 0.11.5 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/845">#845</a>)</li> <li>Additional commits viewable in <a href="https://github.com/astral-sh/setup-uv/compare/cec208311dfd045dd5311c1add060b2062131d57...08807647e7069bb48b6ef5acd8ec9567f424441b">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Python 3.14 is added to unit and integration test matrices with exclusions for all Airflow versions except 3.2. Also adds py313/py314 to Black target-version, removes py39, and adds Python 3.13/3.14 classifiers. CI: https://github.com/astronomer/dag-factory/actions/runs/24879319499/job/72844490915?pr=731 --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Bumps [apache-airflow-core](https://github.com/apache/airflow) from 3.1.8 to 3.2.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/airflow/releases">apache-airflow-core's releases</a>.</em></p> <blockquote> <h2>Apache Airflow 3.2.0</h2> <p>📦 PyPI: <a href="https://pypi.org/project/apache-airflow/3.2.0/">https://pypi.org/project/apache-airflow/3.2.0/</a> 📚 Docs: <a href="https://airflow.apache.org/docs/apache-airflow/3.2.0/">https://airflow.apache.org/docs/apache-airflow/3.2.0/</a> 🛠 Release Notes: <a href="https://airflow.apache.org/docs/apache-airflow/3.2.0/release_notes.html">https://airflow.apache.org/docs/apache-airflow/3.2.0/release_notes.html</a> 🐳 Docker Image: "docker pull apache/airflow:3.2.0" 🚏 Constraints: <a href="https://github.com/apache/airflow/tree/constraints-3.2.0">https://github.com/apache/airflow/tree/constraints-3.2.0</a></p> <h2>Significant Changes</h2> <h3>Asset Partitioning</h3> <p>The headline feature of Airflow 3.2.0 is asset partitioning — a major evolution of data-aware scheduling. Instead of triggering Dags based on an entire asset, you can now schedule downstream processing based on specific partitions of data. Only the relevant slice of data triggers downstream work, making pipeline orchestration far more efficient and precise.</p> <p>This matters when working with partitioned data lakes — date-partitioned S3 paths, Hive table partitions, BigQuery table partitions, or any other partitioned data store. Previously, any update to an asset triggered all downstream Dags regardless of which partition changed. Now only the right work gets triggered at the right time.</p> <p>For detailed usage instructions, see :doc:<code>/authoring-and-scheduling/assets</code>.</p> <h3>Multi-Team Deployments</h3> <p>Airflow 3.2 introduces multi-team support, allowing organizations to run multiple isolated teams within a single Airflow deployment. Each team can have its own Dags, connections, variables, pools, and executors— enabling true resource and permission isolation without requiring separate Airflow instances per team.</p> <p>This is particularly valuable for platform teams that serve multiple data engineering or data science teams from shared infrastructure, while maintaining strong boundaries between teams' resources and access.</p> <p>For detailed usage instructions, see :doc:<code>/core-concepts/multi-team</code>.</p> <p>.. warning::</p> <p>Multi-Team Deployments are experimental in 3.2.0 and may change in future versions based on user feedback.</p> <h3>Synchronous callback support for Deadline Alerts</h3> <p>Deadline Alerts now support synchronous callbacks via <code>SyncCallback</code> in addition to the existing asynchronous <code>AsyncCallback</code>. Synchronous callbacks are executed by the executor (rather than the triggerer), and can optionally target a specific executor via the <code>executor</code> parameter.</p> <p>A Dag can also define multiple Deadline Alerts by passing a list to the <code>deadline</code> parameter, and each alert can use either callback type.</p> <p>.. warning::</p> <p>Deadline Alerts are experimental in 3.2.0 and may change in future versions based on</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/airflow/commit/3bc3ccfacc3dec9f359a3b153bfd4fc706c661ba"><code>3bc3ccf</code></a> Update release notes for 3.2.0rc2</li> <li><a href="https://github.com/apache/airflow/commit/531196158fcb6ac9456d2c6ba004ef93822a31f9"><code>5311961</code></a> [v3-2-test] Guard against null trigger in asset watcher cleanup (<a href="https://redirect.github.com/apache/airflow/issues/64659">#64659</a>) (<a href="https://redirect.github.com/apache/airflow/issues/64">#64</a>...</li> <li><a href="https://github.com/apache/airflow/commit/3840fae2e16ecbd86fda948607a5822a7c506703"><code>3840fae</code></a> [v3-2-test] Fix ObjectStoragePath NoCredentialsError when using conn_id with ...</li> <li><a href="https://github.com/apache/airflow/commit/959ebd83c1335cc3352355ab7e02ff00813120eb"><code>959ebd8</code></a> [v3-2-test] Fix double-serialization issue by unwrapping serialized kwargs in...</li> <li><a href="https://github.com/apache/airflow/commit/d1d2416310513bdf02382381b5dd55275b45ccdb"><code>d1d2416</code></a> [v3-2-test] Run DB check only for core components in prod entrypoint (<a href="https://redirect.github.com/apache/airflow/issues/63413">#63413</a>)...</li> <li><a href="https://github.com/apache/airflow/commit/beee8b693c04e11b63e132ef862b4018fdb03a0d"><code>beee8b6</code></a> [v3-2-test] Remove false-positive RFC3986 underscore warning from Connection....</li> <li><a href="https://github.com/apache/airflow/commit/68d98747872d999929c2bff5f4df97489d921405"><code>68d9874</code></a> [v3-2-test] fix: restore early return in check_for_write_conflict (<a href="https://redirect.github.com/apache/airflow/issues/64062">#64062</a>) (#...</li> <li><a href="https://github.com/apache/airflow/commit/0a000c486d30f533ca45a9df9fdafed73468e53d"><code>0a000c4</code></a> [v3-2-test] Fix deferred task resume failure when worker is older than server...</li> <li><a href="https://github.com/apache/airflow/commit/8d0fb4b34d1253928a671f90c46a5560bf5f2cb0"><code>8d0fb4b</code></a> [v3-2-test] Fix serde deserialization of old-format builtin types in trigger ...</li> <li><a href="https://github.com/apache/airflow/commit/e64b7e41d72e5bad9ff7d90b8d10797f0a9a0756"><code>e64b7e4</code></a> [v3-2-test] Fix: Restore live stdout logging for Elasticsearch log forwarding...</li> <li>Additional commits viewable in <a href="https://github.com/apache/airflow/compare/3.1.8...3.2.0">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/astronomer/dag-factory/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [mako](https://github.com/sqlalchemy/mako) from 1.3.10 to 1.3.11. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/sqlalchemy/mako/releases">mako's releases</a>.</em></p> <blockquote> <h1>1.3.11</h1> <p>Released: Tue Apr 14 2026</p> <h2>bug</h2> <ul> <li> <p><strong>[bug] [template]</strong> Fixed issue in <code>TemplateLookup</code> where a URI with a double-slash prefix (e.g. <code>//../../</code>) could bypass the directory traversal check in <code>Template</code>, allowing reads of arbitrary files outside of the template directory. The issue was caused by an inconsistency in how leading slashes were stripped between <code>TemplateLookup.get_template()</code> and <code>Template</code> initialization.</p> <p>References: <a href="https://redirect.github.com/sqlalchemy/mako/issues/434">#434</a></p> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li>See full diff in <a href="https://github.com/sqlalchemy/mako/commits">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/astronomer/dag-factory/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.6.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/urllib3/urllib3/releases">urllib3's releases</a>.</em></p> <blockquote> <h2>2.6.3</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <p>Thank you for your support.</p> <h2>Changes</h2> <ul> <li>Fixed a security issue where decompression-bomb safeguards of the streaming API were bypassed when HTTP redirects were followed. (CVE-2026-21441 reported by <a href="https://github.com/D47A"><code>@D47A</code></a>, 8.9 High, GHSA-38jv-5279-wg99)</li> <li>Started treating <code>Retry-After</code> times greater than 6 hours as 6 hours by default. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3743">urllib3/urllib3#3743</a>)</li> <li>Fixed <code>urllib3.connection.VerifiedHTTPSConnection</code> on Emscripten. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3752">urllib3/urllib3#3752</a>)</li> </ul> <h2>2.6.2</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <p>Thank you for your support.</p> <h2>Changes</h2> <ul> <li>Fixed <code>HTTPResponse.read_chunked()</code> to properly handle leftover data in the decoder's buffer when reading compressed chunked responses. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3734">urllib3/urllib3#3734</a>)</li> </ul> <h2>2.6.1</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <p>Thank you for your support.</p> <h2>Changes</h2> <ul> <li>Restore previously removed <code>HTTPResponse.getheaders()</code> and <code>HTTPResponse.getheader()</code> methods. (<a href="https://redirect.github.com/urllib3/urllib3/issues/3731">#3731</a>)</li> </ul> <h2>2.6.0</h2> <h2>🚀 urllib3 is fundraising for HTTP/2 support</h2> <p><a href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3 is raising ~$40,000 USD</a> to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects <a href="https://opencollective.com/urllib3">please consider contributing financially</a> to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.</p> <p>Thank you for your support.</p> <h2>Security</h2> <ul> <li>Fixed a security issue where streaming API could improperly handle highly compressed HTTP content ("decompression bombs") leading to excessive resource consumption even when a small amount of data was requested. Reading small chunks of compressed data is safer and much more efficient now. (CVE-2025-66471 reported by <a href="https://github.com/Cycloctane"><code>@Cycloctane</code></a>, 8.9 High, GHSA-2xpw-w6gg-jr37)</li> <li>Fixed a security issue where an attacker could compose an HTTP response with virtually unlimited links in the <code>Content-Encoding</code> header, potentially leading to a denial of service (DoS) attack by exhausting system resources during decoding. The number of allowed chained encodings is now limited to 5. (CVE-2025-66418 reported by <a href="https://github.com/illia-v"><code>@illia-v</code></a>, 8.9 High, GHSA-gm62-xv2j-4w53)</li> </ul> <blockquote> <p>[!IMPORTANT]</p> <ul> <li>If urllib3 is not installed with the optional <code>urllib3[brotli]</code> extra, but your environment contains a Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security fixes and avoid warnings. Prefer using <code>urllib3[brotli]</code> to install a compatible Brotli package automatically.</li> </ul> </blockquote> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's changelog</a>.</em></p> <blockquote> <h1>2.6.3 (2026-01-07)</h1> <ul> <li>Fixed a high-severity security issue where decompression-bomb safeguards of the streaming API were bypassed when HTTP redirects were followed. (<code>GHSA-38jv-5279-wg99 <https://github.com/urllib3/urllib3/security/advisories/GHSA-38jv-5279-wg99></code>__)</li> <li>Started treating <code>Retry-After</code> times greater than 6 hours as 6 hours by default. (<code>[#3743](urllib3/urllib3#3743) <https://github.com/urllib3/urllib3/issues/3743></code>__)</li> <li>Fixed <code>urllib3.connection.VerifiedHTTPSConnection</code> on Emscripten. (<code>[#3752](urllib3/urllib3#3752) <https://github.com/urllib3/urllib3/issues/3752></code>__)</li> </ul> <h1>2.6.2 (2025-12-11)</h1> <ul> <li>Fixed <code>HTTPResponse.read_chunked()</code> to properly handle leftover data in the decoder's buffer when reading compressed chunked responses. (<code>[#3734](urllib3/urllib3#3734) <https://github.com/urllib3/urllib3/issues/3734></code>__)</li> </ul> <h1>2.6.1 (2025-12-08)</h1> <ul> <li>Restore previously removed <code>HTTPResponse.getheaders()</code> and <code>HTTPResponse.getheader()</code> methods. (<code>[#3731](urllib3/urllib3#3731) <https://github.com/urllib3/urllib3/issues/3731></code>__)</li> </ul> <h1>2.6.0 (2025-12-05)</h1> <h2>Security</h2> <ul> <li>Fixed a security issue where streaming API could improperly handle highly compressed HTTP content ("decompression bombs") leading to excessive resource consumption even when a small amount of data was requested. Reading small chunks of compressed data is safer and much more efficient now. (<code>GHSA-2xpw-w6gg-jr37 <https://github.com/urllib3/urllib3/security/advisories/GHSA-2xpw-w6gg-jr37></code>__)</li> <li>Fixed a security issue where an attacker could compose an HTTP response with virtually unlimited links in the <code>Content-Encoding</code> header, potentially leading to a denial of service (DoS) attack by exhausting system resources during decoding. The number of allowed chained encodings is now limited to 5. (<code>GHSA-gm62-xv2j-4w53 <https://github.com/urllib3/urllib3/security/advisories/GHSA-gm62-xv2j-4w53></code>__)</li> </ul> <p>.. caution::</p> <ul> <li>If urllib3 is not installed with the optional <code>urllib3[brotli]</code> extra, but your environment contains a Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security fixes and avoid warnings. Prefer using</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/urllib3/urllib3/commit/0248277dd7ac0239204889ca991353ad3e3a1ddc"><code>0248277</code></a> Release 2.6.3</li> <li><a href="https://github.com/urllib3/urllib3/commit/8864ac407bba8607950025e0979c4c69bc7abc7b"><code>8864ac4</code></a> Merge commit from fork</li> <li><a href="https://github.com/urllib3/urllib3/commit/70cecb27ca99d56aaaeb63ac27ee270ef2b24c5c"><code>70cecb2</code></a> Fix Scorecard issues related to vulnerable dev dependencies (<a href="https://redirect.github.com/urllib3/urllib3/issues/3755">#3755</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/41f249abe1ef3e20768588969c4035aba060a359"><code>41f249a</code></a> Move "v2.0 Migration Guide" to the end of the table of contents (<a href="https://redirect.github.com/urllib3/urllib3/issues/3747">#3747</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/fd4dffd2fc544166b76151a2fa3d7b7c0eab540c"><code>fd4dffd</code></a> Patch <code>VerifiedHTTPSConnection</code> for Emscripten (<a href="https://redirect.github.com/urllib3/urllib3/issues/3752">#3752</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/13f0bfd55e4468fe1ea9c6f809d3a87b0f93ebab"><code>13f0bfd</code></a> Handle massive values in Retry-After when calculating time to sleep for (<a href="https://redirect.github.com/urllib3/urllib3/issues/3743">#3743</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/8c480bf87bcefd321b3a1ae47f04e908b6b2ed7b"><code>8c480bf</code></a> Bump actions/upload-artifact from 5.0.0 to 6.0.0 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3748">#3748</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/4b40616e959c0a2c466e8075f2a785a9f99bb0c1"><code>4b40616</code></a> Bump actions/cache from 4.3.0 to 5.0.1 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3750">#3750</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/82b8479663d037d220c883f1584dd01a43bb273b"><code>82b8479</code></a> Bump actions/download-artifact from 6.0.0 to 7.0.0 (<a href="https://redirect.github.com/urllib3/urllib3/issues/3749">#3749</a>)</li> <li><a href="https://github.com/urllib3/urllib3/commit/34284cb01700bb7d4fdd472f909e22393e9174e2"><code>34284cb</code></a> Mention experimental features in the security policy (<a href="https://redirect.github.com/urllib3/urllib3/issues/3746">#3746</a>)</li> <li>Additional commits viewable in <a href="https://github.com/urllib3/urllib3/compare/2.4.0...2.6.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/astronomer/dag-factory/network/alerts). </details> > **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [filelock](https://github.com/tox-dev/py-filelock) from 3.18.0 to 3.20.3. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tox-dev/py-filelock/releases">filelock's releases</a>.</em></p> <blockquote> <h2>3.20.3</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>Fix TOCTOU symlink vulnerability in SoftFileLock by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/465">tox-dev/filelock#465</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/tox-dev/filelock/compare/3.20.2...3.20.3">https://github.com/tox-dev/filelock/compare/3.20.2...3.20.3</a></p> <h2>3.20.2</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>Support Unix systems without O_NOFOLLOW by <a href="https://github.com/mwilliamson"><code>@mwilliamson</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/463">tox-dev/filelock#463</a></li> <li>[pre-commit.ci] pre-commit autoupdate by <a href="https://github.com/pre-commit-ci"><code>@pre-commit-ci</code></a>[bot] in <a href="https://redirect.github.com/tox-dev/filelock/pull/464">tox-dev/filelock#464</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/mwilliamson"><code>@mwilliamson</code></a> made their first contribution in <a href="https://redirect.github.com/tox-dev/filelock/pull/463">tox-dev/filelock#463</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/tox-dev/filelock/compare/3.20.1...3.20.2">https://github.com/tox-dev/filelock/compare/3.20.1...3.20.2</a></p> <h2>3.20.1</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>CVE-2025-68146: Fix TOCTOU symlink vulnerability in lock file creation by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/461">tox-dev/filelock#461</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/tox-dev/filelock/compare/3.20.0...3.20.1">https://github.com/tox-dev/filelock/compare/3.20.0...3.20.1</a></p> <h2>3.20.0</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>Add tox.toml to sdist by <a href="https://github.com/mtelka"><code>@mtelka</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/436">tox-dev/filelock#436</a></li> <li>Update docs with example by <a href="https://github.com/znichollscr"><code>@znichollscr</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/438">tox-dev/filelock#438</a></li> <li>Add 3.14 support and drop 3.9 by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/448">tox-dev/filelock#448</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/mtelka"><code>@mtelka</code></a> made their first contribution in <a href="https://redirect.github.com/tox-dev/filelock/pull/436">tox-dev/filelock#436</a></li> <li><a href="https://github.com/znichollscr"><code>@znichollscr</code></a> made their first contribution in <a href="https://redirect.github.com/tox-dev/filelock/pull/438">tox-dev/filelock#438</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/tox-dev/filelock/compare/3.19.1...3.20.0">https://github.com/tox-dev/filelock/compare/3.19.1...3.20.0</a></p> <h2>3.19.1</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>add 3.14t (free threading) to matrix by <a href="https://github.com/paultiq"><code>@paultiq</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/433">tox-dev/filelock#433</a></li> <li>Increase test coverage by <a href="https://github.com/paultiq"><code>@paultiq</code></a> in <a href="https://redirect.github.com/tox-dev/filelock/pull/434">tox-dev/filelock#434</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/tox-dev/filelock/commit/41b42dd2c72aecf7da83dbda5903b8087dddc4d5"><code>41b42dd</code></a> Fix TOCTOU symlink vulnerability in SoftFileLock (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/465">#465</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/f2e7d4046b6a2b375a573bcfbad21827b99f8939"><code>f2e7d40</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/464">#464</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/50888548eb2f008d372e71f2835a47851ab83836"><code>5088854</code></a> Support Unix systems without O_NOFOLLOW (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/463">#463</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/377f62251d7cdf30768cc9ee1eb31cea1551c71b"><code>377f622</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/460">#460</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/4724d7f8c3393ec1f048c93933e6e3e6ec321f0e"><code>4724d7f</code></a> Fix TOCTOU symlink vulnerability in lock file creation (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/461">#461</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/cb69414a2327cf0a9887e12054d1dc112ee700af"><code>cb69414</code></a> Bump actions/upload-artifact from 5 to 6 (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/459">#459</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/0769294f14a6c62eea64741722f7acef5386b4cd"><code>0769294</code></a> Bump actions/download-artifact from 6 to 7 (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/458">#458</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/414193a188892bd376eb5c56eb45a9cf8ecc9284"><code>414193a</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/457">#457</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/1456797beb94ad59e5627462ad29f7ed3a966626"><code>1456797</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/456">#456</a>)</li> <li><a href="https://github.com/tox-dev/filelock/commit/8d6bf90af313ac7fd6e41ef2b715d91dd6858f5c"><code>8d6bf90</code></a> Bump actions/checkout from 5 to 6 (<a href="https://redirect.github.com/tox-dev/py-filelock/issues/455">#455</a>)</li> <li>Additional commits viewable in <a href="https://github.com/tox-dev/py-filelock/compare/3.18.0...3.20.3">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/astronomer/dag-factory/network/alerts). </details> > **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [virtualenv](https://github.com/pypa/virtualenv) from 20.31.2 to 20.36.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pypa/virtualenv/releases">virtualenv's releases</a>.</em></p> <blockquote> <h2>20.36.1</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>release 20.36.0 by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3011">pypa/virtualenv#3011</a></li> <li>fix: resolve TOCTOU vulnerabilities in app_data and lock directory creation by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3013">pypa/virtualenv#3013</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/pypa/virtualenv/compare/20.36.0...20.36.1">https://github.com/pypa/virtualenv/compare/20.36.0...20.36.1</a></p> <h2>20.36.0</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>release 20.35.3 by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2981">pypa/virtualenv#2981</a></li> <li>fix: Prevent NameError when accessing _DISTUTILS_PATCH during file ov… by <a href="https://github.com/gracetyy"><code>@gracetyy</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2982">pypa/virtualenv#2982</a></li> <li>Upgrade pip and fix 3.15 picking old wheel by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2989">pypa/virtualenv#2989</a></li> <li>release 20.35.4 by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2990">pypa/virtualenv#2990</a></li> <li>fix: wrong path on migrated venv by <a href="https://github.com/sk1234567891"><code>@sk1234567891</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2996">pypa/virtualenv#2996</a></li> <li>test_too_many_open_files: assert on <code>errno.EMFILE</code> instead of <code>strerror</code> by <a href="https://github.com/pltrz"><code>@pltrz</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3001">pypa/virtualenv#3001</a></li> <li>fix: update filelock dependency version to 3.20.1 to fix CVE CVE-2025-68146 by <a href="https://github.com/pythonhubdev"><code>@pythonhubdev</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3002">pypa/virtualenv#3002</a></li> <li>fix: resolve EncodingWarning in tox upgrade environment by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3007">pypa/virtualenv#3007</a></li> <li>Fix Interpreter discovery bug wrt. Microsoft Store shortcut using Latin-1 by <a href="https://github.com/rahuldevikar"><code>@rahuldevikar</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3006">pypa/virtualenv#3006</a></li> <li>Add support for PEP 440 version specifiers in the <code>--python</code> flag. by <a href="https://github.com/rahuldevikar"><code>@rahuldevikar</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/3008">pypa/virtualenv#3008</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/gracetyy"><code>@gracetyy</code></a> made their first contribution in <a href="https://redirect.github.com/pypa/virtualenv/pull/2982">pypa/virtualenv#2982</a></li> <li><a href="https://github.com/sk1234567891"><code>@sk1234567891</code></a> made their first contribution in <a href="https://redirect.github.com/pypa/virtualenv/pull/2996">pypa/virtualenv#2996</a></li> <li><a href="https://github.com/pltrz"><code>@pltrz</code></a> made their first contribution in <a href="https://redirect.github.com/pypa/virtualenv/pull/3001">pypa/virtualenv#3001</a></li> <li><a href="https://github.com/pythonhubdev"><code>@pythonhubdev</code></a> made their first contribution in <a href="https://redirect.github.com/pypa/virtualenv/pull/3002">pypa/virtualenv#3002</a></li> <li><a href="https://github.com/rahuldevikar"><code>@rahuldevikar</code></a> made their first contribution in <a href="https://redirect.github.com/pypa/virtualenv/pull/3006">pypa/virtualenv#3006</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/pypa/virtualenv/compare/20.35.3...20.36.0">https://github.com/pypa/virtualenv/compare/20.35.3...20.36.0</a></p> <h2>20.35.4</h2> <!-- raw HTML omitted --> <h2>What's Changed</h2> <ul> <li>release 20.35.3 by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2981">pypa/virtualenv#2981</a></li> <li>fix: Prevent NameError when accessing _DISTUTILS_PATCH during file ov… by <a href="https://github.com/gracetyy"><code>@gracetyy</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2982">pypa/virtualenv#2982</a></li> <li>Upgrade pip and fix 3.15 picking old wheel by <a href="https://github.com/gaborbernat"><code>@gaborbernat</code></a> in <a href="https://redirect.github.com/pypa/virtualenv/pull/2989">pypa/virtualenv#2989</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/gracetyy"><code>@gracetyy</code></a> made their first contribution in <a href="https://redirect.github.com/pypa/virtualenv/pull/2982">pypa/virtualenv#2982</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/pypa/virtualenv/compare/20.35.3...20.35.4">https://github.com/pypa/virtualenv/compare/20.35.3...20.35.4</a></p> <h2>20.35.3</h2> <!-- raw HTML omitted --> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pypa/virtualenv/blob/main/docs/changelog.rst">virtualenv's changelog</a>.</em></p> <blockquote> <h2>v20.36.1 (2026-01-09)</h2> <p>Bugfixes - 20.36.1</p> <pre><code>- Fix TOCTOU vulnerabilities in app_data and lock directory creation that could be exploited via symlink attacks - reported by :user:`tsigouris007`, fixed by :user:`gaborbernat`. (:issue:`3013`) <h2>v20.36.0 (2026-01-07)</h2> <p>Features - 20.36.0 </code></pre></p> <ul> <li>Add support for PEP 440 version specifiers in the <code>--python</code> flag. Users can now specify Python versions using operators like <code>>=</code>, <code><=</code>, <code>~=</code>, etc. For example: <code>virtualenv --python=">=3.12" myenv</code> <code>. (:issue:</code>2994`)</li> </ul> <h2>v20.35.4 (2025-10-28)</h2> <p>Bugfixes - 20.35.4</p> <pre><code>- Fix race condition in ``_virtualenv.py`` when file is overwritten during import, preventing ``NameError`` when ``_DISTUTILS_PATCH`` is accessed - by :user:`gracetyy`. (:issue:`2969`) - Upgrade embedded wheels: <ul> <li>pip to <code>25.3</code> from <code>25.2</code> (:issue:<code>2989</code>)</li> </ul> <h2>v20.35.3 (2025-10-10)</h2> <p>Bugfixes - 20.35.3 </code></pre></p> <ul> <li>Accept RuntimeError in <code>test_too_many_open_files</code>, by :user:<code>esafak</code> (:issue:<code>2935</code>)</li> </ul> <h2>v20.35.2 (2025-10-10)</h2> <p>Bugfixes - 20.35.2</p> <pre><code>- Revert out changes related to the extraction of the discovery module - by :user:`gaborbernat`. (:issue:`2978`) <h2>v20.35.1 (2025-10-09)</h2> <p>Bugfixes - 20.35.1 </code></pre></p> <ul> <li>Patch get_interpreter to handle missing cache and app_data - by :user:<code>esafak</code> (:issue:<code>2972</code>)</li> <li>Fix backwards incompatible changes to <code>PythonInfo</code> - by :user:<code>gaborbernat</code>. (:issue:<code>2975</code>)</li> </ul> <h2>v20.35.0 (2025-10-08)</h2> <p>Features - 20.35.0</p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pypa/virtualenv/commit/d0ad11d1146e81ea74d2461be9653f1da9cf3fd1"><code>d0ad11d</code></a> release 20.36.1</li> <li><a href="https://github.com/pypa/virtualenv/commit/dec4cec5d16edaf83a00a658f32d1e032661cebc"><code>dec4cec</code></a> Merge pull request <a href="https://redirect.github.com/pypa/virtualenv/issues/3013">#3013</a> from gaborbernat/fix-sec</li> <li><a href="https://github.com/pypa/virtualenv/commit/5fe5d38beb1273b489591a7b444f1018af2edf0a"><code>5fe5d38</code></a> release 20.36.0 (<a href="https://redirect.github.com/pypa/virtualenv/issues/3011">#3011</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/9719376addaa710b61d9ed013774fa26f6224b4e"><code>9719376</code></a> release 20.36.0</li> <li><a href="https://github.com/pypa/virtualenv/commit/0276db6fcf8849c519d75465f659b12aefb2acd8"><code>0276db6</code></a> Add support for PEP 440 version specifiers in the <code>--python</code> flag. (<a href="https://redirect.github.com/pypa/virtualenv/issues/3008">#3008</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/4f900c29044e17812981b5b98ddce45604858b7f"><code>4f900c2</code></a> Fix Interpreter discovery bug wrt. Microsoft Store shortcut using Latin-1 (<a href="https://redirect.github.com/pypa/virtualenv/issues/3">#3</a>...</li> <li><a href="https://github.com/pypa/virtualenv/commit/13afcc62a3444d0386c8031d0a62277a8274ab07"><code>13afcc6</code></a> fix: resolve EncodingWarning in tox upgrade environment (<a href="https://redirect.github.com/pypa/virtualenv/issues/3007">#3007</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/31b5d31581df3e3a7bbc55e52568b26dd01b0d57"><code>31b5d31</code></a> [pre-commit.ci] pre-commit autoupdate (<a href="https://redirect.github.com/pypa/virtualenv/issues/2997">#2997</a>)</li> <li><a href="https://github.com/pypa/virtualenv/commit/7c284221b4751388801355fc6ebaa2abe60427bd"><code>7c28422</code></a> fix: update filelock dependency version to 3.20.1 to fix CVE CVE-2025-68146 (...</li> <li><a href="https://github.com/pypa/virtualenv/commit/365628c544cd5498fbf0a3b6c6a8c1f41d25a749"><code>365628c</code></a> test_too_many_open_files: assert on <code>errno.EMFILE</code> instead of <code>strerror</code> (<a href="https://redirect.github.com/pypa/virtualenv/issues/3001">#3001</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pypa/virtualenv/compare/20.31.2...20.36.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/astronomer/dag-factory/network/alerts). </details> > **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Partially resolves: astronomer#687 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
… for inlets/outlets (astronomer#737) ## Summary `dagbuilder.DagBuilder.adjust_general_task_params` was gating the conversion of bare URI strings in `inlets`/`outlets` to `Dataset`/`Asset` objects to Airflow < 3.0.0 only. On Airflow 3 this caused outlets declared as URI strings (e.g. `outlets: ["s3://bucket/key"]`) to remain as plain strings on the resulting task, so: - no `AssetEvent` was emitted on task success, and - consumer DAGs scheduled on the corresponding `Asset` never triggered. The conversion is also valid on Airflow 3 because the `Dataset` symbol imported in `dagbuilder.py` resolves to `airflow.sdk.definitions.asset.Asset` on Airflow 3 (and to `airflow.datasets.Dataset` on Airflow 2). Removing the version gate makes asset-based scheduling work for DAGs authored via dag-factory YAML on Airflow 3. ## Changes - `dagfactory/dagbuilder.py`: remove the `if version.parse(AIRFLOW_VERSION) < version.parse("3.0.0"):` gate so the inlets/outlets URI-to-Asset conversion runs on Airflow 3 too. De-indent the body and add an explanatory comment. - `tests/test_dagbuilder.py`: remove the `@pytest.mark.skipif(... > 3.0.0 ...)` from `test_make_task_inlets_outlets`. With the fix in place, the test passes on Airflow 3 — its assertions use `Dataset` imported from `dagfactory.dagbuilder`, which is already aliased to `airflow.sdk.Asset` on Airflow 3. ## Test plan - [x] Existing `test_make_task_inlets_outlets` parametrized cases (3 cases) pass on Airflow 2 and on Airflow 3 (previously skipped on AF3). - [x] Manually validated end-to-end on Airflow 3.2.1: a producer DAG with YAML `outlets: ["s3://producer/example.txt"]` writes a row to the `asset_event` table on success, and a consumer DAG declared with `schedule:` referencing the same Asset is auto-triggered by the scheduler. **Setup.** Airflow 3.2.1 (official `apache/airflow:3.2.1` image), Postgres 16, LocalExecutor, dag-factory mounted editable into `/opt/airflow/dag-factory` and exposed via `PYTHONPATH`. Two DAGs authored in YAML via dag-factory: - **Producer** `asset_producer` declares one PythonOperator with `outlets: ["s3://producer/example.txt"]`. - **Consumer** `asset_consumer` declares `schedule:` referencing an Asset whose URI is `s3://producer/example.txt` (declared with `__type__: airflow.sdk.Asset` — see Note 2 at the bottom). **Procedure.** With each variant of `dagbuilder.py`, restart the scheduler so the dag-processor reloads the dag-factory module, trigger the producer once, and observe (a) the resolved outlets on the task, (b) the `asset_event` table, (c) whether the consumer DAG auto-triggers within a few seconds. ### Test fixtures used to validate this change I authored a minimal producer/consumer pair via dag-factory YAML to exercise asset-based scheduling end-to-end on Airflow 3.2.1. #### `dags/asset_producer.yml` — declares an outlet as a bare URI string ```yaml default: default_args: owner: "experiment" start_date: "2025-01-01" schedule: null catchup: false tags: - "experiment" - "asset-producer" asset_producer: description: "Producer DAG: declares an outlet using a bare URI string." tasks: - task_id: produce operator: airflow.providers.standard.operators.bash.BashOperator bash_command: "echo 'producer fired at' $(date -u +%FT%TZ)" outlets: - "file:///tmp/dagfactory_demo_asset" ``` #### `dags/asset_producer.py` — dag-factory loader ```python """Loader for asset_producer.yml - registers the producer DAG with Airflow.""" import os from pathlib import Path from dagfactory import load_yaml_dags DAGS_DIR = Path(os.path.dirname(__file__)) load_yaml_dags( globals_dict=globals(), config_filepath=str(DAGS_DIR / "asset_producer.yml"), ) ``` <img width="2436" height="1688" alt="image" src="https://github.com/user-attachments/assets/e3d36fb2-ac7d-41ab-93a1-cf2fd70fc773" /> #### `dags/asset_consumer.yml` — schedule references the same Asset ```yaml default: default_args: owner: "experiment" start_date: "2025-01-01" catchup: false tags: - "experiment" - "asset-consumer" asset_consumer: description: "Consumer DAG: schedule is a real Asset (built via __type__) pointing at the same URI as the producer's outlet." schedule: - __type__: airflow.sdk.Asset uri: "file:///tmp/dagfactory_demo_asset" name: "file:///tmp/dagfactory_demo_asset" # match what dag-factory produces from a URI-string outlet (URI used as name) tasks: - task_id: consume operator: airflow.providers.standard.operators.bash.BashOperator bash_command: "echo 'CONSUMER FIRED at' $(date -u +%FT%TZ)" ``` #### `dags/asset_consumer.py` — dag-factory loader ```python """Loader for asset_consumer.yml - registers the consumer DAG with Airflow.""" import os from pathlib import Path from dagfactory import load_yaml_dags DAGS_DIR = Path(os.path.dirname(__file__)) load_yaml_dags( globals_dict=globals(), config_filepath=str(DAGS_DIR / "asset_consumer.yml"), ) ``` <img width="2436" height="1688" alt="image" src="https://github.com/user-attachments/assets/5fdbb918-8403-4933-ab7c-8caf0c333ddd" /> #### Notes on the fixture choices - **Producer outlet is a bare URI string** — this is the natural way users write asset outlets in YAML and the case this PR fixes. - **Consumer schedule uses explicit `__type__: airflow.sdk.Asset`** — a workaround for a *separate* dag-factory bug (consumer-side `schedule:` doesn't convert URI strings to Asset on Airflow 3 either; out of scope here). - **`name` is set equal to `uri`** — Airflow 3 keys assets by both `name` and `uri`. dag-factory's `Dataset(uri)` constructor (which becomes `Asset(uri)` on AF3) uses the URI string as both, so the consumer fixture must match exactly to avoid `AirflowInactiveAssetInInletOrOutletException` on the producer task. ### Observations | Signal | BEFORE (gate at line 975 present) | AFTER (gate removed) | |---|---|---| | Type of `task.outlets[0]` | `str` (literal `'s3://producer/example.txt'`) | `airflow.sdk.definitions.asset.Asset` | | Rows in `asset_event` after producer success | **0** | **1** | | Consumer DAG runs auto-triggered (`run_type=asset_triggered`) | **0** — never starts | **1** — auto-triggered within seconds of producer success | | `airflow assets list-events --asset-uri s3://producer/example.txt` | empty | one event row attributed to `asset_producer.process` | | Producer task final state in UI | `success` | `success` (unchanged) | ### Why the conversion is needed on Airflow 3 <img width="1129" height="1924" alt="image" src="https://github.com/user-attachments/assets/95b9aa9c-db7b-4885-8435-8b4276277f1b" /> The `Dataset` symbol imported at the top of `dagfactory/dagbuilder.py` already resolves to `airflow.sdk.definitions.asset.Asset` on Airflow 3, so removing the gate is sufficient — no other changes are needed for the conversion to be correct. ## Pre-commit / lint validation Per the [contributing guide](https://github.com/astronomer/dag-factory/blob/main/docs/contributing/howto.md#pre-commit-and-linting), ran the same checks `pre-commit run --all-files` would run, against this PR's diff. Clean. | Hook (from `.pre-commit-config.yaml`) | Pinned version | Local version run | Result on this PR's diff | |---|---|---|---| | `ruff` (`astral-sh/ruff-pre-commit`) | `v0.6.9` | `0.15.12` (newer = stricter) | clean | | `black` (`psf/black`) | `26.3.1` | `26.3.1` (exact match) | clean | | `codespell` (`codespell-project/codespell`) | `v2.2.4` | `2.4.2` (newer = stricter) | clean | | trailing-whitespace, end-of-file-fixer, mixed-line-ending (`pre-commit/pre-commit-hooks`) | `v5.0.0` | n/a (file-content check) | clean | ### Reproduction snippet ```bash docker compose exec -T postgres psql -U airflow -d airflow -c \ "SELECT ae.id, ae.source_dag_id, ae.source_task_id, ae.source_run_id, a.uri, ae.timestamp FROM asset_event ae JOIN asset a ON a.id = ae.asset_id WHERE a.uri = 's3://producer/example.txt' ORDER BY ae.timestamp DESC;" ``` (Returns 0 rows on `main`, 1 row with this PR applied (after one producer run). ### Notes 1. The unit test `test_make_task_inlets_outlets` in `tests/test_dagbuilder.py` was previously skipped on Airflow 3 (it could not pass with the gate in place). This PR re-enables it. Assertions use `Dataset` imported from `dagfactory.dagbuilder`, which is already aliased to `airflow.sdk.Asset` on AF3. 2. A symmetric bug exists on the *consumer* side: YAML `schedule:` declared as a list of bare URI strings is not converted to `Asset` on Airflow 3. Workaround in the consumer fixture above is to declare schedule entries with explicit `__type__: airflow.sdk.Asset`. Happy to send that fix as a follow-up PR if maintainers prefer; kept out of scope here to keep the diff tightly focused on `adjust_general_task_params`. Co-authored-by: Arushee Verma <ArusheeVerma@users.noreply.github.com>
…in schedule to Asset (companion to astronomer#737) (astronomer#738) ## Summary Companion to astronomer#737. Together, these two PRs restore end-to-end asset-based scheduling for DAGs authored via dag-factory YAML on Airflow 3. `DagBuilder.configure_schedule`'s Airflow 3 branch passed the `schedule` value through to the DAG constructor unmodified. So a YAML schedule of the form ```yaml my_consumer_dag: schedule: - "s3://bucket/data.parquet" ``` …reaches Airflow 3 as a `list[str]`, and Airflow 3's `_default_timetable` validator (in `airflow.sdk.definitions.dag`) raises: ``` ValueError: All elements in 'schedule' should be either assets, asset references, or asset aliases ``` ## Changes - `dagfactory/dagbuilder.py`: in the Airflow 3 branch of `configure_schedule`, when `schedule` is a list, convert each string element to a `Dataset`. The `Dataset` symbol imported at the top of `dagbuilder.py` already aliases to `airflow.sdk.definitions.asset.Asset` on Airflow 3 (and to `airflow.datasets.Dataset` on Airflow 2), so the conversion is correct on both versions. - The `isinstance(uri, str)` guard preserves already-typed items (e.g. `airflow.sdk.AssetAlias` entries declared via `__type__`) so mixed lists keep working — same defensive pattern used in astronomer#601 and mirrored from astronomer#737. - `tests/test_dagbuilder.py`: three new tests under `TestConfigureSchedule`: - `test_configure_schedule_airflow3_list_of_uri_strings` — bare URI list converts to `[Asset, Asset]` - `test_configure_schedule_airflow3_list_mixed_string_and_object` — `isinstance` guard preserves pre-built objects - `test_configure_schedule_airflow3_empty_list` — empty list round-trips as empty list ## Test plan - [x] Three new unit tests pass. - [x] Manually validated end-to-end on Airflow 3.2.1, with **both** this PR and astronomer#737 applied: - Consumer DAG declared with `schedule: ["file:///tmp/dagfactory_demo_asset"]` (bare URI list) loads cleanly — verified via `airflow dags list`. - Triggered the producer DAG (which has `outlets: ["file:///tmp/dagfactory_demo_asset"]`). - `asset_event` table records the producer's task success: `id=1, source_dag_id=asset_producer, source_task_id=produce`. - Consumer DAG auto-triggered immediately, with run_id prefix `asset_triggered__…` and final state `success`. - [x] Confirmed without this fix (consumer YAML using bare URI list), the consumer DAG fails parsing on Airflow 3 with the `ValueError` quoted above. ## Detailed testing evidence ### TL;DR comparison | Signal | BEFORE (without this fix) | AFTER (with this fix) | |---|---|---| | Consumer YAML using `schedule: ["file:///tmp/dagfactory_demo_asset"]` | DAG **fails to load** at parse time | DAG loads cleanly | | Error raised | `ValueError: All elements in 'schedule' should be either assets, asset references, or asset aliases` | None | | Consumer DAG visible in `airflow dags list` | No (parse error) | Yes | | Producer task success → AssetEvent emitted (with astronomer#737 also applied) | Event emitted but no consumer is subscribed because consumer never loaded | Event emitted, consumer is subscribed | | Consumer auto-triggered after producer success | No (consumer doesn't exist) | Yes — within ~1 second | --- ### Hypothesis `configure_schedule`'s Airflow 3 branch passes the `schedule` value to the DAG constructor unmodified. On Airflow 3, the `DAG` class validates schedule lists via `_default_timetable` and raises `ValueError` if any list element is not a `BaseAsset`. Therefore a YAML `schedule: ["s3://..."]` is unloadable on Airflow 3 today. The fix wraps each string element in `Dataset(...)`, which the conditional import at [`dagfactory/dagbuilder.py:34-37`](https://github.com/astronomer/dag-factory/blob/main/dagfactory/dagbuilder.py#L34-L37) aliases to `airflow.sdk.definitions.asset.Asset` on Airflow 3. The wrapped objects satisfy the `BaseAsset` check. --- ### BEFORE evidence (the failure) **The validator that rejects bare URI strings on Airflow 3** — [`apache/airflow` v3.2.1, `task-sdk/src/airflow/sdk/definitions/dag.py` lines 627-643](https://github.com/apache/airflow/blob/3.2.1/task-sdk/src/airflow/sdk/definitions/dag.py#L627-L643): ```python @timetable.default def _default_timetable(instance: DAG) -> BaseTimetable | CoreTimetable: schedule = instance.schedule # ... if isinstance(schedule, Collection) and not isinstance(schedule, str): if not all(isinstance(x, BaseAsset) for x in schedule): raise ValueError( "All elements in 'schedule' should be either assets, asset references, or asset aliases" ) return AssetTriggeredTimetable(AssetAll(*schedule)) return _create_timetable(schedule, instance.timezone) ``` **The YAML that triggers it on Airflow 3 without this fix:** ```yaml asset_consumer: description: "Consumer DAG: schedule is a bare URI string list" schedule: - "file:///tmp/dagfactory_demo_asset" tasks: - task_id: consume operator: airflow.providers.standard.operators.bash.BashOperator bash_command: "echo 'CONSUMER FIRED'" ``` **To reproduce the BEFORE failure**: check out `main`, load that YAML through dag-factory on Airflow 3.x, and you'll see (in the dag-processor logs): ``` ValueError: All elements in 'schedule' should be either assets, asset references, or asset aliases ``` The DAG never appears in `airflow dags list` and consumer-side asset scheduling is fully blocked. The repository's own example [`dev/dags/datasets/example_dag_datasets.yml`](https://github.com/astronomer/dag-factory/blob/main/dev/dags/datasets/example_dag_datasets.yml) uses exactly this `schedule: [...]` form and is currently broken on Airflow 3. --- ### AFTER evidence (validated end-to-end on Airflow 3.2.1) **Consumer fixture used** ```yaml asset_consumer: description: "Consumer DAG: schedule is a bare URI string list (testing the Airflow 3 schedule conversion fix)." schedule: - "file:///tmp/dagfactory_demo_asset" tasks: - task_id: consume operator: airflow.providers.standard.operators.bash.BashOperator bash_command: "echo 'CONSUMER FIRED at' $(date -u +%FT%TZ)" ``` **1) Both DAGs load successfully** (`airflow dags list`): ``` dag_id | fileloc | owners | is_paused asset_consumer | /opt/airflow/dags/asset_consumer.py | experiment | False asset_producer | /opt/airflow/dags/asset_producer.py | experiment | False ``` **2) Producer task emits an `AssetEvent`** (live `asset_event` table after one producer trigger): ```sql SELECT ae.id, ae.source_dag_id, ae.source_task_id, ae.source_run_id, a.uri, a.name, ae.timestamp FROM asset_event ae LEFT JOIN asset a ON a.id = ae.asset_id ORDER BY ae.timestamp DESC LIMIT 5; ``` Output: ``` id | source_dag_id | source_task_id | source_run_id | uri | name | timestamp ----+----------------+----------------+------------------------------------------+-----------------------------------+-----------------------------------+------------------------------- 1 | asset_producer | produce | manual__2026-05-03T18:01:17.905819+00:00 | file:///tmp/dagfactory_demo_asset | file:///tmp/dagfactory_demo_asset | 2026-05-03 18:01:19.298442+00 ``` Note `name == uri` — that's exactly what dag-factory's `Dataset(uri)` constructor produces (URI is used as both `name` and `uri` on Airflow 3's `Asset`). The matching `asset` row is what enables the consumer to find this event. **3) Consumer DAG auto-triggered ~0.7s after producer success** (`airflow dags list-runs asset_consumer`): ``` dag_id | run_id | state | start_date | end_date asset_consumer | asset_triggered__2026-05-03T18:01:19.303196+00:00_K24jYRYX | success | 2026-05-03T18:01:19.989371+00:00 | 2026-05-03T18:01:20.882142+00:00 ``` The `asset_triggered__` prefix in `run_id` is Airflow 3's marker for runs created by the scheduler in response to an `AssetEvent` — distinguishable from `manual__` (manual triggers) or `scheduled__` (cron-driven). Within ~0.7s of the producer's task succeeding, the scheduler created this run and it completed successfully. ### Unit test evidence Three new tests added under `TestConfigureSchedule` in [`tests/test_dagbuilder.py`](https://github.com/astronomer/dag-factory/blob/main/tests/test_dagbuilder.py). They use the existing `patch_airflow_version` fixture so they exercise the Airflow 3 code path even when run in an Airflow 2 CI environment. What each test proves: - **Test 1 (the core conversion)**: validates the happy path — given a list of bare URI strings, every element is wrapped in `Dataset` (which is `Asset` on Airflow 3 via the conditional import). This is the case that fixes the `ValueError`. - **Test 2 (the guard)**: validates the `isinstance(uri, str)` defensive check — pre-built `Dataset`/`Asset` objects in the list pass through unchanged, so users mixing bare URIs with explicit `__type__: airflow.sdk.AssetAlias` entries don't get a double-wrap or a crash. Same defensive pattern PR astronomer#601 introduced for outlets. - **Test 3 (the empty list)**: validates the boundary condition — an empty schedule list produces an empty list, doesn't crash on iteration. ## Pre-commit / lint validation Per the [contributing guide](https://github.com/astronomer/dag-factory/blob/main/docs/contributing/howto.md#pre-commit-and-linting), ran the same checks `pre-commit run --all-files` would run, against this PR's diff. All clean. | Hook (from `.pre-commit-config.yaml`) | Pinned version | Local version run | Result on this PR's diff | |---|---|---|---| | `ruff` (`astral-sh/ruff-pre-commit`) | `v0.6.9` | `0.15.12` (newer = stricter) | clean | | `black` (`psf/black`) | `26.3.1` | `26.3.1` (exact match) | clean | | `codespell` (`codespell-project/codespell`) | `v2.2.4` | `2.4.2` (newer = stricter) | clean | | trailing-whitespace, end-of-file-fixer, mixed-line-ending (`pre-commit/pre-commit-hooks`) | `v5.0.0` | n/a (file-content check) | clean | ## Related - Fixes the consumer half of issue astronomer#718. - Sibling PR: astronomer#737 (producer-side outlets). - Reuses the defensive pattern from astronomer#601. Co-authored-by: Arushee Verma <ArusheeVerma@users.noreply.github.com>
…ath (astronomer#725) Argument `default_args_config_path` is renamed to `defaults_config_path`, while the old argument is still mentioned in multiple docs and codes. This PR corrects the docs and codes, ensuring this argument is fully renamed.
Airflow supports use of {decorated_task}[{xcom_key}] for xcoms that are
not return_value
[link](https://airflow.apache.org/docs/apache-airflow/stable/tutorial/taskflow.html#the-big-picture-a-taskflow-pipeline)
Here I support for fetching upstream task's xcom from decorated task
with `multiple_outputs=True`
The following example has already been tested in Airflow 3.1.0 (and
should also works in Airflow 3)
```
example_multiple_outputs_taskflow:
schedule: null
render_template_as_native_obj: True
catchup: false
tags: [example, test]
tasks:
- task_id: collect
multiple_outputs: true
decorator: airflow.decorators.task
python_callable: sample.collect
- task_id: echo1
decorator: airflow.decorators.task
python_callable: sample.echo
value: "+collect['key1']"
- task_id: echo2
decorator: airflow.decorators.task
python_callable: sample.echo
value: "+collect['key2']"
```
…stronomer#734) ## Summary Fixes astronomer#728. When running DAG Factory with Airflow 3.1.0+, the warning `"The sla_miss_callback has been removed in Airflow 3.1.0."` was being logged on **every DAG parse**, even when `sla_miss_callback` was never configured by the user. **Root cause:** In `dagbuilder.py`, the warning was emitted unconditionally inside the callback-type loop whenever `sla_miss_callback` was iterated — regardless of whether it was present in the DAG config. **Fix:** The warning is now only emitted if `sla_miss_callback` (or its `_name`/`_file` variant) is actually present in the DAG config. The `continue` that prevents registering the removed callback still fires unconditionally, preserving correct runtime behavior. ## Changes - `dagfactory/dagbuilder.py` — Guard the warning behind a config presence check - `tests/test_dagbuilder.py` — Added two tests: - `test_sla_miss_callback_no_warning_when_not_configured` — asserts no warning when `sla_miss_callback` is absent - `test_sla_miss_callback_warning_when_configured` — asserts exactly one warning when `sla_miss_callback` is present ## Test plan ```bash python -m pytest tests/test_dagbuilder.py -m callbacks -v ``` --------- Co-authored-by: AI Assistant <bot@antigravity.dev>
support DatasetOrTimeSchedule
add processing key "timetable" for schedule with key "datasets"
example:
```
schedule:
datasets: "((s3://dataset_custom_1 & s3://dataset_custom_2) | s3://dataset_custom_3)"
timetable:
__type__: airflow.timetables.trigger.CronTriggerTimetable
cron: "0 1 * * 5"
timezone: UTC
```
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…ro Hosted (astronomer#613) When running Airflow 3 on Astro Hosted with DAG Versioning (https://www.astronomer.io/docs/astro/dag-versioning) enabled, DAGs are packaged into versioned bundles. In this model, hardcoded absolute paths (e.g. /usr/local/airflow/dags/) and environment variable fallbacks (CONFIG_ROOT_DIR) are not reliable — the bundle's location on disk is managed by Astro and not guaranteed to match the old path assumption. Using Path(__file__).resolve().parent resolves the YAML config path relative to the DAG file itself, which works correctly regardless of where Astro places the bundle. **Changes** - Replace os.getenv("CONFIG_ROOT_DIR", "/usr/local/airflow/dags/") with Path(__file__).resolve().parent across all dev/dags/ example loaders - Bump dev Dockerfile base image to astrocrpublic.azurecr.io/runtime:3.1-5 **Test plan** - Deploy to Astro Hosted with DAG Versioning enabled and verify all example DAGs parse and load correctly --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
…stronomer#654) This PR adds support for `.airflowignore` file in `load_yaml_dags()` function, allowing users to exclude specific YAML files from being loaded. This feature follows the same format as Airflow's `.airflowignore` file and supports glob-style pattern matching. closes astronomer#527 - **`.airflowignore` file support**: `load_yaml_dags()` now automatically reads and respects `.airflowignore` files in the DAGs folder - **Pattern matching**: Supports glob-style patterns including: - Filename patterns: `test_*.yml`, `*_test.yaml` - Directory patterns: `old_dags/*.yaml`, `backup/**/*.yml` - Recursive matching: `**` for matching files in subdirectories at any depth - **Comment support**: Lines starting with `#` are treated as comments and ignored - Added `_load_airflowignore()` function to read and parse `.airflowignore` files - Added `_should_ignore_file()` function to check if a file matches any ignore patterns - Integrated ignore logic into `load_yaml_dags()` to filter files before loading - Patterns are matched against both filename and relative path from `dags_folder` - Files outside `dags_folder` (e.g., symlinks) fall back to filename-only matching Create a `.airflowignore` file in your DAGs folder: ``` test_*.yml *_test.yaml old_dags/*.yaml backup/**/*.yml ``` The `load_yaml_dags()` function will automatically respect these patterns: ```python from dagfactory import load_yaml_dags load_yaml_dags(globals_dict=globals(), dags_folder="/path/to/dags") ``` - Added comprehensive test suite covering: - Loading ignore patterns from `.airflowignore` file - Pattern matching by filename and relative path - Recursive matching with `**` patterns - Comment and whitespace handling - Integration tests with actual DAG loading scenarios Fully backward compatible: If no `.airflowignore` file exists, the function behaves exactly as before.
…stronomer#736) ## Summary - Fixes astronomer#679: on Airflow 2.x with `apache-airflow-providers-standard` installed, `airflow.operators.python.PythonOperator` and `airflow.providers.standard.operators.python.PythonOperator` are **different, unrelated classes**. The single `try/except` import in `dagbuilder.py` bound only one of them, so the `issubclass` check in `make_task` missed YAMLs that referenced the other module path. The result was `python_callable_name`/`python_callable_file` never being resolved into `python_callable`, producing `airflow.exceptions.AirflowException: missing keyword argument 'python_callable'` at parse time. - Now imports both module paths when available and checks `issubclass` against the union (`PYTHON_CALLABLE_CLASSES`). Both YAML import paths work regardless of which providers are installed. - Adds a parametrized regression test that exercises both `airflow.operators.python.PythonOperator` and `airflow.providers.standard.operators.python.PythonOperator`, with skip guards for AF3 (no core path) and providers-standard not installed. - Adds a dedicated CI matrix row — `py3.12 / af2.11 / providers-standard==1.9.1` — wired into the unit-tests workflow so this exact combination is always covered. Achieved by adding a `providers-standard` Hatch matrix variable, a `{matrix:providers-standard:}` substitution in `pre-install-commands`, an optional 3rd arg in `scripts/test/pre-install-airflow.sh`, and an `include:` row + computed env name in `cicd.yaml`. ## Verification Reproduced the bug locally on Airflow 2.11.0 + `apache-airflow-providers-standard==1.9.1` + `apache-airflow-providers-common-compat==1.14.3` (the reported environment), confirmed both module paths fail before the fix and succeed after. Ran the existing `tests/test_dagbuilder.py` suite on this exact combo: 67 pass, 5 skip, 0 regressions. | Scenario | Result | |---|---| | YAML uses `airflow.operators.python.PythonOperator`, before fix | `AirflowException: missing keyword argument 'python_callable'` | | YAML uses `airflow.operators.python.PythonOperator`, after fix | works | | Regression test, providers-standard installed | both params PASS | | Regression test, providers-standard absent | `[providers-standard]` SKIPPED, `[core]` PASS | 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…low support (astronomer#740) The howto referenced an Airflow 2.5 hatch env that no longer exists and capped Python at 3.12 over a stale `google-re2` 3.13 wheel issue; the Astro quick-start still listed Python 3.9. Update them to match the current supported range (Python 3.10–3.14, Airflow 2.9+). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ronomer#739) Follow-up: astronomer#613 In Astro Hosted, DAG bundles deploy to a timestamped path (/tmp/airflow/dag_bundles/.../dags/) that is not in sys.path, so helper modules (hacker_news, pypi_stats, sample) are not importable. Each file now inserts its own directory via __file__ so the path resolves correctly regardless of the deployment timestamp. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
## [1.1.0] - 2026-05-07 ### Breaking Changes - Drop Python 3.9 Support by @pankajastro in [astronomer#600](astronomer#600) - Drop support for Airflow < 2.9 by @pankajastro in [astronomer#711](astronomer#711) ### Added - Add user_defined_macros support by @gyli in [astronomer#693](astronomer#693) - feat: support key: +{task}[{xcom_key}] use by @yanshil in [astronomer#593](astronomer#593) - Support dataset timetable by @AppKiv in [astronomer#599](astronomer#599) - Add support for `.airflowignore` file to exclude specific YAML files by @viiccwen in [astronomer#654](astronomer#654) - Add Airflow 3.1 support by @pankajastro in [astronomer#720](astronomer#720) - Add Airflow 3.2 to CI test matrix by @pankajastro in [astronomer#723](astronomer#723) ### Improved - Change log info to warning by @pankajastro in [astronomer#590](astronomer#590) - Unify import_from_string methods by @gyli in [astronomer#694](astronomer#694) - Ensure default_args_config_path is fully renamed to defaults_config_path by @gyli in [astronomer#725](astronomer#725) ### Fixed - Fix main branch tests by @tatiana in [astronomer#638](astronomer#638) - fix: support DatasetAlias in outlets by @AppKiv in [astronomer#601](astronomer#601) - Import TaskGroup in the Airflow 3 way if available by @gyli in [astronomer#706](astronomer#706) - Handle removal of airflow.utils.module_loading in Airflow 3.1+ by @pankajastro in [astronomer#712](astronomer#712) - Fix import for dataset and MappedOperator by @pankajastro in [astronomer#713](astronomer#713) - Fix Asset-based Scheduling on Airflow 3: Convert URI strings to Asset for inlets/outlets by @ArusheeVerma in [astronomer#737](astronomer#737) - Fix Consumer-side Asset Scheduling on Airflow 3: Convert URI strings in schedule to Asset by @ArusheeVerma in [astronomer#738](astronomer#738) - Fix unconditional sla_miss_callback removal warning in Airflow 3.1+ by @manipatnam in [astronomer#734](astronomer#734) - Fix python_callable resolution on Airflow 2 with providers-standard by @pankajkoti in [astronomer#736](astronomer#736) ### Docs - Fix markdown lint issues due to dead links by @viiccwen in [astronomer#642](astronomer#642) - Update docs copyright year to 2026 by @pankajkoti in [astronomer#698](astronomer#698) - Add timezone references by @sri-codes-python in [astronomer#701](astronomer#701) - Add ``items`` as a reserved key in doc by @gyli in [astronomer#726](astronomer#726) - docs: align contributor and quick-start docs with current Python/Airflow support by @pankajastro in [astronomer#740](astronomer#740) ### Other Changes - Pin structlog to fix the deploy docs job by @pankajastro in [astronomer#608](astronomer#608) - Remove structlog pin by @pankajkoti in [astronomer#656](astronomer#656) - Avoid error we're seeing in PRs from external contributors by @tatiana in [astronomer#660](astronomer#660) - Replace tags with SHAs and update to latest version by @fbuechel92 in [astronomer#663](astronomer#663) - Add 7-day cooldown to Dependabot config by @pankajkoti in [astronomer#686](astronomer#686) - Ignore simple_auth_manager_passwords.json.generated from git by @gyli [astronomer#695](astronomer#695) - Ignore more editor local settings folder by @gyli [astronomer#697](astronomer#697) - Add Python 3.14 to CI test matrix by @pankajastro in [astronomer#731](astronomer#731) - Add AGENTS.md for AI coding agents by @gyli in [astronomer#717](astronomer#717) - fix: use __file__-relative paths in dev examples for Airflow 3 on Astro Hosted by @pankajastro in [astronomer#613](astronomer#613) - Fix sys.path for DAG helper modules in Astro bundle environments by @pankajastro in [astronomer#739](astronomer#739) closes: astronomer#733 closes: astronomer#729 --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
pankajastro
pushed a commit
that referenced
this pull request
May 8, 2026
Re-opened #719 after messing up with the git history and addressing feedbacks ## Summary - Introduces `AIRFLOW__DAG_FACTORY__STRICT_MODE` (default: `False`) that, when enabled, raises a `DagFactoryConfigException` for any DAG that fails to build — while still registering all successfully built DAGs. - `build_dags()` now catches each DAG independently and returns `(dags, build_errors)`, so a single broken DAG in a multi-DAG YAML never prevents the others from loading. - In folder-scan mode, each YAML file is processed independently — a broken file does not prevent other files from loading. ## Motivation DAGs silently disappearing from the Airflow UI when a YAML config has a parsing error is hard to debug. Strict mode surfaces these errors as Airflow "Broken DAG" import errors visible in the UI and CI/CD pipelines, while still loading all valid DAGs. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hey,
In our company we had some issues with DAGs suddenly disappearing from the UI without any error being spotted in the CI/CD and after some investigation, it is because there is currently exception being raised if there are any issues in the YAML file of the YAML file DAGs.
This PR tries to address that, creating a
strict modeflag that, when active, will raise exceptions for the troublemakers, which would at least make it easier for us to spot DAG factory parsing errors during CI/CD and keep stakeholders happy, since it is not a very easy problem to debug once it occurs.Results
It is working fine in our setup and we are using this branch for time time being