Add missing support for: securityContexts and containerLifecycleHooks by henry3260 · Pull Request #60677 · apache/airflow

henry3260 · 2026-01-17T05:50:59Z

Keep consistency with the previous PR like #60238

Was generative AI tooling used to co-author this PR?

Yes (please specify the tool below)

Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
When adding dependency, check compliance with the ASF 3rd Party License Policy.
For significant user-facing changes create newsfragment: {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

henry3260 · 2026-01-17T13:13:49Z

@Miretpl To ensure the fallback logic works properly with the standard has check, I believe we should remove the default enabled: false in values.yaml (leaving it unset/nil). Otherwise, the default value overrides the old configuration in tests. WDYT?"

Miretpl · 2026-01-17T18:28:57Z

@henry3260, the change in chart behaviour is by intent (there was a discussion in #59730), so currently, we want to make sure that the behaviour will be the same for every workers.celery/kubernetes work and will be properly communicated.

Behaviour same as in already moved configs means:

If there is a default value specified, the user will need to manually unset the proper field in celery/kubernetes sections (for this, we also want to have a dedicated newsfragment file to make sure that this will be more visible to users in release notes),
If there is no default value specified, the behaviour remains unchanged.

For simple examples for the above points, you can take a look at (also in terms of what is changed, how and where):

If you have further questions, feel free to ask!

Miretpl

As I was here, I've made a quick review of the current state to not be late like last time :D

chart/templates/workers/worker-deployment.yaml

henry3260 · 2026-01-17T18:37:49Z

As I was here, I've made a quick review of the current state to not be late like last time :D

Many thanks for your kind guidance!!!

henry3260 · 2026-01-17T19:14:56Z

@henry3260, the change in chart behaviour is by intent (there was a discussion in #59730), so currently, we want to make sure that the behaviour will be the same for every workers.celery/kubernetes work and will be properly communicated.

Behaviour same as in already moved configs means:

If there is a default value specified, the user will need to manually unset the proper field in celery/kubernetes sections (for this, we also want to have a dedicated newsfragment file to make sure that this will be more visible to users in release notes),

If there is no default value specified, the behaviour remains unchanged.

For simple examples for the above points, you can take a look at (also in terms of what is changed, how and where):

Add workers.celery.replicas field #59730

Add workers.celery.revisionHistoryLimit field #60056

If you have further questions, feel free to ask!

Okay! So we should update the tests by explicitly setting workers.celery.kerberosInitContainer to nil if we want to verify workers.kerberosInitContainer, assuming I understand correctly.

Miretpl · 2026-01-17T21:37:22Z

Okay! So we should update the tests by explicitly setting workers.celery.kerberosInitContainer to nil if we want to verify workers.kerberosInitContainer, assuming I understand correctly.

Depends where. If you are testing the enabled flag behaviour - yes (not always, of course :D), if it is required to test something else, then no

Miretpl · 2026-01-18T11:15:19Z

Hi @henry3260, can I finish this section? I need it for #60420, which I would like to finish and properly tested today

henry3260 · 2026-01-18T11:17:35Z

Hi @henry3260, can I finish this section? I need it for #60420, which I would like to finish and properly tested today

Feel free

henry3260 · 2026-01-18T15:50:44Z

Hi @henry3260, can I finish this section? I need it for #60420, which I would like to finish and properly tested today

Do you need my help to add additional test cases for securityContext and lifecycle?
I can add them in this pr

Miretpl · 2026-01-18T17:00:25Z

In my PR, I will leave securityContexts and lifecycle as they are (as I believe that not using those fields anywhere is a different issue that current implementation in terms of separation and worker sets). I noticed that the kerberosInitContainer is also in pod-template-file, which changes the logic a little, and I'm not sure now in which direction we should go with it. If you want to take a look - it is here #60751

Miretpl · 2026-01-18T17:57:01Z

I think that we came to conclustion in the mentioned PR, so for this one, you would need only add missing support for:

securityContexts
containerLifecycleHooks

under kerberosInitContainer section of workers, workers.kubernetes and workers.celery.

+ of course test cases for each. Will that be good?

henry3260 · 2026-01-18T18:08:16Z

I think that we came to conclustion in the mentioned PR, so for this one, you would need only add missing support for:

securityContexts

containerLifecycleHooks

under kerberosInitContainer section of workers, workers.kubernetes and workers.celery.

of course test cases for each. Will that be good?

looks good!

Miretpl · 2026-02-16T23:04:01Z

Hi @henry3260, getting back to this, we would need to implement the missing support, which I've mentioned somewhere above + test cases for them. If you have any questions, feel free to ask!

henry3260 · 2026-02-19T18:27:13Z

Hi @henry3260, getting back to this, we would need to implement the missing support, which I've mentioned somewhere above + test cases for them. If you have any questions, feel free to ask!

Sorry for late update! I will complete this pr soon.

Miretpl · 2026-02-19T22:29:39Z

No worries. Just letting you know that we can get back to this after 1.19.0 was released, so take your time

henry3260 · 2026-02-22T18:06:59Z

No worries. Just letting you know that we can get back to this after 1.19.0 was released, so take your time

Hi @Miretpl , feel free to share your thoughts when you have time :)

Miretpl

In general looks good! Thanks for the change. Just small nits

+ could you add a test cases in airflow_core/test_worker_sets.py for overwrite of these two parameters by worker sets feature?

chart/files/pod-template-file.kubernetes-helm-yaml

chart/templates/workers/worker-deployment.yaml

helm-tests/tests/helm_tests/airflow_aux/test_pod_template_file.py

helm-tests/tests/helm_tests/airflow_core/test_worker.py

henry3260 · 2026-02-25T02:29:57Z

@Miretpl @jscheffl Thanks!

…apache#60677) * keep consistency * support kerberos init securityContext/lifecycle in workers

keep consistency

249f7d2

henry3260 requested review from dstandish, hussein-awala, jedcunningham and jscheffl as code owners January 17, 2026 05:51

boring-cyborg bot added the area:helm-chart Airflow Helm Chart label Jan 17, 2026

henry3260 mentioned this pull request Jan 17, 2026

Add workers.celery.kerberosInitContainer field #60427

Merged

1 task

henry3260 marked this pull request as draft January 17, 2026 06:51

Miretpl reviewed Jan 17, 2026

View reviewed changes

chart/templates/workers/worker-deployment.yaml Outdated Show resolved Hide resolved

chart/templates/workers/worker-deployment.yaml Outdated Show resolved Hide resolved

Miretpl mentioned this pull request Jan 17, 2026

Fix Compatibility of Celery Worker Sets with Workers Separation #60420

Merged

1 task

Merge branch 'main' into chart-2

e9dc0a6

henry3260 changed the title ~~Keep consistency with the previous PR~~ Add missing support for: securityContexts and containerLifecycleHooks Feb 22, 2026

henry3260 force-pushed the chart-2 branch from e7eb13b to d53fa6e Compare February 22, 2026 18:04

henry3260 marked this pull request as ready for review February 22, 2026 18:05

henry3260 requested a review from bugraoz93 as a code owner February 22, 2026 18:05

Miretpl suggested changes Feb 22, 2026

View reviewed changes

henry3260 force-pushed the chart-2 branch from d53fa6e to 8f3a173 Compare February 24, 2026 06:46

support kerberos init securityContext/lifecycle in workers

183cea2

henry3260 force-pushed the chart-2 branch from 8f3a173 to 183cea2 Compare February 24, 2026 07:24

henry3260 added 2 commits February 24, 2026 15:47

Merge branch 'main' into chart-2

f3cc1a2

Merge branch 'main' into chart-2

25d6278

jscheffl approved these changes Feb 24, 2026

View reviewed changes

Miretpl approved these changes Feb 24, 2026

View reviewed changes

jscheffl added this to the Airflow Helm Chart 1.20.0 milestone Feb 24, 2026

jscheffl merged commit c85246c into apache:main Feb 24, 2026
96 checks passed

henry3260 deleted the chart-2 branch February 25, 2026 06:19

dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026

Add missing support for: securityContexts and containerLifecycleHooks (…

c8940fb

…apache#60677) * keep consistency * support kerberos init securityContext/lifecycle in workers

Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026

Add missing support for: securityContexts and containerLifecycleHooks (…

888a355

…apache#60677) * keep consistency * support kerberos init securityContext/lifecycle in workers

jscheffl mentioned this pull request Mar 17, 2026

Status of testing of Apache Airflow Helm Chart 1.20.0rc1 #63859

Open

33 tasks

Conversation

henry3260 commented Jan 17, 2026

Was generative AI tooling used to co-author this PR?

Uh oh!

henry3260 commented Jan 17, 2026

Uh oh!

Miretpl commented Jan 17, 2026

Uh oh!

Miretpl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

henry3260 commented Jan 17, 2026

Uh oh!

henry3260 commented Jan 17, 2026

Uh oh!

Miretpl commented Jan 17, 2026

Uh oh!

Miretpl commented Jan 18, 2026

Uh oh!

henry3260 commented Jan 18, 2026

Uh oh!

henry3260 commented Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Miretpl commented Jan 18, 2026

Uh oh!

Miretpl commented Jan 18, 2026

Uh oh!

henry3260 commented Jan 18, 2026

Uh oh!

Miretpl commented Feb 16, 2026

Uh oh!

henry3260 commented Feb 19, 2026

Uh oh!

Miretpl commented Feb 19, 2026

Uh oh!

henry3260 commented Feb 22, 2026

Uh oh!

Miretpl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

henry3260 commented Feb 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

henry3260 commented Jan 18, 2026 •

edited

Loading