Remove automatic KUBERNETES_ENVIRONMENT_VARIABLES and KUBERNETES_SECRETS prefixes from chart helpers

[uplsh580]

Description

Remove the automatic addition of the AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__ and AIRFLOW__KUBERNETES_SECRETS__ prefixes from the container_extra_envs and custom_airflow_environment helper functions.

Related Issue

• Issue: AIRFLOW__API__EXPOSE_CONFIG when set to 'non-sensitive-only' is exposing some secrets #60668 (comment)

Changes

• Modified container_extra_envs helper to stop automatically prepending the AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__ prefix.
• Modified custom_airflow_environment helper to stop automatically prepending both the AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__ prefix (for .Values.env) and the AIRFLOW__KUBERNETES_SECRETS__ prefix (for .Values.secret).

Motivation

• Prevent Unintended Exposure of Sensitive Data

  • Sensitive information (e.g., client_secret) that should be securely handled via secretKeyRef for specific components was being automatically prefixed with AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__.
  • This prefixing causes these variables to be recognized as part of Airflow's internal configuration, leading to their unintended exposure in the Airflow Web UI (under Admin -> Configuration).
  • This occurs even when AIRFLOW__API__EXPOSE_CONFIG is set to non-sensitive-only, creating a security vulnerability. For more details, see the issue description.

• Avoid Unintended Environment Propagation to Workers

  • These configurations (.values.{SOME_COMPONENTS}.env, .values.env, .values.secret) appear to be intended strictly for specific components (e.g., apiServer, Scheduler).
  • However, the previous behavior caused these variables and secrets to be inadvertently passed to worker pods, which may result in unintended configuration conflicts and unexpected side effects in the worker environment.

Migration

If you need to pass environment variables or secrets specifically to Kubernetes Executor worker pods:

• Use the .values.env field:

env:
  - name: my_var
    value: "my_value"

[jscheffl]

We are planning soonish a version 2.0 with cleanups and reduction of scope. I'd assume we should hold-back this PR for the version 2.0 and put the breaking change there.

Otherwise I think it is a good idea not to expose these ecrets and wonder why this had been in there first place. Maybe it never affected me as I an not using KubernetesExecutor but Celery.

[uplsh580]

@jscheffl
I agree that the breaking changes should be held back for version 2.0. To address the issue in the current version without causing any breaking changes, I have submitted a new PR: #60783.

I believe the changes in this PR should be reconsidered when we start working on the 2.0 roadmap, so I will close this one for now. I would appreciate it if you could review the new PR (#60750) instead.

[jscheffl]

@uplsh580 do not worry about failed docs build. This is broken on main and is in fixing in parallel. Unrelated to this PR. (Fixed in #63630)

@jedcunningham okay for you to merge for 1.20?