Improve Agent Injector Vault Address Environment Variable Options #789

[baurmatt]

(Copy from hashicorp/vault-helm#789)

Describe the bug In environments where we are utilizing vault as part of this helm-chart but using Istio LoadBalancers, or AWS ELBs forwarding traffic to the Vault server the included agent injector deployment is not able to get the correct VAULT_ADDR with the existing options to set AGENT_INJECT_VAULT_ADDR

To Reproduce Steps to reproduce the behavior:

1. Install chart

2. Create secret and serviceAccount and all included options to configure another deployment to utilize a secret from Vault.

3. Add kubernetes labels to inject deployment so it can utilize vault secrets

4. Pod is never able to communicate with Vault because we use our own cert and (E)LB and VAULT_ADDR is set to `https://vault-vault.vault.svc:8200`

Expected behavior Agent Injector VAULT_ADDR endpoint is configurable

Environment

* Kubernetes version: 1.23.8

  * Distribution or cloud vendor (OpenShift, EKS, GKE, AKS, etc.): RKE2
  * Other configuration options or runtime services (istio, etc.): istio 1.14.2

* vault-helm version: 0.21.0

Chart values:

global:
tlsDisable: false
server:
extraEnvironmentVars:
VAULT_SKIP_VERIFY: "true"
VAULT_LOG_FORMAT: "json"
ha:
enabled: true
replicas: 3
apiAddr: "https://vault.example.com"
raft:
enabled: true
setNodeId: true
config: |
ui = true
seal "awskms" {
region = "us-gov-west-1"
kms_key_id = "XXXXXXXXXXXX"
endpoint = "https://kms.us-gov-west-1.amazonaws.com"
}
listener "tcp" {
tls_disable = false
address = "[::]:8200"
cluster_address = "[::]:8201"
tls_cert_file = "/vault/tls/tls.crt"
tls_key_file = "/vault/tls/tls.key"
telemetry {
unauthenticated_metrics_access = true
}
}

    storage "raft" {
      path = "/vault/data"

      retry_join {
        leader_api_addr = "https://vault-vault-0.vault-vault-internal:8200"
        leader_client_cert_file = "/vault/tls/tls.crt"
        leader_client_key_file = "/vault/tls/tls.key"
        leader_tls_servername = "example.com"
      }

      retry_join {
        leader_api_addr = "https://vault-vault-1.vault-vault-internal:8200"
        leader_client_cert_file = "/vault/tls/tls.crt"
        leader_client_key_file = "/vault/tls/tls.key"
        leader_tls_servername = "vault.example.com"
      }

      retry_join {
        leader_api_addr = "https://vault-vault-2.vault-vault-internal:8200"
        leader_client_cert_file = "/vault/tls/tls.crt"
        leader_client_key_file = "/vault/tls/tls.key"
        leader_tls_servername = "vault.example.com"
      }
    }

    telemetry {
      prometheus_retention_time = "24h"
      disable_hostname = true
    }

    service_registration "kubernetes"

volumes:

• name: tls
  secret:
  secretName: vault-tls
  volumeMounts:
• name: tls
  mountPath: "/vault/tls"
  readOnly: true
  dataStorage:
  enabled: true
  size: 50Gi
  mountPath: "/vault/data"
  accessMode: ReadWriteOnce

Additional context Add any other context about the problem here.

[baurmatt]

We have the same issue and I like to solve it for the OpenBao chart.

While investigating, I found that the main issue is, that .Values.injector.externalVaultAddr switch openbao.mode to external and this disables the installation of the server.

openbao-helm/charts/openbao/templates/_helpers.tpl

Line 162 in 091cb3a

{{- if or (.Values.injector.externalVaultAddr) (.Values.global.externalVaultAddr) (.Values.global.externalBaoAddr) -}}

openbao-helm/charts/openbao/templates/server-statefulset.yaml

Line 7 in 091cb3a

{{- if ne .mode "external" }}

While I understand that .Values.global.externalVaultAddr/externalBaoAddr influences the operations mode of the OpenBao server, I have trouble finding an argument why .Values.injector.externalVaultAddr should.

Any objections removing .Values.injector.externalVaultAddr from the openbao.mode function?

Additionally one could rename .Values.injector.externalVaultAddr to Values.injector.externalBaoAddr.

This would IMHO match what users expect if global and services specific variables are available: Service specific take preference and override global one.

[olivergrabinski]

In this example the config.hcl has:

"vault"  {
      "address" = "https://vault.demo.svc.cluster.local:8200"
      "ca_cert" = "/vault/tls/ca.crt"
      "client_cert" = "/vault/tls/client.crt"
      "client_key" = "/vault/tls/client.key"
    }

However, in my attempts the address always falls back to the VAULT_ADDR. Is there a way to make it so that the Agent doesn't come pre-populated with the VAULT_ADDR env var? Is this related to this issue? Sorry if it is not.

[baurmatt]

@olivergrabinski Can you share your Helm values? 🤔 My PR (#158) gives more flexibility of the address value, but removing it is currently not planned. Why would you like to remove it completely?

[olivergrabinski]

@baurmatt Thanks the reply, the values I used were

global:
  enabled: true

server:
  enabled: false

injector:
  enabled: true
  replicas: 1
  webhook:
    failurePolicy: Ignore

And in my config.hcl file I have

vault {
  address = "https://my.bao.address.org:8200"
}

With this setup, I get an connection error because the bao agent is trying to connect to openbao-agent-injector.openbao-agent-injector.svc.

2026-03-24T10:42:35.910Z [INFO]  agent.auth.handler: authenticating                           │
2026-03-24T10:42:36.213Z [ERROR] agent.auth.handler: error authenticating: error="Put \"http://openbao-injector.openbao-injector.svc:8200/v1/auth/redacted/login\": dial tcp: lookup openbao-injector.openbao-injector.svc on 10.43.0.10:53: no such host" backoff=1m4.57s

This address must be derived from the Helm release name as far as I understand.

As for the use case, I have a situation where there are two OpenBao instances, and I wanted to have the init agent run with one (to fetch a static secret), and the agent sidecar with the other (to fetch a dynamic token). But both the init and sidecar agents get the VAULT_ADDR, which takes precedence.

In any case, I also just found it a bit confusing that the address can be set in the config but it will be always overridden? 🤔 Unless that's something that changed after the fork.

[baurmatt]

This address must be derived from the Helm release name as far as I understand.

That's the default, yes ->

openbao-helm/charts/openbao/templates/injector-deployment.yaml

Line 67 in 091cb3a

value: {{ include "openbao.scheme" . }}://{{ template "openbao.fullname" . }}.{{ include "openbao.namespace" . }}.svc:{{ .Values.server.service.port }}

But you can override it with global.externalBaoAddr or injector.externalVaultAddr. This should work fine with the current version of the chart, as you disabled the server component. This ticket (and the linked PR) is intended to better support usecases where server and injector are supposed to deployed to the same cluster. In this case we're currently limited to the autogenerated in-cluster URL. The PR makes this configurable.

As for the use case, I have a situation where there are two OpenBao instances, and I wanted to have the init agent run with one (to fetch a static secret), and the agent sidecar with the other (to fetch a dynamic token). But both the init and sidecar agents get the VAULT_ADDR, which takes precedence.
In any case, I also just found it a bit confusing that the address can be set in the config but it will be always overridden? 🤔 Unless that's something that changed after the fork.

That sounds more complicated. I'm not yet deep into injector deployments and can't answer this question, sorry.

[eyenx]

Hi there, I haven't read yet through the whole issue, but pls be aware of this issue as well #113

[baurmatt]

TIL: global is a Helm internal thing 🤯 Thanks for the pointer! But since you don't want to get rid of the global variable in the near future, I think this issue/PR makes still sense.