Improve Agent Injector Vault Address Environment Variable Options

[rgarcia6520]

Describe the bug
In environments where we are utilizing vault as part of this helm-chart but using Istio LoadBalancers, or AWS ELBs forwarding traffic to the Vault server the included agent injector deployment is not able to get the correct VAULT_ADDR with the existing options to set AGENT_INJECT_VAULT_ADDR

To Reproduce
Steps to reproduce the behavior:

1. Install chart
2. Create secret and serviceAccount and all included options to configure another deployment to utilize a secret from Vault.
3. Add kubernetes labels to inject deployment so it can utilize vault secrets
4. Pod is never able to communicate with Vault because we use our own cert and (E)LB and VAULT_ADDR is set to https://vault-vault.vault.svc:8200

Expected behavior
Agent Injector VAULT_ADDR endpoint is configurable

Environment

• Kubernetes version: 1.23.8
  • Distribution or cloud vendor (OpenShift, EKS, GKE, AKS, etc.): RKE2
  • Other configuration options or runtime services (istio, etc.): istio 1.14.2
• vault-helm version: 0.21.0

Chart values:

global:
  tlsDisable: false
server:
  extraEnvironmentVars:
    VAULT_SKIP_VERIFY: "true"
    VAULT_LOG_FORMAT: "json"
  ha:
    enabled: true 
    replicas: 3
    apiAddr: "https://vault.example.com"
    raft:
      enabled: true
      setNodeId: true
      config: |
        ui = true
        seal "awskms" {
          region     = "us-gov-west-1"
          kms_key_id = "XXXXXXXXXXXX"
          endpoint   = "https://kms.us-gov-west-1.amazonaws.com"
        }
        listener "tcp" {
          tls_disable = false
          address = "[::]:8200"
          cluster_address = "[::]:8201"
          tls_cert_file = "/vault/tls/tls.crt"
          tls_key_file  = "/vault/tls/tls.key"
          telemetry {
            unauthenticated_metrics_access = true
          }
        }

        storage "raft" {
          path = "/vault/data"

          retry_join {
            leader_api_addr = "https://vault-vault-0.vault-vault-internal:8200"
            leader_client_cert_file = "/vault/tls/tls.crt"
            leader_client_key_file = "/vault/tls/tls.key"
            leader_tls_servername = "example.com"
          }
  
          retry_join {
            leader_api_addr = "https://vault-vault-1.vault-vault-internal:8200"
            leader_client_cert_file = "/vault/tls/tls.crt"
            leader_client_key_file = "/vault/tls/tls.key"
            leader_tls_servername = "vault.example.com"
          }
  
          retry_join {
            leader_api_addr = "https://vault-vault-2.vault-vault-internal:8200"
            leader_client_cert_file = "/vault/tls/tls.crt"
            leader_client_key_file = "/vault/tls/tls.key"
            leader_tls_servername = "vault.example.com"
          }
        }

        telemetry {
          prometheus_retention_time = "24h"
          disable_hostname = true
        }

        service_registration "kubernetes" 
  volumes:
  - name: tls
    secret:
      secretName: vault-tls
  volumeMounts:
  - name: tls
    mountPath: "/vault/tls"
    readOnly: true
  dataStorage:
    enabled: true
    size: 50Gi
    mountPath: "/vault/data"
    accessMode: ReadWriteOnce

Additional context
Add any other context about the problem here.

[dtrts]

+1 on this .We are facing a similar issue as we need to customize the AGENT_INJECT_VAULT_ADDR value to support our TLS setup but without disabling the vault server deployment!

[jr200]

+1 here too.

Looking at the helm chart manifest, I see:

            - name: AGENT_INJECT_VAULT_ADDR
            {{- if .Values.global.externalVaultAddr }}
              value: "{{ .Values.global.externalVaultAddr }}"
            {{- else if .Values.injector.externalVaultAddr }}
              value: "{{ .Values.injector.externalVaultAddr }}"
            {{- else }}
              value: {{ include "vault.scheme" . }}://{{ template "vault.fullname" . }}.{{ include "vault.namespace" . }}.svc:{{ .Values.server.service.port }}
            {{- end }}

But in the values.yaml,

section injector:

  # Deprecated: Please use global.externalVaultAddr instead.
  externalVaultAddr: ""

section global:

  # External vault server address for the injector and CSI provider to use.
  # Setting this will disable deployment of a vault server.
  externalVaultAddr: ""

At some point I guess the plan is to remove injector.externalVaultAddr. When this is done, can you provide an alternative method of setting the address of vault in the injector.

I want to set the external Vault Addr for the injector (for tls/cert reasons), and also deploy the server. There are cases where its not a mutually exclusive setting.

Thanks,

Edit:
I think its not currently possible to deploy vault server set and set an externalVaultAddr (i.e., configure AGENT_INJECT_VAULT_ADDR . I had to amend my ansible role to post-json-patch the AGENT_INJECT_VAULT_ADDR env variable.