✨ (kustomize/v2): Add NamespaceTransformer to support multi-namespace…#5299
✨ (kustomize/v2): Add NamespaceTransformer to support multi-namespace…#5299AlirezaPourchali wants to merge 1 commit intokubernetes-sigs:masterfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: AlirezaPourchali The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
|
|
Welcome @AlirezaPourchali! |
|
Hi @AlirezaPourchali. Thanks for your PR. I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
f346fbd to
e22404a
Compare
...ial/testdata/project/dist/chart/templates/prometheus/controller-manager-metrics-monitor.yaml
Outdated
Show resolved
Hide resolved
camilamacedo86
left a comment
There was a problem hiding this comment.
Hi @AlirezaPourchali 👋
Thank you for raising this and for the work on it — much appreciated.
I think the first step should be opening a PR against controller-tools.
See this comment for context:
#5148 (comment)
Once we have the fix in controller-runtime, we can then evaluate what is needed in kubebuilder. Having the fix in place will allow us to properly test and make sure the solution really works.
With the controller-runtime fix, I don’t think we need all of these changes in kubebuilder.
It seems we would only need one additional file:
config/default/namespace-transformer.yaml
(as shown in my example comment).
Also, instead of adding this to the default scaffold, it might be better to document it. This is probably not needed for ~90% of users, and adding a new default file may not help most people.
Possible alternatives:
- Add documentation in the FAQ: https://book.kubebuilder.io/faq
- Or create a small reference/tutorial explaining this use case
We could include a concrete example (similar to the CronJob references), with:
- a small mock project
- e2e tests
- and Helm validation
to ensure the approach works end-to-end.
What do you think? Does this approach make sense?
Thanks again for the contribution 🙏
|
Hi @camilamacedo86 👋 Thank you for the feedback! I wanted to clarify the situation with controller-tools after testing extensively. Controller-gen Already Works Correctly ✅I've verified that controller-gen from controller-tools is NOT the issue - it already correctly generates namespace fields in RBAC manifests. Here's the proof: Test SetupI have a controller with namespace-specific RBAC markers: // +kubebuilder:rbac:groups=apps,namespace=infrastructure,resources=deployments,verbs=get;list;watch;update;patch
// +kubebuilder:rbac:groups="",namespace=production,resources=secrets,verbs=get
// +kubebuilder:rbac:groups=coordination.k8s.io,namespace=production,resources=leases,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups="",namespace=production,resources=events,verbs=get;list;watch;create;update;patch;deleteTest ResultsStep 1: Fresh generation with controller-gen v0.19.0 rm config/rbac/role.yaml
make manifests # Uses controller-gen
cat config/rbac/role.yamlOutput - controller-gen DOES generate namespace fields: ---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: manager-role
namespace: infrastructure # ← Generated by controller-gen
rules:
- apiGroups:
- apps
resources:
- deployments
verbs:
- get
- list
- patch
- update
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: manager-role
namespace: production # ← Generated by controller-gen
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
# ... more rulesStep 2: Testing with Kustomize's hardcoded # config/default/kustomization.yaml
namespace: test-override
namePrefix: osiris-
resources:
- ../rbackustomize build config/defaultResult: The hardcoded Step 3: Testing with NamespaceTransformer + # config/default/namespace-transformer.yaml
apiVersion: builtin
kind: NamespaceTransformer
metadata:
name: namespace-transformer
namespace: test-override
setRoleBindingSubjects: none
unsetOnly: true # Only transform if namespace not already set
fieldSpecs:
- path: metadata/namespace
create: true# config/default/kustomization.yaml
namePrefix: osiris-
resources:
- ../rbac
transformers:
- namespace-transformer.yamlkustomize build config/defaultResult: SUCCESS ✅ - Explicit namespaces are preserved: ---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: osiris-manager-role
namespace: infrastructure # ← Preserved
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: osiris-manager-role
namespace: production # ← PreservedConclusion
Regarding Your SuggestionYou mentioned:
After this testing, I believe controller-tools doesn't need changes. The namespace field is already in the generated YAML.
You're right that only one file is needed per project. However, this PR modifies how Kubebuilder scaffolds new projects - it generates that file automatically during Documentation vs Default ScaffoldingI understand your concern about this being needed for <10% of users. I'm happy to pivot to either approach: Option A: Documentation only (your preference)
Option B: Default scaffolding (current PR)
Which approach would you prefer? I'm happy to adjust the PR accordingly. About the Helm Side EffectI've noted the Helm chart issue in the PR description and will handle it separately if needed. The core Kustomize fix is independent of Helm concerns. Thanks again for your guidance! 🙏 |
1 similar comment
|
Hi @camilamacedo86 👋 Thank you for the feedback! I wanted to clarify the situation with controller-tools after testing extensively. Controller-gen Already Works Correctly ✅I've verified that controller-gen from controller-tools is NOT the issue - it already correctly generates namespace fields in RBAC manifests. Here's the proof: Test SetupI have a controller with namespace-specific RBAC markers: // +kubebuilder:rbac:groups=apps,namespace=infrastructure,resources=deployments,verbs=get;list;watch;update;patch
// +kubebuilder:rbac:groups="",namespace=production,resources=secrets,verbs=get
// +kubebuilder:rbac:groups=coordination.k8s.io,namespace=production,resources=leases,verbs=get;list;watch;create;update;patch;delete
// +kubebuilder:rbac:groups="",namespace=production,resources=events,verbs=get;list;watch;create;update;patch;deleteTest ResultsStep 1: Fresh generation with controller-gen v0.19.0 rm config/rbac/role.yaml
make manifests # Uses controller-gen
cat config/rbac/role.yamlOutput - controller-gen DOES generate namespace fields: ---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: manager-role
namespace: infrastructure # ← Generated by controller-gen
rules:
- apiGroups:
- apps
resources:
- deployments
verbs:
- get
- list
- patch
- update
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: manager-role
namespace: production # ← Generated by controller-gen
rules:
- apiGroups:
- ""
resources:
- secrets
verbs:
- get
# ... more rulesStep 2: Testing with Kustomize's hardcoded # config/default/kustomization.yaml
namespace: test-override
namePrefix: osiris-
resources:
- ../rbackustomize build config/defaultResult: The hardcoded Step 3: Testing with NamespaceTransformer + # config/default/namespace-transformer.yaml
apiVersion: builtin
kind: NamespaceTransformer
metadata:
name: namespace-transformer
namespace: test-override
setRoleBindingSubjects: none
unsetOnly: true # Only transform if namespace not already set
fieldSpecs:
- path: metadata/namespace
create: true# config/default/kustomization.yaml
namePrefix: osiris-
resources:
- ../rbac
transformers:
- namespace-transformer.yamlkustomize build config/defaultResult: SUCCESS ✅ - Explicit namespaces are preserved: ---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: osiris-manager-role
namespace: infrastructure # ← Preserved
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: osiris-manager-role
namespace: production # ← PreservedConclusion
Regarding Your SuggestionYou mentioned:
After this testing, I believe controller-tools doesn't need changes. The namespace field is already in the generated YAML.
You're right that only one file is needed per project. However, this PR modifies how Kubebuilder scaffolds new projects - it generates that file automatically during Documentation vs Default ScaffoldingI understand your concern about this being needed for <10% of users. I'm happy to pivot to either approach: Option A: Documentation only (your preference)
Option B: Default scaffolding (current PR)
Which approach would you prefer? I'm happy to adjust the PR accordingly. About the Helm Side EffectI've noted the Helm chart issue in the PR description and will handle it separately if needed. The core Kustomize fix is independent of Helm concerns. Thanks again for your guidance! 🙏 |
|
This is really helpful—thank you so much for the thorough explanations and for sharing your tests and research. I’d like a bit of time to review everything properly, and I’ll reply back here once I’ve gone through it. |
|
Sure, it's totally fine! |
e22404a to
05681d7
Compare
|
Hi @camilamacedo86, Could you checkout my changes. |
05681d7 to
57461e6
Compare
| app.kubernetes.io/managed-by: kustomize | ||
| name: serving-cert # this name should match the one appeared in kustomizeconfig.yaml | ||
| namespace: system | ||
| namespace: project-system |
There was a problem hiding this comment.
That would mean we’d need to rename the namespace in all cases in the default scaffold, and I don’t think we should do that. The namespace should be centralised in the default config.
I think we need to look at how to make it work with as few changes as possible.
Also, we should consider whether we really want this in the default scaffold, or if it’s better to provide it as sample test data and/or documentation under References.
There was a problem hiding this comment.
Hi @camilamacedo86 👋
Thanks for the review — you're right. I've reworked the PR completely based on your feedback.
What changed:
The previous approach renamed namespace: system across ~15 template files. This new version touches only 3 source files with zero changes to any existing template files:
- New file: namespace_transformer.go — scaffolds
config/default/namespace_transformer.yaml(not active by default) - Modified: kustomization.go — added a small
[NAMESPACE-TRANSFORMER]commented-out section at the bottom (same pattern as[WEBHOOK],[CERTMANAGER],[PROMETHEUS]) and a one-line breadcrumb on thenamespace:field - Modified: init.go — registered the new template (1 line)
Default behavior is 100% unchanged. The namespace: field stays active and centralized.
Why scaffold instead of docs-only:
The namespace_transformer.yaml contains all the detailed instructions — what the problem is, how roleName helps (from controller-tools#1334), and step-by-step opt-in. The kustomization.yaml itself only adds 8 lines total (2 at top + 6 at bottom).
The main motivation is discoverability: when someone hits the namespace transformation produces ID conflict error, they open kustomization.yaml and the [NAMESPACE-TRANSFORMER] breadcrumb is right there pointing them to the solution file. No docs searching needed.
The full PR diff is 15 files, 375 additions, 0 deletions — the additions are the new namespace_transformer.yaml files in testdata/docs + 8 comment lines in each kustomization.yaml.
Happy to adjust further!
57461e6 to
c0d3eee
Compare
c0d3eee to
ea1ba90
Compare
ea1ba90 to
6d629c6
Compare
6d629c6 to
f0ead8d
Compare
f0ead8d to
c06d3c5
Compare
… RBAC Scaffold a commented-out NamespaceTransformer configuration as an opt-in alternative to the "namespace:" field in config/default/kustomization.yaml. When using namespace-scoped RBAC markers (e.g. //+kubebuilder:rbac:namespace=infrastructure), the global "namespace:" field overrides all namespaces, causing ID conflict errors. The NamespaceTransformer with "unsetOnly: true" preserves explicit namespaces while assigning the default namespace to resources without one. Changes: - Add namespace_transformer.go template scaffolding namespace_transformer.yaml - Add commented-out [NAMESPACE-TRANSFORMER] section in kustomization.yaml with inline documentation explaining the ID conflict issue and opt-in steps - Add NOTE comment on the "namespace:" field warning about the override behavior - Register NamespaceTransformer in init.go Default behavior is completely unchanged. Users who need multi-namespace RBAC can follow the inline instructions to switch from the "namespace:" field to the NamespaceTransformer.
c06d3c5 to
6231387
Compare
Description
This PR fixes namespace conflicts when using namespace-scoped RBAC markers in multi-namespace scenarios.
Problem
When using RBAC markers with explicit namespaces:
The
controller-gencorrectly generates separate Role resources for each namespace. However, duringmake deploy, Kustomize's globalnamespace:field inconfig/default/kustomization.yamloverrides ALL namespaces, causing both Roles to end up in the same namespace with identical names → ID conflict error.Solution
Replaces the hardcoded
namespace:field with aNamespaceTransformerusingunsetOnly: true. This approach:namespace:field)Changes
Created:
pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/kdefault/namespace_transformer.gonamespace-transformer.yamlwithunsetOnly: trueModified:
pkg/plugins/common/kustomize/v2/scaffolds/internal/templates/config/kdefault/kustomization.gonamespace:fieldtransformers:section referencingnamespace-transformer.yamlModified:
pkg/plugins/common/kustomize/v2/scaffolds/init.goNamespaceTransformerin templates sliceThe Helm chart generation was affected by this change. Some generated Helm templates changed from:
to:
Root cause: The
metrics_service.yamltemplate has a hardcodednamespace: systemfield. WithunsetOnly: true, the NamespaceTransformer preserves this existing namespace. The Helm v2-alpha plugin'ssubstituteNamespace()function inhelm_templater.goexpects the format<projectname>-system(e.g.,project-v4-system) but encounters justsystem, so it doesn't convert it to{{ .Release.Namespace }}.Question for maintainers: Should the Helm v2-alpha plugin be updated to also replace the
namespace: systempattern? I can make that change if needed, or if there's a preferred alternative solution. The v1-alpha plugin handles this viastrings.ReplaceAll(contentStr, "namespace: system", "namespace: {{ .Release.Namespace }}")inedit.go:442.Testing
make generate- All testdata and docs regenerated successfullymake lint-fix- No linting issuesmake test-unit- All unit tests passnamespace-transformer.yamlin test projectsBackward Compatibility
Existing projects are unaffected. New projects created with
kubebuilder initwill use the NamespaceTransformer approach. Users who prefer the old behavior can:namespace: {{ .ProjectName }}-systeminconfig/default/kustomization.yamltransformers:sectionnamespace-transformer.yamlFixes #5148