WIP: Add support for dual stack load balancers by nrb · Pull Request #1313 · kubernetes/cloud-provider-aws

nrb · 2025-12-02T21:18:23Z

What type of PR is this?
/kind feature

What this PR does / why we need it:

Adds basic support for configuring the IP address family on Services.

This functionality overlaps with the ALBC somewhat, but the goal here is to provide much more basic functionality, not the full scope of ALBC.

Which issue(s) this PR fixes:

Fixes #1219

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

Support for the Kubernetes Service dual-stack networking API.

k8s-ci-robot · 2025-12-02T21:18:26Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

k8s-ci-robot · 2025-12-02T21:18:31Z

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

nrb · 2025-12-02T21:32:24Z

/test pull-cloud-provider-aws-test

damdo · 2025-12-09T19:49:55Z

Hey @kmala do we know what's happening with the Netlify checks? It looks like they are broken across many open PRs. TY

damdo · 2025-12-10T07:50:27Z

/retest

nrb · 2025-12-11T16:00:12Z

/retest

nrb · 2025-12-12T14:00:46Z

/retest

nrb · 2025-12-12T16:06:24Z

/test pull-cloud-provider-aws-test

One more run to see if this is a rate limiting issue or some sort of genuine issue with my code.

tthvo

I tested with a simple service below:

apiVersion: v1
kind: Service
metadata:
  name: wordpress-mysql
  labels:
    app: wordpress
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-ip-address-type: dualstack
spec:
  ipFamilyPolicy: PreferDualStack
  ports:
    - port: 3306
  selector:
    app: wordpress
  type: LoadBalancer

The CCM does create the dualstack NLB as expected 🚀

$ kubectl -n myns get svc/wordpress-mysql -o yaml | yq .status.loadBalancer.ingress
- hostname: <id>.elb.us-east-1.amazonaws.com

$ nslookup <id>.elb.us-east-1.amazonaws.com
# nslookup shows both public IPv4 EIP and IPv6 GUA addresses

$ aws elbv2 describe-load-balancers --names=<nlb-id> --query='LoadBalancers[0].[Type,IpAddressType,Scheme]'
[
    "network",
    "dualstack",
    "internet-facing"
]

pkg/providers/v1/aws_validations_test.go

pkg/providers/v1/aws.go

nrb · 2026-01-16T15:08:12Z

/test pull-cloud-provider-aws-test

nrb · 2026-01-22T22:41:32Z

/test pull-cloud-provider-aws-test

nrb · 2026-01-22T22:42:51Z

/test pull-cloud-provider-aws-check

nrb · 2026-01-23T17:38:16Z

/test pull-cloud-provider-aws-test
/test pull-cloud-provider-aws-check

nrb · 2026-01-26T15:29:02Z

@kmala I'm going to do some manual testing with this PR starting today before I mark it as ready for review, but would you be able to give it a look by any chance?

tthvo

I just have a few questions 🙏 But overall, I think we are having great progress!

pkg/providers/v1/aws.go

pkg/providers/v1/aws_loadbalancer.go

tthvo · 2026-02-25T06:35:12Z

pkg/providers/v1/aws.go

 		err = c.updateInstanceSecurityGroupsForNLB(ctx, loadBalancerName, instances, subnetCidrs, sourceRangeCidrs, v2Mappings)
 		if err != nil {
 			klog.Warningf("Error opening ingress rules for the load balancer to the instances: %q", err)
 			return nil, err
 		}


We should consider/allow IPv6 CIDRs when updating the instance's security groups (to allow traffic from NLB), right? IIUC, there are 2 types of CIDRs here:

sourceRangeCidrs: Already defined on line 2396-2405

subnetCidrs: IPv6 CIDRs can be extracted from subnet info returned by AWS.

I guess that means we need to update the following functions to set the Ipv6Range field correctly?

cloud-provider-aws/pkg/providers/v1/aws_loadbalancer.go

Line 957 in b6c9d76

func (c *Cloud) updateInstanceSecurityGroupsForNLB(ctx context.Context, lbName string, instances map[InstanceID]*ec2types.Instance, subnetCIDRs []string, clientCIDRs []string, portMappings []nlbPortMapping) error {

cloud-provider-aws/pkg/providers/v1/aws_loadbalancer.go

Line 1049 in b6c9d76

func (c *Cloud) updateInstanceSecurityGroupForNLBTraffic(ctx context.Context, sgID string, sgPerms IPPermissionSet, ruleDesc string, protocol string, ports sets.Set[int32], cidrs []string) error {

tthvo · 2026-02-25T06:42:40Z

pkg/providers/v1/aws_loadbalancer.go

+	if service.Spec.IPFamilyPolicy != nil && *service.Spec.IPFamilyPolicy == v1.IPFamilyPolicySingleStack {
+		if serviceRequestsIPv6(service) {
+			return nil, fmt.Errorf("single stack IPv6 is not supported for network load balancers")
+		}
+	}


I wonder if we can:

Place this validation in pkg/providers/v1/aws_validations.go

Additionally, we should add a validation that only allows dualstack settings (i.e. ipFamilies: [IPv4, IPv6], ipFamilyPolicy: RequireDualStack) when LB type is NLB.

WDYT?

Yep, makes sense.

pkg/providers/v1/aws_loadbalancer.go

tthvo · 2026-02-25T06:58:31Z

docs/service_controller.md

+
+Some limitations to be aware of when using dual stack load balancers:
+
+- The `spec.ipFamilies` field can have a second family added or removed, but the first entry is immutable after Service creation.


Maybe, we should mention what would happen if the secondary family is removed? Like what will be the state of:

The Load Balancer

The Security Groups (those for LB and those for instances)

Target Group

(Optional) Listeners

tthvo · 2026-02-25T06:59:14Z

docs/service_controller.md

+        - IPv6
+        - IPv4


nit: The list indentation is a lot more than usual. Let's format it a bit 😁

damdo · 2026-02-26T12:15:29Z

/retest

damdo · 2026-02-26T13:09:22Z

/test pull-cloud-provider-aws-e2e-kubetest2-quick

damdo · 2026-02-27T10:09:17Z

/test pull-cloud-provider-aws-e2e-kubetest2-quick

nrb · 2026-03-03T23:46:14Z

/test pull-cloud-provider-aws-test

nrb · 2026-03-03T23:47:17Z

Current e2e failure is legit - the provisioned cluster is not set up for IPv6.

   STEP: creating LoadBalancer service in Kubernetes @ 03/03/26 17:45:37.338
  I0303 17:45:37.386843 32011 loadbalancer.go:508] Unexpected error: 
      <*errors.errorString | 0xc000321140>: 
      failed to create LoadBalancer Service "lbconfig-test-clb-ds": Service "lbconfig-test-clb-ds" is invalid: spec.ipFamilies[1]: Invalid value: "IPv6": not configured on this cluster
      {
          s: "failed to create LoadBalancer Service \"lbconfig-test-clb-ds\": Service \"lbconfig-test-clb-ds\" is invalid: spec.ipFamilies[1]: Invalid value: \"IPv6\": not configured on this cluster",
      }
  [FAILED] in [It] - /home/prow/go/src/k8s.io/cloud-provider-aws/tests/e2e/loadbalancer.go:508 @ 03/03/26 17:45:37.387
  STEP: Destroying namespace "cloud-provider-aws-593" for this suite. @ 03/03/26 17:45:37.387
  << Timeline

Will work on getting support for that plumbed in to kubetest-ec2.

nrb · 2026-03-11T21:38:33Z

I've opened a PR against kubetest2-ec2 in order to provide an IPv6-capable subnet to CI.

kubernetes-sigs/provider-aws-test-infra#530

linux-foundation-easycla · 2026-03-12T19:29:37Z

The committers listed above are authorized under a signed CLA.

✅ login: nrb / name: Nolan Brubaker (573c4ca, 5d22fe2, 69b6bb9, aa4d51b, eb56af1, fa180ec)

- Create load balancers according to the Kubernetes Service API Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com> Assisted-by: Claude Sonnet 4.5 <noreply@anthropic.com>

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

Changing the IP address type would invalidate the target group name, because you cannot have an IPv4 and IPv6 target for the same port/protocol set. Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

k8s-ci-robot · 2026-03-24T21:15:14Z

@nrb: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cloud-provider-aws-check	`5d22fe2`	link	true	`/test pull-cloud-provider-aws-check`
pull-cloud-provider-aws-e2e	`5d22fe2`	link	true	`/test pull-cloud-provider-aws-e2e`
pull-cloud-provider-aws-e2e-kubetest2-quick	`5d22fe2`	link	false	`/test pull-cloud-provider-aws-e2e-kubetest2-quick`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. release-note Denotes a PR that will be considered when it comes time to generate release notes. labels Dec 2, 2025

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 2, 2025

k8s-ci-robot requested review from bridgetkromhout and cheftako December 2, 2025 21:18

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Dec 2, 2025

nrb force-pushed the dual-stack-lb branch from ba4b93b to 83110d8 Compare December 11, 2025 19:04

tthvo reviewed Jan 7, 2026

View reviewed changes

pkg/providers/v1/aws_validations_test.go Outdated Show resolved Hide resolved

pkg/providers/v1/aws.go Outdated Show resolved Hide resolved

nrb force-pushed the dual-stack-lb branch from 83110d8 to ffbb4b8 Compare January 16, 2026 13:58

tthvo reviewed Feb 25, 2026

View reviewed changes

k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Mar 2, 2026

nrb force-pushed the dual-stack-lb branch from 1b3815c to 2dd0dac Compare March 4, 2026 14:20

alebedev87 mentioned this pull request Mar 5, 2026

NE-2421: Support dual-stack IngressController on AWS openshift/cluster-ingress-operator#1376

Open

damdo mentioned this pull request Mar 10, 2026

WIP: Add dual-stack (IPv6) support to kubetest2-ec2 deployer kubernetes-sigs/provider-aws-test-infra#530

Open

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. and removed cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 12, 2026

nrb force-pushed the dual-stack-lb branch from a880153 to 2dd0dac Compare March 12, 2026 19:33

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 13, 2026

nrb mentioned this pull request Mar 18, 2026

OCPCLOUD-3215: Add support for dual networking stack services openshift/cloud-provider-aws#135

Open

nrb force-pushed the dual-stack-lb branch from 72b94ae to 25b5ffb Compare March 18, 2026 18:36

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Mar 18, 2026

nrb added 2 commits March 18, 2026 16:56

Add support for dual networking stack services

69b6bb9

- Create load balancers according to the Kubernetes Service API Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com> Assisted-by: Claude Sonnet 4.5 <noreply@anthropic.com>

Add dual-stack e2e test cases

eb56af1

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

nrb force-pushed the dual-stack-lb branch from 25b5ffb to eb56af1 Compare March 18, 2026 20:56

nrb added 4 commits March 24, 2026 15:01

Support IP address type changes on update

fa180ec

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

Hash target group names with IP address family

aa4d51b

Changing the IP address type would invalidate the target group name, because you cannot have an IPv4 and IPv6 target for the same port/protocol set. Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

Take IPv6 CIDRs into account for all open traffic

573c4ca

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>

fixup! Support IP address type changes on update

5d22fe2

Signed-off-by: Nolan Brubaker <nolan@nbrubaker.com>


		Some limitations to be aware of when using dual stack load balancers:

		- The `spec.ipFamilies` field can have a second family added or removed, but the first entry is immutable after Service creation.

Conversation

nrb commented Dec 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Dec 2, 2025

Uh oh!

k8s-ci-robot commented Dec 2, 2025

Uh oh!

nrb commented Dec 2, 2025

Uh oh!

damdo commented Dec 9, 2025

Uh oh!

damdo commented Dec 10, 2025

Uh oh!

nrb commented Dec 11, 2025

Uh oh!

nrb commented Dec 12, 2025

Uh oh!

nrb commented Dec 12, 2025

Uh oh!

tthvo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

nrb commented Jan 16, 2026

Uh oh!

nrb commented Jan 22, 2026

Uh oh!

nrb commented Jan 22, 2026

Uh oh!

nrb commented Jan 23, 2026

Uh oh!

nrb commented Jan 26, 2026

Uh oh!

tthvo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tthvo Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

tthvo Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

tthvo Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

nrb Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tthvo Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

tthvo Feb 25, 2026

Choose a reason for hiding this comment

Uh oh!

damdo commented Feb 26, 2026

Uh oh!

damdo commented Feb 26, 2026

Uh oh!

damdo commented Feb 27, 2026

Uh oh!

nrb commented Mar 3, 2026

Uh oh!

nrb commented Mar 3, 2026

Uh oh!

nrb commented Mar 11, 2026

Uh oh!

linux-foundation-easycla bot commented Mar 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Mar 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

nrb commented Dec 2, 2025 •

edited

Loading

linux-foundation-easycla bot commented Mar 12, 2026 •

edited

Loading