cli: Socket timed out without establishing a connection when --asset-parallelism=true

[apoorvmote]

Describe the bug

I have anywhere between 20-50 nodejs lambda functions in single stack and I update their dependencies and deploy with cdk.

But lately I am not able to deploy updates. I get following error when I deploy.

current credentials could not be used to assume 'arn:aws:iam::******:role/cdk-hnb659fds-lookup-role-******-us-east-1', but are for the right account. Proceeding anyway.
(To get rid of this warning, please upgrade to bootstrap version >= 8)
current credentials could not be used to assume 'arn:aws:iam::******:role/cdk-hnb659fds-file-publishing-role-******-us-east-1', but are for the right account. Proceeding anyway.
current credentials could not be used to assume 'arn:aws:iam::******:role/cdk-hnb659fds-file-publishing-role-******-us-east-1', but are for the right account. Proceeding anyway.
[9%] fail: Socket timed out without establishing a connection
[18%] fail: Socket timed out without establishing a connection

I keep trying again and again and sometimes it goes through and most of the time it doesn't work.
Only stack with lower number of lambda functions sometimes gets deployed. But stack with large number of lambda functions fails 100% of the time.

Expected Behavior

I expected it to deploy no matter number of lambda functions in the stack. It used to get deployed without any problem.

Current Behavior

current credentials could not be used to assume 'arn:aws:iam::******:role/cdk-hnb659fds-lookup-role-******-us-east-1', but are for the right account. Proceeding anyway.
(To get rid of this warning, please upgrade to bootstrap version >= 8)

I don't know how to upgrade bootstrap version. I ran cdk bootstrap multiple times and it says no changes.

Reproduction Steps

const testSignUpFn = new NodejsFunction(this, 'testSignUpNodeJS', {
      runtime: Runtime.NODEJS_14_X,
      entry: `${__dirname}/../lambda-fns/sign-up/index.ts`,
      handler: 'signUp',
      architecture: Architecture.ARM_64,
      memorySize: 1024
    })

It was working before but suddenly stopped working.

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.20.0 (build 738ef49)

Framework Version

No response

Node.js Version

v16.14.2

OS

Ubuntu 20.04 on WSL 2

Language

Typescript

Language Version

~3.9.7

Other information

No response

[corymhall]

@apoorvmote do you know what changed between when it used to work and now? Did you recently update
the version of the CDK?

[apoorvmote]

Of course I am regularly changing CDK versions as it comes out. But in my opinion problem is not with CDK. But AWS is denying my specific IP address whenever I do large update for 30-50 functions. All the other stacks has very little update and it always goes through.

Also I have another cdk project that I run in docker development environment and I am able to deploy over 50 functions without any problem from same computer. I can run that project in docker because all functions are nodejs built with esbuild. This (failing) project has functions written in golang and I use docker for building golang functions. I am not able to run docker inside docker so long story long I run this in WSL and deploy normally and it fails. But if I build it on WSL and docker then deploy from docker development environment because golang function is already built then it gets deployed.

[corymhall]

@apoorvmote can you try this with a version of the CDK <2.17.0/1.149.0? We added parallel asset
publishing starting in those versions and i'm curious if that could be the issue.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[apoorvmote]

I did upgrade to 2.22.0 and suddenly problem is disappeared. If the problem appears again then I will open another issue.