(cdk import): (imports of a certain size cause ENOENT error)

[tbreysse]

Describe the bug

Currently, it seems that there is a bug with the 'cdk import' functionality which causes an ENOENT error when attempting to import multiple resources in certain templates.

Error:

 [100%] fail:ENOENT: no such file or directory, open '\cdk-sample\ cdk.out\ test-stack.template.json-4dd9ae34ff7bf1fc59854e70adaf75041674eba29d3d02e7a34f8521ce.yaml' 

After some extensive testing, I have not been able to find a definite root cause, but it seems that the issue appears to be related to template size and number of resources imported.

Expected Behavior

The expected behavior is that the import process will successfully import the resources without error

Current Behavior

After importing a certain number of resources the import command fails with an ENOENT error.

Reproduction Steps

1. Create a CDK stack with the following code:

from constructs import Construct
from aws_cdk import (
    Stack,
    aws_iam,
    aws_apigateway,
    aws_lambda
)
import aws_cdk as cdk

class SampleAppStack(Stack):
    
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest1', 
            _replace_name='sampleTest1',
            #_alias='test'
        )

        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest2', 
            _replace_name='sampleTest2', 
            #_alias='test'
        )

        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest3', 
            _replace_name='sampleTest3', 
            #_alias='test'
        )
        
        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest4', 
            _replace_name='sampleTest4',
            #_alias='test'
        )       
        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest5', 
            _replace_name='sampleTest5', 
           #_alias='test'
        )
        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest6', 
            _replace_name='sampleTest6',
            #_alias='ST'
        )    
        
        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest7', 
            _replace_name='sampleTest7',
            #_alias='ST'
        )
        
        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest8', 
            _replace_name='sampleTest8',
            #_alias='test'
        )
         
        lambda_function = self.create_lambda_function(
            'manufacture-sampleTest9', 
            _replace_name = 'sampleTest9',
            #_alias='test'
        )
               
    def set_lambda_variables(self, _lmd_func):
        _lmd_func.add_environment(
            'testParam1', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 
        _lmd_func.add_environment(
            'testParam2', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 
        _lmd_func.add_environment(
            'testParam3', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 
        _lmd_func.add_environment(
            'testParam4', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 


    def create_lambda_function(self, _lambda_name: str, _replace_name: str, _alias=None):
        result_lambda = aws_lambda.Function(
            self,
            id='id-{}'.format(_lambda_name),
            function_name='lmd-an2-st-t20-common-{}'.format(_lambda_name),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        self.set_lambda_variables(result_lambda)
        function_version = result_lambda.current_version
        if _alias:
            _function_alias = aws_lambda.Alias(
                self,
                id='id-alias-{}'.format(_lambda_name),
                alias_name='{}'.format(_alias),
                version=function_version)
        cfn_lambda = result_lambda.node.default_child
        cfn_lambda.override_logical_id(_replace_name)
    

2. Manually create an Alias for each of the created functions in the Lambda console.
3. Uncomment the alias definition for reach resource in the code
4. Run cdk import and import each of the Aliases.

For a second reproduction you can utilize the following template which shows the same behavior, but only after importing 10 Aliases as opposed to 6 which is the number that can import before failure for the above template.

from constructs import Construct
from aws_cdk import (
    Stack,
    aws_iam,
    aws_apigateway,
    aws_lambda
)
import aws_cdk as cdk

class Cdkpython3Stack(Stack):
    
    def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)
        result_lambda1 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest1'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest1'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda2 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest2'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest2'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda3 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest3'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest3'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda4 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest4'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest4'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda5 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest5'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest5'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda6 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest6'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest6'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda7 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest7'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest7'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda8 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest8'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest8'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        result_lambda9 = aws_lambda.Function(
            self,
            id='id-{}'.format('manufacture-sampleTest9'),
            function_name='lmd-an2-st-t20-common-{}'.format('manufacture-sampleTest9'),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )

        self.set_lambda_variables(result_lambda1)
        self.set_lambda_variables(result_lambda2)
        self.set_lambda_variables(result_lambda3)
        self.set_lambda_variables(result_lambda4)
        self.set_lambda_variables(result_lambda5)
        self.set_lambda_variables(result_lambda6)
        self.set_lambda_variables(result_lambda7)
        self.set_lambda_variables(result_lambda8)
        self.set_lambda_variables(result_lambda9)


        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest1'),
            alias_name='{}'.format('ST'),
            version=result_lambda1.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest2'),
            alias_name='{}'.format('ST'),
            version=result_lambda2.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest3'),
            alias_name='{}'.format('ST'),
            version=result_lambda3.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest4'),
            alias_name='{}'.format('ST'),
            version=result_lambda4.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest5'),
            alias_name='{}'.format('ST'),
            version=result_lambda5.latest_version)
        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest6'),
            alias_name='{}'.format('ST'),
            version=result_lambda6.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest7'),
            alias_name='{}'.format('ST'),
            version=result_lambda7.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest8'),
            alias_name='{}'.format('ST'),
            version=result_lambda8.latest_version)

        _function_alias = aws_lambda.Alias(
            self,
            id='id-alias-{}'.format('manufacture-sampleTest9'),
            alias_name='{}'.format('ST'),
            version=result_lambda9.latest_version)
               
    def set_lambda_variables(self, _lmd_func):
        _lmd_func.add_environment(
            'testParam1', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 
        _lmd_func.add_environment(
            'testParam2', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 
        _lmd_func.add_environment(
            'testParam3', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 
        _lmd_func.add_environment(
            'testParam4', '''"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx \
                    xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx "''' 
                ) 


    def create_lambda_function(self, _lambda_name: str, _replace_name: str, _alias=None):
        result_lambda = aws_lambda.Function(
            self,
            id='id-{}'.format(_lambda_name),
            function_name='lmd-an2-st-t20-common-{}'.format(_lambda_name),
            runtime=aws_lambda.Runtime.PYTHON_3_9,
            code=aws_lambda.Code.from_asset('lambda'),
            handler='hello.handler',
        )
        self.set_lambda_variables(result_lambda)
        function_version = result_lambda.current_version
        cfn_lambda = result_lambda.node.default_child
        cfn_lambda.override_logical_id(_replace_name)

Possible Solution

N/A

Additional Information/Context

As mentioned in the reproduction steps, the behavior of this bug is peculiar.

With the first provided reproduction template, you can import 6 Aliases before the failure occurs. The error will occur for any additional imports after 6.

With the second provided template, you can successfully import 9 Aliases, but once you try to import a 10th, the same error occurs again. Also the error occurs with both Python and TypeScript stacks.

It is understandable that issue such as this may occur since the cdk import feature is still currently in preview, however this behavior seems worth reporting.

CDK CLI Version

2.41.0

Framework Version

No response

Node.js Version

14.17.0

OS

Windows 10

Language

Typescript, Python

Language Version

No response

Other information

No response

[mascur]

Hi @tbreysse,

I was able to successfully reproduce this issue. Thank you for reporting this! We will try to look at this soon.

If anyone has any ideas on where the root cause of this issue may be occurring please feel free to comment below!

[rix0rrr]

@mascur does this reproduce anywhere besides Windows?

The ENOENT thoroughly confuses me. That means "file not found", but I don't see how template size would impact that.

The best I can think of is that there might be a race condition somewhere that only shows itself on Windows...

[rix0rrr]

I also wonder what kind of file test-stack.template.json-4dd9ae34ff7bf1fc59854e70adaf75041674eba29d3d02e7a34f8521ce.yaml is supposed to be...

[rix0rrr]

@mascur can you throw a repro up on GitHub?