(cli) cdk import : resource-mapping options doesn't work

[sudhirkumar04]

Describe the bug

cdk import --resource-mapping=resource-mapping.json doesn't work. This returns

Unrecognized resource identifiers in mapping file: ImportedKMSKey1, ImportedKMSKey2
No resources selected for import.

ImportedKMSKey1, ImportedKMSKey2 are resource names

--resource-mapping option also return same result when it used after --record-resource-mapping

cdk import --record-resource-mapping=resource-mapping.json // this command prompted for keyId twice
cdk import -resource-mapping=resource-mapping.json

This returns same as above

CoreAwsInfraStack
CoreAwsInfraStack/ImportedKMSKey1D83208A7/Resource: skipping
CoreAwsInfraStack/ImportedKMSKey2B3E62479/Resource: skipping
Unrecognized resource identifiers in mapping file: ImportedKMSKey1D83208A7, ImportedKMSKey2B3E62479
No resources selected for import.

Expected Behavior

cdk import --resource-mapping=resource-mapping.json should import the resources mentioned in the resource-mapping.json file

Current Behavior

cdk import --resource-mapping=resource-mapping.json doesn't work. This returns

CoreAwsInfraStack
CoreAwsInfraStack/ImportedKMSKey1/Resource: skipping
CoreAwsInfraStack/ImportedKMSKey2/Resource: skipping
Unrecognized resource identifiers in mapping file: ImportedKMSKey1, ImportedKMSKey2
No resources selected for import.

ImportedKMSKey1, ImportedKMSKey2 are resource names

Reproduction Steps

1. Create a resource-mapping file, in this test, resource-mapping file is created inside lib/config/dev.json. E.g. below

{
  "ImportedKMSKey1": {
      "KeyId": "xxxx-ad04-42ae-9b36-xxxxxxx"
  },
  "ImportedKMSKey2": {
      "KeyId": "xxxxx-dca0-4d07-9fe7-xxxxxx"
  }
}

2. Create file inside lib directory, this is for importing two kms keys.

import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { Key } from 'aws-cdk-lib/aws-kms'
import {Tags} from 'aws-cdk-lib'
import * as fs from 'fs';

export class CoreAwsInfraStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
    const config = JSON.parse(fs.readFileSync(`lib/config/dev.json`, 'utf8'));
    for (const [key] of Object.entries(config)) {
      new Key(this, key, {
      });
    }
  }
}

3. Create a file inside bin directory

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { CoreAwsInfraStack } from '../lib/core-aws-infra-stack';

const app = new cdk.App();
new CoreAwsInfraStack(app, 'CoreAwsInfraStack', {
});

4. AWS_DEFAULT_PROFILE is set
5. Run cdk import --resource-mapping=lib/config/dev.json

Possible Solution

cdk import works but it prompts the kms key id which is a pain for rolling out for different accounts and environment. In order to avoid prompting keyId, resource-mapping should allow to map the resource.

Additional Information/Context

No response

CDK CLI Version

2.122

Framework Version

No response

Node.js Version

21.7.1

OS

mac

Language

TypeScript

Language Version

No response

Other information

No response

[khushail]

@sudhirkumar04 , Thanks for reporting this.

I am able to repro the situation and resources are not getting imported however in my case, the error is a bit different which only says Skipping import. I found a somewhat similar past issue , there is a workaround suggested, not really sure if that works. You could check that out. Please feel free to share if that works!

I am currently investigating the root cause and looking into it. Will share my findings soon if there is an alternate on how this could be achieved.

[sudhirkumar04]

thanks for looking into it @khushail
Skipping import means its not importing the resource with the --resource-mapping options. The workaround you suggested has been checked before this issue raised. One of the solution suggested to use RemovalPolicy.DESTROY/RETAIN has also been checked.
In the issue you suggested not using --resource-mapping to map the resource. The main purpose of using --resource-mapping option is to import resources using ci/cd where the resources is taken from a file rather entering details for each resource manually in the terminal

[khushail]

@sudhirkumar04 , I understood your point in providing the resources through the file and I tried to repro the same scenario and faced similar error -

As per my investigation, this code leads to reading the data from the file -

aws-cdk/packages/aws-cdk/lib/cdk-toolkit.ts

Line 551 in ba8edb3

: await resourceImporter.loadResourceIdentifiers(additions, options.resourceMappingFile);

which is invoking this module -

aws-cdk/packages/aws-cdk/lib/import.ts

Line 92 in ba8edb3

public async loadResourceIdentifiers(available: ImportableResource[], filename: string): Promise<ImportMap> {

Looks like, the file is not being read properly and error is produced by contents being read as Unknown -

aws-cdk/packages/aws-cdk/lib/import.ts

Line 111 in ba8edb3

if (unknown.length > 0) {

Since the direct option of passing the logical id exists through cdk import, marking this as P2 which means it would be on team's radar but won't be immediately addressed. This is also open for community contribution.

[TheRealAmazonKendra]

The issue here is that the resource mapping is looking for the entire ID, including the path, for the resource. It must be in the format of <stack-name>/<potential-parent-resource>/<maybe-more-potential-parent-resources>/<resource-id>/Resource.

[TheRealAmazonKendra]

This is definitely a usability issue, but it is not a bug so I'm removing that label here.

[TheRealAmazonKendra]

The docstring gives an example of what this should look like:

/**
 * Mapping of CDK resources (L1 constructs) to physical resources to be imported
 * in their place, example:
 *
 * ```
 * {
 *   "MyStack/MyS3Bucket/Resource": {
 *     "BucketName": "my-manually-created-s3-bucket"
 *   },
 *   "MyStack/MyVpc/Resource": {
 *     "VpcId": "vpc-123456789"
 *   }
 * }
 * 

[Luchiap]

Leaving this for reference and assistance to others.

This is definitely not a usability issue.
The only way this currently works is when you provide the logical ID of that resource that you can get from the manifest.json file.
For example if you want to import an sns topic, this would be what goes inside the file you are referencing to import

{
  "CdkrdsimportTopic215EE3043": {
    "TopicArn": "arn:aws:sns:us-east-1:************:fgdfgdg.fifo"
  }
}

Again, the CdkrdsimportTopic215EE3043 (this a logical id) is retrieved from the manifest.json file.
And the manifest.json file is generated after a cdk synth or cdk deploy or other cdk actions.