(aws-cdk-lib/aws-ec2): selectSubnets not handling CloudWAN Subnet correctly

[VardyNg]

Describe the bug

CloudWAN create a global network for VPC, Data Centres, and branch offices.

When configuring VPC to join the network, we create VPC Attachment, then configure Subnet Route Table to direct traffic to the CloudWAN network.

This set up is similar to using TGW, where we create an attachment and configure Subnet Route Table.

However, when using the selectSubnets function to query subnets from VPC, the CloudWAN Subnet is being considered as PRIVATE_ISOLATED , instead of PRIVATE_WITH_EGRESS which categorizes the TGW Subnet.

Refer to the related source code, it appears that the selectSubnets function does not take CloudWAN into consideration, but only IGW, NAT Gateway, and TGW.

const subnets: Subnet[] = listedSubnets.map((subnet) => {
      let type = getTag('aws-cdk:subnet-type', subnet.Tags);
      if (type === undefined && subnet.MapPublicIpOnLaunch) {
        type = SubnetType.Public;
      }
      if (type === undefined && routeTables.hasRouteToIgw(subnet.SubnetId)) {
        type = SubnetType.Public;
      }
      if (type === undefined && routeTables.hasRouteToNatGateway(subnet.SubnetId)) {
        type = SubnetType.Private;
      }
      if (type === undefined && routeTables.hasRouteToTransitGateway(subnet.SubnetId)) {
        type = SubnetType.Private;
      }
      if (type === undefined) {
        type = SubnetType.Isolated;
      }
...

https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk/lib/context-providers/vpcs.ts#L79

Please advice whether this behaviour is expected, or not CloudWAN is missed from the current subnet selection logic.

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

Latest, as appeared the main branch

Expected Behavior

"CloudWAN Subnet" to be categorized as PRIVATE_WITH_EGRESS

Current Behavior

"CloudWAN Subnet" being categorized as PRIVATE_ISOLATED

Reproduction Steps

Deploy a CloudWAN and Configure a "CloudWAN Subnet"

Here I use Terraform

• CloudWAN

  resource "aws_networkmanager_global_network" "global_network" {
    tags = {
      Name = local.name
    }
  }
  
  resource "aws_networkmanager_core_network" "core_network" {
    global_network_id = aws_networkmanager_global_network.global_network.id
  }
  
  data "aws_networkmanager_core_network_policy_document" "policy" {
    core_network_configuration {
      asn_ranges = ["65022-65534"]
  
      edge_locations {
        location = local.region
        asn      = "65500"
      }
    }
  
    segments {
      name = "segment"
    }
  }
  resource "aws_networkmanager_core_network_policy_attachment" "policy" {
    core_network_id = aws_networkmanager_core_network.core_network.id
    policy_document = data.aws_networkmanager_core_network_policy_document.policy.json
  }
  
  resource "aws_networkmanager_vpc_attachment" "attachment" {
    core_network_id = aws_networkmanager_core_network.core_network.id
    vpc_arn         = module.vpc.vpc_arn
    subnet_arns     = [aws_subnet.cloudwan.arn]
  
    depends_on = [
      aws_networkmanager_core_network_policy_attachment.policy
    ]
  }

• Subnet

  resource "aws_subnet" "cloudwan" {
    vpc_id     = module.vpc.vpc_id
    cidr_block = "10.0.103.0/24"
    map_public_ip_on_launch = false
    tags = {
      Name = "${local.name}-cloudwan"
    }
  }
  
  resource "aws_route_table" "cloudwan-rt" {
    vpc_id = module.vpc.vpc_id
  }
  
  resource "aws_route_table_association" "cloudwan-rt-assoication" {
    route_table_id = aws_route_table.cloudwan-rt.id
    subnet_id = aws_subnet.cloudwan.id
  }
  
  resource "aws_route" "cloudwan-route" {
    route_table_id = aws_route_table.cloudwan-rt.id 
    destination_cidr_block = "0.0.0.0/0"
    core_network_arn = aws_networkmanager_core_network.core_network.arn
  }

Note that the above created a subnet with ID subnet-0acf41adc9e62d5e7

Use AWS CDK to query VPC subnets

const vpc = ec2.Vpc.fromLookup(this, "Vpc", {
      vpcId: '...' 
})

var private_isolated_subnet;
    try {
      console.log("private_isolated_subnet")

      private_isolated_subnet = vpc.selectSubnets({
        subnetType: ec2.SubnetType.PRIVATE_ISOLATED
      })
  
      console.log(private_isolated_subnet.subnetIds);
    } catch (e) {
      console.log(e)
    }

new cdk.CfnOutput(this, 'PRIVATE_ISOLATED', {
      value: JSON.stringify(private_isolated_subnet!.subnetIds)
    })

Test:
CDK version:

$ npx cdk version
2.172.0 (build 0f666c5)

CDK clear context value

npx cdk context --clear   

Run CDK Synthesize to query subnets

npx cdk synthesize

private_isolated_subnet
[ 'subnet-0acf41adc9e62d5e7' ]

As shown above, the subnet-0acf41adc9e62d5e7 is being considered as PRIVATE_ISOLATED

Possible Solution

Implement an additional function in RouteTable class hasRouteToCloudWAN, use this function in readVpcProps to check subnet CloudWAN config, and set subnet type as SubnetType.Private.

Additional Information/Context

No response

CDK CLI Version

2.172.0 (build 0f666c5)

Framework Version

No response

Node.js Version

v20.11.1

OS

macOS

Language

TypeScript

Language Version

5.3.3

Other information

No response

[ashishdhingra]

@VardyNg Thanks for detailed analysis and PR. Could you also please point to documentation which specifies that for a CloudWAN Subnet, PRIVATE_WITH_EGRESS should be used.

[VardyNg]

@ashishdhingra Please refer to internal ticket ID: 2ea4b9ce-ef13-4dd8-b422-bcc505869f33

[ashishdhingra]

Internal Ticket: V1604202007

[pahud]

Not sure if subnets with CloudWAN as default route should be classified as PRIVATE_WITH_EGRESS. Bringing this up to the team for inputs. Meanwhile, we welcome any pull requests to move this forward.

[ashishdhingra]

Not sure if subnets with CloudWAN as default route should be classified as PRIVATE_WITH_EGRESS. Bringing this up to the team for inputs. Meanwhile, we welcome any pull requests to move this forward.

@pahud Draft PR #32494 has been contributed by requestor, but we need to validate if default route in CloudWAN should be classified as PRIVATE_WITH_EGRESS.

[VardyNg]

Not sure if subnets with CloudWAN as default route should be classified as PRIVATE_WITH_EGRESS. Bringing this up to the team for inputs. Meanwhile, we welcome any pull requests to move this forward.

@pahud Draft PR #32494 has been contributed by requestor, but we need to validate if default route in CloudWAN should be classified as PRIVATE_WITH_EGRESS.

Yes, I drafted that PR in case it is required, agree that we should validate if it is the correct classification.