(aws_rds): DatabaseCluster s3ExportBuckets fails with Aurora MySQL 3

[vkemthur-luna]

Describe the bug

Trying to Setup MySQL8 Aurora via CDK

this.importBucket = new s3.Bucket(this, this.stackConfig.PaymentsS3ImportBucket);
this.exportBucket = new s3.Bucket(this, this.stackConfig.PaymentsS3ExportBucket);

const paymentsAuroraCluster = new rds.DatabaseCluster(this, this.stackConfig.PaymentsDatabaseId, {
    engine: rds.DatabaseClusterEngine.auroraMysql({version: rds.AuroraMysqlEngineVersion.VER_3_01_0}),
    credentials: rds.Credentials.fromGeneratedSecret('someadmin'),
    iamAuthentication: true,
    instanceProps: {
        instanceType: ec2.InstanceType.of(ec2.InstanceClass.R5, ec2.InstanceSize.LARGE),
        vpcSubnets: {
            subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
        },
        vpc: somevpc
    },
    defaultDatabaseName: this.stackConfig.DefaultDatabase,
    s3ImportBuckets: [this.importBucket],
    s3ExportBuckets: [this.exportBucket]
});

CDK synth template has this

"Type": "AWS::RDS::DBClusterParameterGroup",
      "Properties": {
        "Description": "Cluster parameter group for aurora-mysql8.0",
        "Family": "aurora-mysql8.0",
        "Parameters": {
   "aws_default_s3_role": {
     "Fn::GetAtt": [
       "foomysql8S3ImportRoleA2225BF9",
       "Arn"
     ]
   },
   "aurora_select_into_s3_role": {
     "Fn::GetAtt": [
       "foomysql8S3ExportRoleE8D8EE1E",
       "Arn"
     ]
   }
 }

aurora_select_into_s3_role is not a valid parameter for MySQL8 and deploy fails

10:44:22 AM | UPDATE_FAILED        | AWS::RDS::DBClusterParameterGroup           | fooauroramysq...meterGroup73951D0D
Invalid / Unsupported DB Parameter: aurora_select_into_s3_role

See a similar issue reported here #19126, but probably this only resolved import and needs a similar fix for export

Or is there a workaround?

Expected Behavior

aws_default_s3_role for MySQL8 supports export as well and should be the parameter when s3BucketExport is requested.

Current Behavior

10:44:22 AM | UPDATE_FAILED        | AWS::RDS::DBClusterParameterGroup           | fooauroramysq...meterGroup73951D0D
Invalid / Unsupported DB Parameter: aurora_select_into_s3_role

Reproduction Steps

Included in description

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.19.0 (build e0d3e62)

Framework Version

No response

Node.js Version

v17.7.2

OS

Mac

Language

Typescript

Language Version

4.6.2

Other information

No response

[skinny85]

Hey @vkemthur-luna,

thanks for opening the issue. I have one follow-up question though.

Right now, for versions of MySQL below 8.0, the parameter names for import/export are different (aurora_load_from_s3_role and aurora_select_into_s3_role, respectively).

However, it seems like you're implying that, for MySQL 8.0, the parameter needs to be the same for both import and export (aws_default_s3_role). Because a given Parameter Group can only have a single value for a given Parameter, it seems to me like the import and export Roles have to be identical for MySQL 8.0.

Can you confirm that is the case? Because right now, for all Clusters, the Roles we generate for import and export are different (code here, you can see the combineRoles parameter is passed as false).

Thanks,
Adam

[vkemthur-luna]

https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Reference.html#AuroraMySQL.Reference.Parameters.Cluster

Only aws_default_s3_role (Cluster parameter) is available for MySQL 8

Used when invoking the **LOAD DATA FROM S3, LOAD XML FROM S3,** or **SELECT INTO OUTFILE S3** statement from your DB cluster

So probably need to combine role for engine 8.0 and need to set parameter once

As of now this path is adding a parameter not supported on 8.0

As a workaround, looks like I can create a custom role with s3 read/write policy and then set this in s3ImportRole prop.

[skinny85]

Thanks for the explanation @vkemthur-luna! Prepared a fix here: #19775.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.