[Bug]: create glue crawler with delta_target without connection_name

[ConstantinoSchillebeeckx]

Terraform Core Version

1.1.5

AWS Provider Version

4.24.0

Affected Resource(s)

aws_glue_crawler

Expected Behavior

I believe I should be able to create a Glue Crawler that uses a delta_target without specifying the connection_name; however it is required:

terraform-provider-aws/internal/service/glue/crawler.go

Lines 225 to 228 in 5b85a57

	"connection_name": {
	Type: schema.TypeString,
	Required: true,
	},

An AWS blog post describes this as being optional. Furthermore, I have manually setup a crawler in the console to use a delta target, the UI shows this connection as being optional:

The crawler I've setup, without a connection name, successfully crawls S3; introspecting that crawler with aws glue get-crawler --name "foo" --output json shows:

{
    "Crawler": {
        "Name": "foo",
        "Role": "foo",
        "Targets": {
            "S3Targets": [],
            "JdbcTargets": [],
            "MongoDBTargets": [],
            "DynamoDBTargets": [],
            "CatalogTargets": [],
            "DeltaTargets": [
                {
                    "DeltaTables": [
                        "s3://test-cs-glue-crawler/data/"
                    ],
                    "WriteManifest": true
                }
            ]
        },
        "DatabaseName": "foo",
        "Classifiers": [],
        "RecrawlPolicy": {
            "RecrawlBehavior": "CRAWL_EVERYTHING"
        },
        "SchemaChangePolicy": {
            "UpdateBehavior": "UPDATE_IN_DATABASE",
            "DeleteBehavior": "DEPRECATE_IN_DATABASE"
        },
        "LineageConfiguration": {
            "CrawlerLineageSettings": "DISABLE"
        },
        "State": "READY",
        "CrawlElapsedTime": 0,
        "CreationTime": 1665598443.0,
        "LastUpdated": 1665599113.0,
        "LastCrawl": {
            "Status": "SUCCEEDED",
            "LogGroup": "/aws-glue/crawlers",
            "LogStream": "foo",
            "MessagePrefix": "foo",
            "StartTime": 1665619047.0
        },
        "Version": 2,
        "LakeFormationConfiguration": {
            "UseLakeFormationCredentials": false,
            "AccountId": ""
        }
    }
}

Notice how DeltaTargets does not return any ConnectionName

Actual Behavior

Omitting connection_name produces terraform error:

╷
│ Error: Missing required argument
│ 
│   on crawler.tf line 60, in resource "aws_glue_crawler" "this":
│   60:   delta_target {
│ 
│ The argument "connection_name" is required, but no definition was found.
╵

Relevant Error/Panic Output Snippet

No response

Terraform Configuration Files

NA

Steps to Reproduce

apply resource

Debug Output

No response

Panic Output

No response

Important Factoids

No response

References

No response

Would you like to implement a fix?

yes

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[olenm]

a workaround for this is to create a new connection, type network, attach it to: vcp, subnet, and security group giving it full ingress+egress

[ConstantinoSchillebeeckx]

FWIW you can provide a blank connection name

resource "aws_glue_crawler" "this" {
  database_name = "foo"
  name          = "bar"
  role          = module.crawler_role.iam_role_arn

  delta_target {
    connection_name = ""
    delta_tables    = ["buckets"]
    write_manifest  = true
  }
}

[olenm]

@ConstantinoSchillebeeckx thanks, yes this worked for our setup as well. seems like easiest fix currently would be to update documentation then do code-magic later

[github-actions]

This functionality has been released in v4.46.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.