Implement VPC Endpoint for AWS Provider

[darryl-sw]

Description of your changes

Fixes #715 #467

I have:

• Read and followed Crossplane's contribution process.
• Run make reviewable test to ensure this PR is ready for review.

How has this code been tested

• Verified that the VPC Endpoint can be created upstream in AWS
• Ensured that resource is marked as READY only when VPC Endpoint status is available
• Verified that the VPC Endpoint can be deleted upstream in AWS

[chlunde]

Thanks for your contribution, could you also look at bit at supporting updates?

[darryl-sw]

@chlunde I've completed the requested changes. Could you help me take a look at the changes please? Thanks!

[darryl-sw]

@turkenh, I failed the check-diff job in the previous run. Did a make generate and git diff locally to get this output.

diff --git a/apis/acm/v1alpha1/zz_generated.deepcopy.go b/apis/acm/v1alpha1/zz_generated.deepcopy.go
index 36535911..cf8dfdbe 100644
--- a/apis/acm/v1alpha1/zz_generated.deepcopy.go
+++ b/apis/acm/v1alpha1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
 // +build !ignore_autogenerated
 
 /*
diff --git a/apis/acmpca/v1alpha1/zz_generated.deepcopy.go b/apis/acmpca/v1alpha1/zz_generated.deepcopy.go
index 05d1ef6f..943bc892 100644
--- a/apis/acmpca/v1alpha1/zz_generated.deepcopy.go
+++ b/apis/acmpca/v1alpha1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
 // +build !ignore_autogenerated
 
 /*
diff --git a/apis/apigatewayv2/v1alpha1/zz_generated.deepcopy.go b/apis/apigatewayv2/v1alpha1/zz_generated.deepcopy.go
index 6de8677b..bcc03079 100644
--- a/apis/apigatewayv2/v1alpha1/zz_generated.deepcopy.go
+++ b/apis/apigatewayv2/v1alpha1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
 // +build !ignore_autogenerated
 
 /*
diff --git a/apis/cache/v1alpha1/zz_generated.deepcopy.go b/apis/cache/v1alpha1/zz_generated.deepcopy.go
index ca1083d6..3fb44a5e 100644
--- a/apis/cache/v1alpha1/zz_generated.deepcopy.go
+++ b/apis/cache/v1alpha1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
 // +build !ignore_autogenerated
 
 /*
diff --git a/apis/cache/v1beta1/zz_generated.deepcopy.go b/apis/cache/v1beta1/zz_generated.deepcopy.go
index d671c164..3543222b 100644
--- a/apis/cache/v1beta1/zz_generated.deepcopy.go
+++ b/apis/cache/v1beta1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
 // +build !ignore_autogenerated

 ...
// 

34 files came appended with //go:build !ignore_autogenerated. By any chance, would you happen to know how I can resolve this?

[chlunde]

+++ b/apis/cache/v1beta1/zz_generated.deepcopy.go
@@ -1,3 +1,4 @@
+//go:build !ignore_autogenerated
 // +build !ignore_autogenerated

34 files came appended with //go:build !ignore_autogenerated. By any chance, would you happen to know how I can resolve this?

this happens when you have go 1.17 locally, and the build environment uses go 1.16 or older. A workaround is currently something like sed -i /go:build/ apis/*/*/*.go (and more) before commit/push.

[darryl-sw]

@chlunde can you take a look at the changes please?

[chlunde]

@AaronME is there an issue in detect-noop?

@darryl-sw The tests are not running, when I run them locally I get an issue with a panic in IsPolicyUpToDate. I think we should harden it by using StringValue:

diff --git a/pkg/clients/aws.go b/pkg/clients/aws.go
index 7cd91a7f..83a8b35e 100644
--- a/pkg/clients/aws.go
+++ b/pkg/clients/aws.go
@@ -658,11 +658,11 @@ func IsPolicyUpToDate(local, remote *string) bool {
        var remoteUnmarshalled interface{}
 
        var err error
-       err = json.Unmarshal([]byte(*local), &localUnmarshalled)
+       err = json.Unmarshal([]byte(StringValue(local)), &localUnmarshalled)
        if err != nil {
                return false
        }
-       err = json.Unmarshal([]byte(*remote), &remoteUnmarshalled)
+       err = json.Unmarshal([]byte(StringValue(remote)), &remoteUnmarshalled)
        if err != nil {
                return false
        }

@darryl-sw I don't think you have run code gen, I get a diff here, could you run make generate?

diff --git a/pkg/controller/ec2/vpcendpoint/zz_conversions.go b/pkg/controller/ec2/vpcendpoint/zz_conversions.go
index 2be39997..e52368a3 100644
--- a/pkg/controller/ec2/vpcendpoint/zz_conversions.go
+++ b/pkg/controller/ec2/vpcendpoint/zz_conversions.go
@@ -88,11 +88,6 @@ func GenerateVPCEndpoint(resp *svcsdk.DescribeVpcEndpointsOutput) *svcapitypes.V
                } else {
                        cr.Spec.ForProvider.VPCEndpointType = nil
                }
-               if elem.VpcId != nil {
-                       cr.Spec.ForProvider.VPCID = elem.VpcId
-               } else {
-                       cr.Spec.ForProvider.VPCID = nil
-               }
                found = true
                break
        }
@@ -177,9 +172,6 @@ func GenerateCreateVpcEndpointInput(cr *svcapitypes.VPCEndpoint) *svcsdk.CreateV
        if cr.Spec.ForProvider.VPCEndpointType != nil {
                res.SetVpcEndpointType(*cr.Spec.ForProvider.VPCEndpointType)
        }
-       if cr.Spec.ForProvider.VPCID != nil {
-               res.SetVpcId(*cr.Spec.ForProvider.VPCID)
-       }
 
        return res
 }

[darryl-sw]

@chlunde I've made the requested changes to isPolicyUpToDate and reran make generate.

[haarchri]

@darryl-sw can you rebase and run make generate again ? And squash commits ? After that we can merge the feature - thanks

[darryl-sw]

@haarchri running make generate with "ec2" added to GENERATED_SERVICES in the Makefile results in an error that I have not been able to resolve after rebasing.

13:15:38 [ OK ] go modules dependencies verified
13:15:39 [ .. ] go generate darwin_amd64
github.com/crossplane/provider-aws/apis/ec2/v1alpha1:-: CRD for Instance.ec2.aws.crossplane.io has no storage version
Error: not all generators ran successfully
run `controller-gen object:headerFile=../hack/boilerplate.go.txt paths=./... crd:trivialVersions=true,allowDangerousTypes=true,crdVersions=v1 output:artifacts:config=../package/crds -w` to see all available markers, or `controller-gen object:headerFile=../hack/boilerplate.go.txt paths=./... crd:trivialVersions=true,allowDangerousTypes=true,crdVersions=v1 output:artifacts:config=../package/crds -h` for usage
exit status 1
apis/generate.go:26: running "go": exit status 1

I took a look at the file package/crds/ec2.aws.crossplane.io_instances.yaml and identified that there were two versions, both named v1alpha1, both of them with storage: false.

To further isolate the problem, I verified that simply adding "ec2" to GENERATED_SERVICES on the master branch and running make generate produces the same error message.

Would you, or anybody, be able to advice on how I can fix this please?

[haarchri]

@darryl-sw sorry i missed that we are blocked with ec2 services because of #876

[darryl-sw]

@haarchri shall I wait for #876 first before proceeding with the rebase then?

[hanlins]

Any updates on the PR? Blockers? cc @haarchri

[haarchri]

@darryl-sw you can go forward the blockers are solved in master branch - thanks

[xvzf]

@haarchri what's the ETA for this to be released? :)

[haarchri]

@darryl-sw can rebase and i can have a look - in the next days we will cut next release - so i have time for review since rebase is done

[darryl-sw]

@haarchri rebased and completed another round of upstream testing. :)

[AaronME]

Thanks @darryl-sw for your contribution. I was able to create and delete a VPCEndpoint using this.

@haarchri this is good to rebase and merge.

[haarchri]

i will rebase now and adjust generated resolvers to get this PR merged ;)

[haarchri]

@AaronME done!
can you check build submodule ? looks like we see changes in this PR - after that we are ready to merge =)

tested also on my side:

NAME                                                   READY   SYNCED   EXTERNAL-NAME
vpcendpoint.ec2.aws.crossplane.io/sample-vpcendpoint   True    True     vpce-0037d4f0338c77881