Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CloudSQLInstance stuck in ready:false with reason: not ready for use #226

Closed
prasek opened this issue May 1, 2020 · 6 comments
Closed

CloudSQLInstance stuck in ready:false with reason: not ready for use #226

prasek opened this issue May 1, 2020 · 6 comments
Assignees
@hasheddan
Labels
cc

Comments

@prasek
Copy link
Contributor

prasek commented May 1, 2020 •
edited
Loading

CloudSQLInstance stuck in ready:false with reason: not ready for use

Using Crossplane v0.10 and related packages.

status:
  atProvider:
    backendType: SECOND_GEN
    connectionName: 'crossplane-playground:us-west1:workspace1-my-cool-app-sql-pb9vp'
    gceZone: us-west1-b
    project: crossplane-playground
    selfLink: >-
      https://sqladmin.googleapis.com/sql/v1beta4/projects/crossplane-playground/instances/workspace1-my-cool-app-sql-pb9vp
    settingsVersion: 1
    state: FAILED
  conditions:
    - lastTransitionTime: '2020-05-01T22:59:59Z'
      reason: Successfully reconciled resource
      status: 'True'
      type: Synced
    - lastTransitionTime: '2020-05-01T23:01:00Z'
      reason: Resource is not available for use
      status: 'False'
      type: Ready

With underlying error from GCP console:

 Failed to create subnetwork. Please create Service Networking connection with service 'servicenetworking.googleapis.com' from consumer project '283222062215' network 'my-cool-stack-network' again.

But the Connection shows

status:
  atProvider:
    peering: servicenetworking-googleapis-com
    service: services/servicenetworking.googleapis.com
  conditions:
    - lastTransitionTime: '2020-05-01T22:59:54Z'
      reason: Successfully reconciled resource
      status: 'True'
      type: Synced
    - lastTransitionTime: '2020-05-01T23:00:27Z'
      reason: Resource is available for use
      status: 'True'
      type: Ready
Full Resources 
apiVersion: database.gcp.crossplane.io/v1beta1
kind: CloudSQLInstance
metadata:
  annotations:
    crossplane.io/external-name: workspace1-my-cool-app-sql-pb9vp
  creationTimestamp: '2020-05-01T22:47:25Z'
  finalizers:
    - finalizer.managedresource.crossplane.io
  generateName: workspace1-my-cool-app-sql-
  generation: 4
  name: workspace1-my-cool-app-sql-pb9vp
  resourceVersion: '989601'
  selfLink: >-
    /apis/database.gcp.crossplane.io/v1beta1/cloudsqlinstances/workspace1-my-cool-app-sql-pb9vp
  uid: de1f7820-041c-4b38-9eb5-310b8984d462
spec:
  claimRef:
    apiVersion: database.crossplane.io/v1alpha1
    kind: MySQLInstance
    name: my-cool-app-sql
    namespace: workspace1
    uid: 14d3355a-4170-4efb-b72a-1826f4b57b4c
  classRef:
    apiVersion: database.gcp.crossplane.io/v1beta1
    kind: CloudSQLInstanceClass
    name: my-cool-stack-cloudsqlinstanceclass-mysql
    uid: 92778769-1b8b-4350-8a71-68b08e202497
  forProvider:
    databaseVersion: MYSQL_5_7
    gceZone: us-west1-b
    instanceType: CLOUD_SQL_INSTANCE
    region: us-west1
    settings:
      activationPolicy: ALWAYS
      backupConfiguration:
        startTime: '15:00'
      dataDiskSizeGb: 10
      dataDiskType: PD_SSD
      ipConfiguration:
        ipv4Enabled: false
        privateNetwork: projects/crossplane-playground/global/networks/my-cool-stack-network
        privateNetworkRef:
          name: my-cool-stack-network
      locationPreference:
        zone: us-west1-b
      pricingPlan: PER_USE
      replicationType: SYNCHRONOUS
      storageAutoResize: true
      tier: db-n1-standard-1
      userLabels:
        crossplane-class: my-cool-stack-cloudsqlinstanceclass-mysql
        crossplane-kind: cloudsqlinstance_database_gcp_crossplane_io
        crossplane-name: workspace1-my-cool-app-sql-pb9vp
        crossplane-provider: my-cool-stack-gcp-provider
  providerRef:
    name: my-cool-stack-gcp-provider
  reclaimPolicy: Delete
  writeConnectionSecretToRef:
    name: 14d3355a-4170-4efb-b72a-1826f4b57b4c
    namespace: crossplane-system
status:
  atProvider:
    backendType: SECOND_GEN
    connectionName: 'crossplane-playground:us-west1:workspace1-my-cool-app-sql-pb9vp'
    gceZone: us-west1-b
    project: crossplane-playground
    selfLink: >-
      https://sqladmin.googleapis.com/sql/v1beta4/projects/crossplane-playground/instances/workspace1-my-cool-app-sql-pb9vp
    settingsVersion: 1
    state: FAILED
  conditions:
    - lastTransitionTime: '2020-05-01T22:59:59Z'
      reason: Successfully reconciled resource
      status: 'True'
      type: Synced
    - lastTransitionTime: '2020-05-01T23:01:00Z'
      reason: Resource is not available for use
      status: 'False'
      type: Ready


apiVersion: servicenetworking.gcp.crossplane.io/v1beta1
kind: Connection
metadata:
  annotations:
    crossplane.io/external-name: my-cool-stack-connection
  creationTimestamp: '2020-05-01T22:45:52Z'
  finalizers:
    - finalizer.managedresource.crossplane.io
  generation: 3
  labels:
    core.crossplane.io/parent-group: gcp.stacks.crossplane.io
    core.crossplane.io/parent-kind: GCPSample
    core.crossplane.io/parent-name: my-cool-stack
    core.crossplane.io/parent-namespace: ''
    core.crossplane.io/parent-version: v1alpha1
  name: my-cool-stack-connection
  ownerReferences:
    - apiVersion: gcp.stacks.crossplane.io/v1alpha1
      blockOwnerDeletion: true
      kind: GCPSample
      name: my-cool-stack
      uid: 8821b257-28ca-4b67-ab21-14f022764d70
  resourceVersion: '989510'
  selfLink: >-
    /apis/servicenetworking.gcp.crossplane.io/v1beta1/connections/my-cool-stack-connection
  uid: 16906a89-065d-4f26-8474-d98aa7fae47a
spec:
  forProvider:
    network: projects/crossplane-playground/global/networks/my-cool-stack-network
    networkRef:
      name: my-cool-stack-network
    parent: services/servicenetworking.googleapis.com
    reservedPeeringRangeRefs:
      - name: my-cool-stack-globaladdress
    reservedPeeringRangeSelector: {}
    reservedPeeringRanges:
      - my-cool-stack-globaladdress
  providerRef:
    name: my-cool-stack-gcp-provider
  reclaimPolicy: Delete
status:
  atProvider:
    peering: servicenetworking-googleapis-com
    service: services/servicenetworking.googleapis.com
  conditions:
    - lastTransitionTime: '2020-05-01T22:59:54Z'
      reason: Successfully reconciled resource
      status: 'True'
      type: Synced
    - lastTransitionTime: '2020-05-01T23:00:27Z'
      reason: Resource is available for use
      status: 'True'
      type: Ready
@jbw976
Copy link
Member

jbw976 commented May 1, 2020

that error message mentioned subnetwork is what failed, what's the status output of that object look like?

@prasek
Copy link
Contributor Author

prasek commented May 1, 2020

think it's the connection, the subnet looks ok:

subnetwork 
apiVersion: compute.gcp.crossplane.io/v1beta1
kind: Subnetwork
metadata:
  annotations:
    crossplane.io/external-name: my-cool-stack-subnetwork
  creationTimestamp: '2020-05-01T22:45:51Z'
  finalizers:
    - finalizer.managedresource.crossplane.io
  generation: 2
  labels:
    core.crossplane.io/parent-group: gcp.stacks.crossplane.io
    core.crossplane.io/parent-kind: GCPSample
    core.crossplane.io/parent-name: my-cool-stack
    core.crossplane.io/parent-namespace: ''
    core.crossplane.io/parent-version: v1alpha1
  name: my-cool-stack-subnetwork
  ownerReferences:
    - apiVersion: gcp.stacks.crossplane.io/v1alpha1
      blockOwnerDeletion: true
      kind: GCPSample
      name: my-cool-stack
      uid: 8821b257-28ca-4b67-ab21-14f022764d70
  resourceVersion: '989403'
  selfLink: /apis/compute.gcp.crossplane.io/v1beta1/subnetworks/my-cool-stack-subnetwork
  uid: 00ac49c2-37c9-45e0-8c54-887654534e03
spec:
  forProvider:
    ipCidrRange: 192.168.0.0/24
    network: projects/crossplane-playground/global/networks/my-cool-stack-network
    networkRef:
      name: my-cool-stack-network
    privateIpGoogleAccess: true
    region: us-west1
    secondaryIpRanges:
      - ipCidrRange: 10.128.0.0/20
        rangeName: pods
      - ipCidrRange: 172.16.0.0/16
        rangeName: services
  providerRef:
    name: my-cool-stack-gcp-provider
  reclaimPolicy: Delete
status:
  atProvider:
    creationTimestamp: '2020-05-01T15:59:53.476-07:00'
    fingerprint: NoqUNaZmDZw=
    gatewayAddress: 192.168.0.1
    id: 8151176291559741000
    selfLink: >-
      https://www.googleapis.com/compute/v1/projects/crossplane-playground/regions/us-west1/subnetworks/my-cool-stack-subnetwork
  conditions:
    - lastTransitionTime: '2020-05-01T22:59:53Z'
      reason: Successfully reconciled resource
      status: 'True'
      type: Synced
    - lastTransitionTime: '2020-05-01T22:59:54Z'
      reason: Resource is available for use
      status: 'True'
      type: Ready

@prasek
Copy link
Contributor Author

prasek commented May 1, 2020 •
edited
Loading

and underlying VPC peering looks ok:
image

@prasek
Copy link
Contributor Author

prasek commented May 2, 2020 •
edited
Loading

Deleting the MySQLInstance claim and letting app-wordpress re-create it solved the issue. Would be nice for Crossplane to detect this and try deleting and recreating the underlying managed resource, when these terminal error conditions are encountered.

//cc @hasheddan

@prasek prasek added the cc label May 2, 2020
@Cicatrice
Copy link

We are using CNRM on our GKE cluster, and we have the same error ; this maybe related to this issue : GoogleCloudPlatform/k8s-config-connector#148

@muvaf
Copy link
Member

muvaf commented Feb 24, 2021

I think this had been solved by a small hack that we borrowed from Terraform implementation. Feel free to reopen if it still happens.

@muvaf muvaf closed this as completed Feb 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hasheddan hasheddan

Labels
cc
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@Cicatrice @prasek @jbw976 @muvaf @hasheddan