No such host when restoring an ark backup

[patrocinio]

What steps did you take and what happened:
[A clear and concise description of what the bug is, and what commands you ran.)
I created a Kubernetes environment in a Public Cloud provider, then I followed the instructions I am creating at https://medium.com/@epatro/using-ark-to-back-up-your-applications-deployed-to-ibm-cloud-private-4f2e2dc0ce91 to deploy a sample application (Redis) and restore it.

What did you expect to happen:
When I try to restore ark backup, I am getting the error below:

The output of the following commands will help us better understand what's going on:
(Pasting long output into a GitHub gist or other pastebin is fine.)

patro:~ edu$ ark restore describe my-backup-20180911181631
Name:         my-backup-20180911181631
Namespace:    heptio-ark
Labels:       <none>
Annotations:  <none>

Backup:  my-backup

Namespaces:
  Included:  *
  Excluded:  <none>

Resources:
  Included:        *
  Excluded:        nodes, events, events.events.k8s.io, backups.ark.heptio.com, restores.ark.heptio.com
  Cluster-scoped:  auto

Namespace mappings:  <none>

Label selector:  <none>

Restore PVs:  auto

Phase:  Completed

Validation errors:  <none>

Warnings:  <error getting warnings: Get http://minio.heptio-ark.svc:9000/ark/my-backup/restore-my-backup-20180911181631-results.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minio%2F20180911%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20180911T223336Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=7d076c68eceef9d28cf8ad7c97f9c4bb2bb5d52c7361532aca048de289bea824: dial tcp: lookup minio.heptio-ark.svc on 172.16.0.11:53: no such host>

Errors:  <error getting errors: Get http://minio.heptio-ark.svc:9000/ark/my-backup/restore-my-backup-20180911181631-results.gz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=minio%2F20180911%2Fminio%2Fs3%2Faws4_request&X-Amz-Date=20180911T223336Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=7d076c68eceef9d28cf8ad7c97f9c4bb2bb5d52c7361532aca048de289bea824: dial tcp: lookup minio.heptio-ark.svc on 172.16.0.11:53: no such host>

• kubectl logs deployment/ark -n heptio-ark
• ark backup describe <backupname> or kubectl get backup/<backupname> -n heptio-ark -o yaml
• ark backup logs <backupname>
• ark restore describe <restorename> or kubectl get restore/<restorename> -n heptio-ark -o yaml
• ark restore logs <restorename>

Anything else you would like to add:
[Miscellaneous information that will assist in solving the issue.]
172.16.0.11 is the IP of my iam-token-service:

patro:scripts edu$ k get svc --all-namespaces | grep 172.16.0.11
kube-system     iam-token-service                             ClusterIP      172.16.0.11    <none>        10443/TCP                                                       15d

Environment:

• Ark version (use ark version):

patro:scripts edu$ ark version
Version: v0.9.4
Git commit: v0.9.4
Git tree state: clean

• Kubernetes version (use kubectl version):

patro:scripts edu$ kubectl version
Client Version: version.Info{Major:"1", Minor:"11", GitVersion:"v1.11.1", GitCommit:"b1b29978270dc22fecc592ac55d903350454310a", GitTreeState:"clean", BuildDate:"2018-07-17T18:53:20Z", GoVersion:"go1.10.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.0+icp", GitCommit:"1c6988c2361d678dab8ea5ddfab82cb52680b20b", GitTreeState:"clean", BuildDate:"2018-04-27T05:47:17Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes installer & version:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):

[nrb]

Hello @patrocinio - reading the blog post, it looks like the instructions ask to kubectl apply the entire examples/minio directory. Without any edits to those example files, the Ark config will look for Minio at http://minio.heptio-ark.svc:9000. This likely isn't the right URL within your cluster, which is why it's reporting as not being found.

What is the output of kubectl get deployments -l component=minio?

Also, thanks for taking the time to spread the work about Ark! Let us know when the blog post is ready to be shared and we'll be sure to give it a mention in our office hours.

[ncdc]

@patrocinio I believe the issue is that your ark client, which I'm guessing you're running on your personal computer, isn't able to resolve the DNS name of your minio service that's running in your cluster.

Our long-term approach for fixing this is to add a new configuration option for s3-compatible backup storage that allows you to specify the public DNS name to use when the ark client retrieves backup and restore logs and other detailed information. You could then configure ingress to the minio service and use that DNS name, which would need to be resolvable outside of the cluster.

In the mean time, you could try to switch the minio service to be a NodePort service, and update the s3url in your ark config to use the exposed NodePort.

[patrocinio]

Hello @ncdc and @nrb

Thanks for your recommendations.
I switched minio to use NodePort.

Now, I got a question/issue regarding the ark client (running in my laptop).
How do I pass the minio service? I don't use a KUBECONFIG, but I pass a bunch of options using kubectl options

Thank you!

[ncdc]

@patrocinio I'm not sure what you're asking - could you please clarify?

[patrocinio]

Hello @ncdc

thanks for your quick response.

Where do I specify the s3url information in the ark config?

[ncdc]

See https://heptio.github.io/ark/v0.9.0/config-definition#aws

…

On Thu, Sep 13, 2018 at 9:07 AM Edu Patrocinio ***@***.***> wrote: Hello @ncdc <https://github.com/ncdc> thanks for your quick response. Where do I specify the s3url information in the ark config? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#839 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAABYnCXAs9dNFXLwq6MCFTDwpEsFL5Kks5uaoLLgaJpZM4WkWMJ> .

[skriss]

Closed by #1023