Scope probe fails to report to some of the target apps

[2opremio]

When mirroring scope reports from dev to the new dev-ka cluster, I noticed that two hosts were missing.

Inspecting the logs of the two host probes show evidence of publishing to frontend.dev-ka.weave.works:80 not starting at all (there are no lines like Publish loop for [DevKAIP]:80 starting are present in the logs)

logs: https://gist.github.com/2opremio/5d9335c84bf9f2929acd54df1cc173af

goroutine dump: https://gist.github.com/2opremio/5d9335c84bf9f2929acd54df1cc173af#file-logs-txt-L2129

[2opremio]

After looking at the goroutine dump, there don't seem to be any signs of deadlocks.

[2opremio]

This could had been a DNS error due to kube-dns. To close this I propose:

1. Logging DNS errors to targets if/when they happen
2. Make sure DNS resolutions are retried when --probe.resolver= is provided (probably that's already the case).