probes emit log noise when containers stop

[rade]

I have no containers running except scope 0.16.1, and every few seconds the weavescope container log shows a triplet of events like this

<probe> INFO: 2016/07/14 18:41:54.975843 docker container: collecting stats for e678d8136f448933f6d04317e6fbab3d5f8fe5743804d833947e28ced162183c
<probe> ERRO: 2016/07/14 18:41:55.059077 docker container: error reading event, did container stop? read unix @->/var/run/docker.sock: use of closed network connection
<probe> INFO: 2016/07/14 18:41:55.059097 docker container: stopped collecting stats for e678d8136f448933f6d04317e6fbab3d5f8fe5743804d833947e28ced162183c

...

<probe> INFO: 2016/07/14 18:42:05.212411 docker container: collecting stats for 0aaf7f771a15411c0d3ca9104624347fc6222a106908de1f1c3a85a25b633c33
<probe> ERRO: 2016/07/14 18:42:05.298484 docker container: error reading event, did container stop? read unix @->/var/run/docker.sock: use of closed network connection
<probe> INFO: 2016/07/14 18:42:05.298508 docker container: stopped collecting stats for 0aaf7f771a15411c0d3ca9104624347fc6222a106908de1f1c3a85a25b633c33

The events do also get triggered when I start and then stop a container. So I am guessing that perhaps scope itself is starting and stopping a container every few seconds. In any case though, starting and stopping of containers is not something we should record in the logs at INFO let alone ERRO level.

[rade]

Switching the INFOs to DEBUG is easy, but I am little uncomfortable with swallowing all errors coming back from decoder.Decode(). The loop that is in is bailing on io.EOF. Evidently it's not getting that but a *net.OpError containing errClosing. I would like to treat that specific error the same as io.EOF, but because of the well-known problem of errClosing being private, the only way to do that is checking for the error string. urgh.

@peterbourgon any better idea?

[peterbourgon]

Strong +1 to moving the collecting stats for and stopped collecting stats for logging to DEBUG, or removing it entirely. Beyond that, you correctly summarize the situation. You could check !err.(*net.OpError).Temporary() but it might be too coarse-grained. I don't see an alternative to checking the string; as it is a log optimization rather than a correctness thing, I don't see a huge problem with it.