Prefer least-idle connection when randomly selectiong broker for meta…

…data queries, etc. (#2845)

Reasons:
- this connection is most likely to function properly.
- allows truly idle connections to be killed by the broker's/LB's
  idle connection reaper.

This will fix issues like "Metadata request timed out" when the periodic
metadata refresh picks an idle (typically bootstrap) connection which
has exceeded the load-balancer's idle time (which silently kills the
connection without sending FIN/RST to the client).

CHANGELOG.md

@@ -13,6 +13,9 @@ librdkafka.
    topics to be propagated throughout the cluster before reporting them
    as non-existent. This fixes race issues where CreateTopics() is
    quickly followed by produce().
+ * Prefer least idle connection for periodic metadata refreshes, et.al.,
+   to allow truly idle connections to time out and to avoid load-balancer-killed
+   idle connection errors (#2845)
  * Added Test scenarios which define the cluster configuration
  * Add MinGW-w64 builds (@ed-alertedh, #2553)
  * `./configure --enable-XYZ` now requires the XYZ check to pass,

src/rdkafka.c

@@ -1816,9 +1816,9 @@ static void rd_kafka_metadata_refresh_cb (rd_kafka_timers_t *rkts, void *arg) {
         rd_kafka_t *rk = rkts->rkts_rk;
         int sparse = 1;
 
-        /* Dont do sparse requests if there is a consumer group with an
-         * active subscription since subscriptions need to be able to match
-         * on all topics. */
+        /* Dont do sparse requests if there is a consumer group with an active
+         * wildcard subscription since it needs to be able to match on all
+         * topics. */
         if (rk->rk_type == RD_KAFKA_CONSUMER && rk->rk_cgrp &&
             rk->rk_cgrp->rkcg_flags & RD_KAFKA_CGRP_F_WILDCARD_SUBSCRIPTION)
                 sparse = 0;