Make connections.max.idle.ms configurable

[PSanetra]

The Azure documentation recommends to set connections.max.idle.ms to 180000:
https://learn.microsoft.com/en-us/azure/event-hubs/apache-kafka-configurations#producer-and-consumer-configurations

That option is exposed in the Confluent.Kafka.ClientConfig class via the ConnectionsMaxIdleMs property, but it seems like it is not possible to configure this using this library.

[shrohilla]

@PSanetra could you please share your use-case we need to understand if this issue is not a blocker for you.

[PSanetra]

@shrohilla I am not sure anymore, but can you ask someone at Microsoft why they recommend that setting?

[Gebumgar]

This thread provides some context around what this configuration does and why setting it under 180000 is recommended in Azure: confluentinc/confluent-kafka-dotnet#1544.

This thread provides some more background on the issues that were previously experienced with Kafka and timeouts on idle TCP connections: confluentinc/librdkafka#3109

The expected benefit of implementing this in Azure Functions would be a reduction in the logs that are thrown when the Kafka Client implemented in the Functions extension tries to communicate over an idle connection that has been closed by the Azure network. They commonly look like this and are frequently thrown because of metadata requests on idle topics: "Libkafka: [thrd:GroupCoordinator]: GroupCoordinator: xxx.servicebus.windows.net:9093: 1 request(s) timed out: disconnect (after 61269172ms in state UP, 1 identical error(s) suppressed)".

These errors don't cause Functions to fail because the underlying Kafka library just reopens a new connection when these errors occur. However, implementing this configuration in the Functions Extension would reduce the number of noisy timeout logs that are written from Kafka.

Some other related threads and issues:
confluentinc/confluent-kafka-dotnet#1544
#239
#197