InfluxDBClient.from_config_file() incompatible with config files generated by influx setup

[bgottula]

The method InfluxDBClient.from_config_file() does not seem to be compatible with the config file format generated by influx setup due to quotation marks around strings. Would it be better to make the client deal with the quotation marks correctly, perhaps by .strip()-ing the quote characters, or would it be better to change the config file format generated by influx setup?

Steps to reproduce:

1. Run the following in shell on new install of influxdb2 (added line breaks for clarity):

influx setup
--bucket=telem
--name=influx2  # name expected by InfluxDBClient.from_config_file()
--org=moms-sewing-room
--password=fhqwhgads
--username=edgar_jr
--force

This creates the file ~/.influxdbv2/configs where the first un-commented section is as below, note the quotation marks around all the strings:

[influx2]
  url = "http://localhost:8086"
  token = "7CkNpNw7A6rmeyc_vp0zYPGoOhcZ9zji9C0_RDG1rvb75hQ-JdMyOU81l4SfXJcumH3De1l7O85kODONyzYCKA=="
  org = "moms-sewing-room"
  active = true

2. Run the following in a Python interpreter:

import time
from datetime import datetime
from influxdb_client import InfluxDBClient, Point, WritePrecision
from influxdb_client.client.write_api import WriteOptions

client = InfluxDBClient.from_config_file('~/.influxdbv2/configs')
write_api = client.write_api(
        write_options=WriteOptions(
            batch_size=500,  # number of data points per batch
            flush_interval=1000,  # milliseconds before batch is written
            jitter_interval=0,
            retry_interval=1000,  # milliseconds before retry after failed write
            max_retries=5,
            max_retry_delay=10_000,  # give up after this many milliseconds
            exponential_base=2
        )
    )

point = Point.from_dict(
    dictionary={
        'measurement': 'my_measurement',
        'fields': {'field1': 1.0},
        'time': datetime.utcnow(),
    },
    write_precision=WritePrecision.NS,
)
write_api.write(bucket='telem', record=point)

# wait long enough for write attempts to fail
time.sleep(15)

Expected behavior:
Points are written to the bucket successfully.

Actual behavior:
Multiple unsuccessful attempts to write points, emitting the following to the console:

The retriable error occurred during request. Reason: '<urllib3.connection.HTTPConnection object at 0x7fbf5b9150d0>: Failed to establish a new connection: [Errno -2] Name or service not known'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPConnection object at 0x7fbf5b8fdbe0>: Failed to establish a new connection: [Errno -2] Name or service not known'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPConnection object at 0x7fbf5b913640>: Failed to establish a new connection: [Errno -2] Name or service not known'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPConnection object at 0x7fbf5b915970>: Failed to establish a new connection: [Errno -2] Name or service not known'.
The retriable error occurred during request. Reason: '<urllib3.connection.HTTPConnection object at 0x7fbf5b915640>: Failed to establish a new connection: [Errno -2] Name or service not known'.
The batch item wasn't processed successfully because: HTTPConnectionPool(host='"http', port=80): Max retries exceeded with url: //localhost:8086%22/api/v2/write?org=%22moms-sewing-room%22&bucket=telem&precision=ns (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbf5b913af0>: Failed to establish a new connection: [Errno -2] Name or service not known'))

The useful clue is this part:

Max retries exceeded with url: //localhost:8086%22/api/v2/write?org=%22moms-sewing-room%22&bucket=telem&precision=ns

Additional details:

Based on the error message, clearly the URL is missing the "http:" up front, and there are a bunch of "%22" instances scattered around, which is the URL encoding for ". Inspecting the InfluxDBClient object that was created:

In [5]: client.url
Out[5]: '"http://localhost:8086"'

The strings from the config file were captured with the quotation marks included (this applies to token and org as well). If I edit the config file to remove the quotation marks I'm able to write to the database with no problems.

User Franky1 replied to my post on the user forum with some insight:

The Python client uses the python standard library configparser to read the ini file. The configparser lib does not actually expect quotes in the ini file for strings as a value. If it does, the quotes are included in the read string. This seems to be the case here.

Specifications:

• Client Version: 1.14.0
• InfluxDB Version: 2.0.4
• Platform: Linux Mint 20.1 (based on Ubuntu Focal)

[bednar]

Hi @bgottula,

thanks for using our client.

As you mentioned the configuration for the client should be formatted as a ini file. How the format looks like is described in configparser documentation - https://docs.python.org/3/library/configparser.html.

The InfluxDB config file is formatted as a TOML - https://toml.io/en/. We will prepare an example how to configure client by: '~/.influxdbv2/configs'.

Regards

[bgottula]

Thanks for the comment. I suppose an example would be helpful, but I didn't report this because I couldn't think of a work-around. Was the selection of different and subtly incompatible formats for the client and server configuration files really a deliberate design decision? Forgive me if I find that quite surprising and silly.

[bednar]

We decided to select format which is common in Python community, but we are also open mined so We could support both types of format.

[bgottula]

Okay, fair enough. In my case supporting TOML in the Python client would be handy since I'm running the Python client and the server on the same machine, so the config file generated by influx setup is a convenient way to pass the URL, token, and org name over to the client without having to invent some other mechanism to pass those around. But it's easy enough to work around with this:

import toml
influx_config_filename = '~/.influxdbv2/configs'
config_file_dict = toml.load(influx_config_filename)
influxdb_client = InfluxDBClient(
    url=config_file_dict['influx2']['url'],
    token=config_file_dict['influx2']['token'],
    org=config_file_dict['influx2']['org'],
)

Which is good enough for me.

[ondrej-ivanko]

Hi Jakub @bednar, let me propose another solution. You can create a default ConfigParser, which can accept extra argument. For example.

class DefaultConfigParser(ConfigParser):
    def __init__(..., parse="toml")
        ...

    def get(self, section, option):
        val = ConfigParser.RawConfigParser.get(self, section, option)
        if self.parse = "toml":
            return val.strip('"')

Preferably easier way is to strip the value from get() function on line 86 and write:

config.get("influx2", option="url").strip.('"')

The solution @bgottula is possible, but ultimately does not eliminate the redundancy of code. I would like to reuse already existing configs.

Thank you.

[bednar]

Hi @ondrej-ivanko, thanks for suggestions...

config.get("influx2", option="url").strip.('"')

maybe the easier way will be the best ;)

[ondrej-ivanko]

Hello @bednar,

thank you for taking care of it. Please dont forget to apply the same approach to org and token as well.

config['influx2']['token'].strip('"')

Cheers.