HTTP-based metricbeat modules can no longer perform authentication

[ycombinator]

• Version: master / 8.0.0
• Operating System: Mac OS X
• Steps to Reproduce:
  1. Setup Elasticsearch with Trial license, Security enabled, and elastic user to have password as abc123.
  2. Enable Elasticsearch metricbeat module.

     ./metricbeat modules enable elasticsearch
     

  3. Uncomment and set username and password settings in the Elasticsearch metricbeat module configuration.

     username: elastic
     password: abc123
     

  4. Start Metricbeat.

     ./metricbeat -e
     

You will see a 401 Unauthorized error in the logs.

Looking at Wireshark, it looks like Metricbeat does not add the Authorization header, as one would expect it to with the above configuration:

I did a similar test with the Kibana Metricbeat module and ended up with the same results: 401 error in the logs and Wireshark showing no Authorization header.

[ycombinator]

Just tested with Metricbeat 6.7.0 (built from the 6.7 branch) and this was working as expected.

[ycombinator]

Just tested with Metricbeat 7.0.0 (built from the 7.0 branch) and this is working as expected.

[ycombinator]

Just tested with Metricbeat 7.1.0 (built from the 7.x branch) and this is working as expected.

[ycombinator]

Based on the above versions' testing, it appears that this regression was introduced in master / 8.0.0 recently, that is between the last time master was backported to 7.x and now.

@ruflin @jsoriano can you think of any recent PRs to master that might've introduced this regression?

[ycombinator]

@jsoriano @ruflin I did some git bisect testing and I think #11032 might've caused this regression. Would one of you mind confirming?

[jsoriano]

I'd also bet this regresion is introduced by #11032, investigating it.

@ycombinator could you try to reproduce this issue with 6.7.0 but setting a timeout? So the config is something like:

username: elastic
password: abc123
timeout: 2s

[jsoriano]

@ycombinator PR open to fix this #11353, good catch!