feat: Enable use of secrets via SecretProvider for MQTT broker credentials

[lenny-goodell]

PR Checklist

Please check if your PR fulfills the following requirements:

• The commit message follows our guidelines: https://wiki.edgexfoundry.org/display/FA/Contributor%27s+Guide
• Tests for the changes have been added (for bug fixes / features)
• Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

• Bugfix
• Feature
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• Build related changes
• CI related changes
• Documentation content changes
• Other... Please describe:

What is the current behavior?

MQTT broker credentials are in plain text configuration. No way to get them from SecretStore (aka Vault)

Issue Number: #159

What is the new behavior?

Via use of SpecretProvider, MQTT broker credentials can be pulled from a SecretStore. The SecretStore is Vault when running in secure mode or InsecureSecrets configuration section when running non-secure mode.

This PR is dependent on the following PRs:

edgexfoundry/go-mod-bootstrap#141
edgexfoundry/device-sdk-go#707

Does this PR introduce a breaking change?

• Yes
• No

Are there any new imports or modules? If so, what are they used for and why?

no

Are there any specific instructions or things that should be known prior to reviewing?

Do the following steps to test this PR:

1. Cone developer-scripts
2. cd to releases/nightly-build/compose-files
3. Create password file and config file for MQTT
   • run mkdir mqtt-config
   • run echo 'edgex:$6$OV0KYbPEN4xbWLt9$MkEhwMRMZ8tTYvrlKHnUmZDJtzJGN1RcKNnoM1jNm7zzSgwQo9M0aAAB/8oTqCSQyVy1a42L7jO9xOsuNC9uhg==' > mqtt-config/passwords
   • run echo 'allow_anonymous false' > mqtt-config/mosquitto.conf
   • run echo 'password_file /mosquitto/config/passwords' >> mqtt-config/mosquitto.conf
4. Add MQTT Broker
   • Add the following to nightly-build secure compose file (docker-compose-nexus.yml)

  mqtt-broker:
    container_name: edgex-mqtt-broker
    hostname: edgex-mqtt-broker
    image: eclipse-mosquitto:1.6.3
    networks:
      edgex-network: { }
    ports:
      - 1883:1883/tcp
    volumes:
    - ./mqtt-config:/mosquitto/config

4. Add ADD_SECRETSTORE_TOKENS: "device-mqtt" to Vault Work in compose file.
5. Run the compose file
   • make run
6. MQTT Broker is now running with user/password auth

• User is edge and Password is password
• Use a tool like MQTT.fx to connect to verify the user and password are set correctly and are required.

6. clone the branch for this PR
7. Build Device MQTT
8. cd cmd
9. run sudo WRITABLE_LOGLEVEL=DEBUG EDGEX_SECURITY_SECRET_STORE=true ./device-mqtt -c ./res/example

• Expect to see DEBUG message retrying Using Secrets URL of http://localhost:8200/v1/secret/edgex/device-mqtt/credentials`
• Expect to see ERROR message like "Unable to retrieve MQTT credentials from SecretProvider at path 'credentials': Error found on handling secrets from underlying data-store: Received a '404' response from the secret store. Retrying for 9.9559743s"
  10 Store MQTT credentials
• You have 60secs to store the MQTT credential via /api/v2/secret endpoint before service fails.
  • This can be increased in config via Driver.CredentialsRetryTime setting
• While DEBUG messages are being logged send the following via REST POST to http://localhost:49982/api/v2/secret

{
  "path" : "credentials",
  "secretData" : [
    {
      "key" : "username",
      "value" : "edgex"
    },
	{
      "key" : "password",
      "value" : "password"
    }
  ]
}

 - Expected response it:

{
    "apiVersion": "v2",
    "requestId": "",
    "message": "",
    "statusCode": 201
}

• The following messages will then be logged:

 "[Response listener] Start command response listening."
 "[Incoming listener] Start incoming data listening. "

• Device Service is ready to accept async data from a device

10. From a tool like MQTT.fx publish the following to the DataTopic topic

{
  "name" : "MQTT-test-device",
  "cmd" : "message",
  "message" : "Hellow World"
}

• "SendEvent: Pushed event to core data" should be logged.

11. Use core-data REST API to confirm reading received by sending GET to localhost:48080/api//v1/reading/name/message/0

Other information

[codecov-io]

Codecov Report

Merging #197 (4ee44f0) into master (cdd6c33) will decrease coverage by 2.03%.
The diff coverage is 2.38%.

@@            Coverage Diff             @@
##           master     #197      +/-   ##
==========================================
- Coverage   27.64%   25.60%   -2.04%     
==========================================
  Files           5        5              
  Lines         463      492      +29     
==========================================
- Hits          128      126       -2     
- Misses        321      352      +31     
  Partials       14       14              

Impacted Files	Coverage Δ
internal/driver/driver.go	16.78% <0.00%> (-0.51%)	⬇️
internal/driver/incominglistener.go	0.00% <0.00%> (ø)
internal/driver/responselistener.go	0.00% <0.00%> (ø)
internal/driver/config.go	50.00% <5.00%> (-31.82%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cdd6c33...4ee44f0. Read the comment docs.

[tonyespy]

This looks pretty good. I followed your testing instructions and was able to get everything working as expected. That said, I did have a few minor comments line, some which need addressing.

[tonyespy]

Thanks for the changes/responses...

[iain-anderson]

LGTM