21.11.0

[cgrinds]

Change Log

Releases

21.11.0 / 2021-11-08

Highlights of this major release include:

• Early access to ONTAP REST collector
• Support for Prometheus HTTP service discovery
• New MetroCluster dashboard
• Qtree and Quotas collection
• We heard your ask, and we made it happen. We've separated cDOT and 7mode dashboards so each can evolve independently
• Label sets allow you to add additional key-value pairs to a poller's metrics#538 and expose those labels in your dashboards
• Template merging was improved to keep your template changes separate from Harvest's
• Harvest poller's are more deterministic about picking free ports
• 37 bug fixes

IMPORTANT The LabelAgent value_mapping plugin is being deprecated in this
release and will be removed in the next release of Harvest. Use LabelAgent
value_to_num instead. See
docs
for details.

IMPORTANT After upgrade, don't forget to re-import all dashboards so you get new dashboard enhancements and fixes.
You can re-import via bin/harvest/grafana cli or from the Grafana UI.

IMPORTANT RPM and Debian packages will be deprecated in the future, replaced
with Docker and native binaries. See
#330 for details and tell us
what you think. Several of you have already weighed-in. Thanks! If you haven't,
please do.

Known Issues

The Unix collector is unable to monitor pollers running in containers. See #249 for details.

Enhancements

• 🚧 ONTAP started moving their APIs from ZAPI to REST in ONTAP 9.6. Harvest adds an early access ONTAP REST collector in this release (config only). 🎊 This is our first step among several as we prepare for the day that ZAPIs are turned off. The REST collector and seven templates are included in 21.11. These should be considered early access as we continue to improve them. If you try them out or have any feedback, let us know on Slack or GitHub. #402

• Harvest should have a Prometheus HTTP service discovery end-point to make it easier to add/remove pollers #575

• Harvest should include a MetroCluster dashboard #539 Thanks @darthVikes for reporting

• Harvest should collect Qtree and Quota metrics #522 Thanks @jmg011 for reporting and validating this works in your environment

• SVM dashboard: Make NFS version a variable. SVM variable should allow selecting all SVMs for a cluster wide view #454

• Harvest should monitor ONTAP chassis sensors #384 Thanks to @hashi825 for raising this issue and reviewing the pull request

• Harvest cluster dashboard should include All option in dropdown for clusters #630 thanks @topspeed for raising this on Slack

• Harvest should collect volume sis status #519 Thanks to @jmg011 for raising

• Separate cDOT and 7-mode dashboards allowing each to change independently #489 #501 #547

• Improve collector and object template merging and documentation #493 #555 Thanks @hashi825 for reviewing and suggesting improvements

• Harvest should support label sets, allowing you to add additional key-value pairs to a poller's metrics#538

• bin/grafana import should create a matching label and rewrite queries to use chained variable when using label sets #550

• Harvest poller's should reuse their previous Prometheus port when restarted and be more deterministic about picking free ports #596 #595 Thanks to @cordelster for reporting

• Improve instantiated systemd template by specifying user/group, requires, and moving Unix pollers to the end of the list. #643 Thanks to @mamoep for reporting and providing the changes! ✨

• Harvest's Docker container should use local conf directory instead of copying into image. Makes upgrade and changing template files easier. #511

• Improve Disk dashboard by showing total number of disks by node and aggregate #583

• Harvest 7-mode dashboards should be provisioned when using Docker Compose workflow #544

• When upgrading, bin/harvest grafana import should add dashboards to a release-named folder so earlier dashboards are not overwritten #616

• client_timeout should be overridable in object template files #563

• Increase ZAPI client timeouts for volume and workloads objects #617

• Doctor: Ensure that all pollers export to unique Prometheus ports #597

• Improve execution performance of Harvest management commands 🚀 bin/harvest start|stop|restart #600

• Include eight cDOT dashboards that use InfluxDB datasource #466. Harvest does not support InfluxDB dashboards for 7-mode. Thanks to @SamyukthaM for working on these

• Docs: Describe how Harvest converts template labels into Prometheus labels #585

• Docs: Improve Matrix documentation to better align with code #485

• Docs: Improve ARCHITECTURE.md #603

Fixes

• Poller should report metadata when running on BusyBox #529 Thanks to @charz for reporting issue and providing details

• Space used % calculation was incorrect for Cluster and Aggregate dashboards #624 Thanks to @faguayot and @jorbour for reporting.

• When ONTAP indicates a counter is deprecated, but a replacement is not provided, continue using the deprecated counter #498

• Harvest dashboard panels must specify a Prometheus datasource to correctly handles cases were a non-Prometheus default datasource is defined in Grafana. #639 Thanks for reporting @MrObvious

• Prometheus datasource was missing on five dashboards (Network and Disk) #566 Thanks to @survive-wustl for reporting

• Document permissions that Harvest requires to monitor ONTAP with a read-only user #559 Thanks to @survive-wustl for reporting and working with us to chase this down. 👍

• Metadata dashboard should show correct status for running/stopped pollers #567 Thanks to @cordelster for reporting

• Harvest should serve a human-friendly 🌽 overview page of metric types when hitting the Prometheus end-point #613 Thanks @cordelster for reporting

• SnapMirror plugin should include source_node #608

• Disk dashboard should use better labels in table details #578

• SVM dashboard should show correct units and remove duplicate graph #454

• FCP plugin should work with 7-mode clusters #464

• Node values are missing from some 7-mode perf counters #467

• Nic state is missing from several network related dashboards 486

• Reduce log noise when templates are not found since this is often expected #606

• Use diagnosis-config-get-iter to collect node status from 7-mode systems #499

• Node status is missing from 7-mode #527

• Improve 7-mode templates. Remove cluster from 7-mode. yamllint all templates #531

• When saving Grafana auth token, make sure bin/grafana writes valid Yaml #544

• Improve Yaml parsing when different levels of indention are used in harvest.yml. You should see fewer invalid indentation messages. 👏 #626

• Unix poller should ignore /proc files that aren't readable #249

---

This discussion was created from the release 21.11.0.