diff --git a/README.md b/README.md
index e09867e05..b5f56fba7 100644
--- a/README.md
+++ b/README.md
@@ -277,6 +277,8 @@ For a list of collector-specific parameters, see the documentation of each colle
### [Unix](cmd/collectors/unix/README.md)
+### [Creating/editing templates](conf/README.md)
+
### Labels
Labels offer a way to add additional key-value pairs to a poller's metrics. These allow you to tag a cluster's metrics in a cross-cutting fashion. Here's an example:
diff --git a/cmd/collectors/zapi/README.md b/cmd/collectors/zapi/README.md
index 099b2f2f0..6ebb29c70 100644
--- a/cmd/collectors/zapi/README.md
+++ b/cmd/collectors/zapi/README.md
@@ -65,102 +65,4 @@ aggr-attributes:
- disk-count => disks
```
-will force using `aggr_type` and `aggr_disks` for the label and the metric respectively.
-
-## Creating/editing subtemplates
-
-You can either read [ONTAP's documentation](https://mysupport.netapp.com/documentation/productlibrary/index.html?productID=60427) or use Harvest's `zapi` tool to explore available APIs and metrics on your cluster. Examples:
-
-```sh
-$ harvest zapi --poller <poller> show apis
- # will print list of apis that are available
- # usually apis with the "get-iter" suffix can provide useful metrics
-$ harvest zapi --poller <poller> show attrs --api volume-get-iter
- # will print the attribute tree of the API
-$ harvest zapi --poller <poller> show data --api volume-get-iter
- # will print raw data of the API attribute tree
-```
-
-(Replace `<poller>` with the name of a poller that can connect to an ONTAP system.)
-
-Instead of editing one of the existing templates, it's better to copy one and edit the copy. That way, your custom template will not be overwritten when upgrading Harvest. For example, if you want to change `conf/zapi/cdot/9.8.0/aggr.yaml`, first create a copy (e.g., `conf/zapi/cdot/9.8.0/custom_aggr.yaml`), then add these lines to `conf/zapi/custom.yaml`:
-
-```yaml
-objects:
- Aggregate: custom_aggr.yaml
-```
-
-After restarting your pollers, `aggr.yaml` will be ignored and the new, `custom_aggr.yaml` subtemplate will be used instead.
-
-### Example subtemplate
-
-In this example, we want to collect sensor metrics from the `environment-sensors-get-iter` API. These are the steps that we need to follow:
-
-#### 1. Create a new subtemplate
-
-Create the file `conf/zapi/cdot/9.8.0/sensor.yaml` (optionally replace `9.8.0` with the version of your ONTAP). Add following content:
-
-```yaml
-name: Sensor
-query: environment-sensors-get-iter
-object: sensor
-
-counters:
- environment-sensors-info:
- - critical-high-threshold => critical_high
- - critical-low-threshold => critical_low
- - ^discrete-sensor-state => discrete_state
- - ^discrete-sensor-value => discrete_value
- - ^^node-name => node
- - ^^sensor-name => sensor
- - ^sensor-type => type
- - ^threshold-sensor-state => threshold_state
- - threshold-sensor-value => threshold_value
- - ^value-units => unit
- - ^warning-high-threshold => warning_high
- - ^warning-low-threshold => warning_low
-
-export_options:
- include_all_labels: true
-```
-
-(See [#counters](#counters) for an explanation about the special symbols used).
-
-#### 2. Enable the new subtemplate
-
-To enable the new subtemplate, create `conf/zapi/custom.yaml` with the lines shown below.
-
-```yaml
-objects:
- Sensor: sensor.yaml
-```
-In the future, if you add more subtemplates, you can add those in this same file.
-
-#### 3. Test your changes and restart pollers
-
-Test your new `Sensor` template with a single poller like this:
-```
-./bin/harvest start <poller> --foreground --verbose --collectors Zapi --objects Sensor
-```
-Replace `<poller>` with the name of one of your ONTAP pollers.
-
-Once you have confirmed that the new template works, restart any already running pollers that you want to pick up the new template(s).
-
-### Check the metrics
-
-If you are using the Prometheus exporter, check the metrics on the HTTP endpoint with `curl` or a web browser. E.g., my poller is exporting its data on port `15001`. Adjust as needed for your exporter.
-
-```
-curl -s 'http://localhost:15001/metrics' | grep sensor_
-
-sensor_value{datacenter="WDRF",cluster="shopfloor",critical_high="3664",node="shopfloor-02",sensor="P3.3V STBY",type="voltage",warning_low="3040",critical_low="2960",threshold_state="normal",unit="mV",warning_high="3568"} 3280
-sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="P1.2V STBY",type="voltage",threshold_state="normal",warning_high="1299",warning_low="1105",critical_low="1086",node="shopfloor-02",critical_high="1319",unit="mV"} 1193
-sensor_value{datacenter="WDRF",cluster="shopfloor",unit="mV",critical_high="15810",critical_low="0",node="shopfloor-02",sensor="P12V STBY",type="voltage",threshold_state="normal"} 11842
-sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="P12V STBY Curr",type="current",threshold_state="normal",unit="mA",critical_high="3182",critical_low="0",node="shopfloor-02"} 748
-sensor_value{datacenter="WDRF",cluster="shopfloor",critical_low="1470",node="shopfloor-02",sensor="Sysfan2 F2 Speed",type="fan",threshold_state="normal",unit="RPM",warning_low="1560"} 2820
-sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="PSU2 Fan1 Speed",type="fan",threshold_state="normal",unit="RPM",warning_low="4600",critical_low="4500",node="shopfloor-01"} 6900
-sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="PSU1 InPwr Monitor",type="unknown",threshold_state="normal",unit="mW",node="shopfloor-01"} 132000
-sensor_value{datacenter="WDRF",cluster="shopfloor",critical_high="58",type="thermal",unit="C",warning_high="53",critical_low="0",node="shopfloor-01",sensor="Bat Temp",threshold_state="normal",warning_low="5"} 24
-sensor_value{datacenter="WDRF",cluster="shopfloor",critical_high="9000",node="shopfloor-01",sensor="Bat Charge Volt",type="voltage",threshold_state="normal",unit="mV",warning_high="8900"} 8200
-sensor_value{datacenter="WDRF",cluster="shopfloor",node="shopfloor-02",sensor="PSU1 InPwr Monitor",type="unknown",threshold_state="normal",unit="mW"} 132000
-```
+will force using `aggr_type` and `aggr_disks` for the label and the metric respectively.
\ No newline at end of file
diff --git a/cmd/collectors/zapiperf/README.md b/cmd/collectors/zapiperf/README.md
index 113e3da51..3f9c9ff12 100644
--- a/cmd/collectors/zapiperf/README.md
+++ b/cmd/collectors/zapiperf/README.md
@@ -48,39 +48,6 @@ Parameters in poller section should define (at least) the address and authentica
It is recommended creating a read-only user on the ONTAP system dedicated to Harvest. See section [Authentication](#authentication) for guidance.
-We can define the configuration file of the collector. If no configuration file is specified, the default configuration file (`conf/zapiperf/default.yaml`) will be used and if the file `conf/zapiperf/default.yaml` is present, it will be merged to the default one. If we specify our own configuration file for the collector, it can have any name, and it will not be merged.
-
-Examples:
-
-Define a poller that will run the ZapiPerf collector using its default configuration file:
-
-```yaml
-Pollers:
- jamaica: # name of the poller
- datacenter: munich
- addr: 10.65.55.2
- auth_style: basic_auth
- username: harvest
- password: 3t4ERTW%$W%c
- collectors:
- - ZapiPerf # will use conf/zapiperf/default.yaml and optionally merge with conf/zapiperf/custom.yaml
-```
-
-Define a poller that will run the ZapiPerf collector using a custom configuration file:
-
-```yaml
-Pollers:
- jamaica: # name of the poller
- addr: 10.65.55.2
- auth_style: basic_auth
- username: harvest
- password: 3t4ERTW%$W%c
- collectors:
- - ZapiPerf:
- - limited.yaml # will use conf/zapiperf/limited.yaml
- # if more templates are added, they will be merged
-```
-
### Collector configuration file
This configuration file (the "template") contains a list of objects that should be collected and the filenames of their configuration (explained in the next section).
@@ -140,89 +107,3 @@ Parameters in this section tell the exporters how to handle the collected data.
* `instances_keys` (list): display names of labels to export with each data-point
* `instance_labels` (list): display names of labels to export as a separate data-point
* `include_all_labels` (bool): export all labels with each data-point (overrides previous two parameters)
-
-## Creating/editing subtemplates
-
-You can use Harvest's `zapi` to explore available objects and counters on your cluster. Examples:
-
-```sh
-$ harvest zapi --poller <poller> show objects
- # will print the list of zapiperf objects
-$ harvest zapi --poller <poller> show counters --object ip
- # will print the list of counters of ip object
-$ harvest zapi --poller <poller> export counters --object ip
- # will export the list of counters into a subtemplate
-```
-Instead of editing one of the existing templates, it's better to copy one and edit the copy. That way, your custom template will not be overwritten when upgrading Harvest. For example, if you want to change `conf/zapiperf/cdot/9.8.0/volume.yaml`, first create a copy (e.g., `conf/zapiperf/cdot/9.8.0/custom_volume.yaml`), then add these lines to `conf/zapiperf/custom.yaml` to override the default subtemplate:
-
-```yaml
-objects:
- Volume: custom_volume.yaml
-```
-
-### Example subtemplate
-
-In this example, we want to collect metrics of the `ip` object. These are the steps that we need to follow:
-
-#### 1. Create new subtemplate
-
-Create `conf/zapiperf/cdot/9.8.0/ip.yaml` with the following content:
-
-```yaml
-name: IP
-query: ip
-object: ip
-instance_key: uuid
-
-counters:
- - instance_name
- - instance_uuid
- - node_name => node
- - packets_delivered
- - packets_forwarded
- - packets_redirected
- - packets_unforwardable
-
-export_options:
- instance_keys:
- - ip
- - node
-```
-
-### Enable the new template
-
-To enable the new template, create `conf/zapiperf/custom.yaml` with the lines shown below.
-
-In the future, if you add more templates, you can add those in this same file.
-
-```yaml
-objects:
- IP: ip.yaml
-```
-
-### Test your changes and restart pollers
-
-Test your new `IP` template with a single poller like this:
-```
-./bin/harvest start <poller> --foreground --verbose --collectors ZapiPerf --objects IP
-```
-Replace `<poller>` with the name of one of your ONTAP pollers.
-
-Once you have confirmed that the new template works, restart any already running pollers that you want to pick up the new template(s).
-
-### Check the metrics
-
-If you are using the Prometheus exporter, check the metrics on the HTTP endpoint with `curl` or a web browser. E.g., my poller is exporting its data on port `15001`. Adjust as needed for your exporter. Note that the ZapiPerf collector will emit metrics only after the second poller, so you have to wait about 1 minute.
-
-```
-curl -s 'http://localhost:15001/metrics' | grep ip_
-ip_packets_unforwardable{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967294",node="shopfloor-02"} 0
-ip_packets_delivered{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967294",node="shopfloor-02"} 4245
-ip_packets_unforwardable{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-02"} 0
-ip_packets_delivered{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-02"} 0
-ip_packets_redirected{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-02"} 0
-ip_packets_forwarded{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-02"} 0
-ip_packets_unforwardable{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-01"} 0
-ip_packets_delivered{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-01"} 0
-ip_packets_redirected{datacenter="WDRF",cluster="shopfloor",ip="ip_ipsid_0014967293",node="shopfloor-01"} 0
-```
diff --git a/conf/README.md b/conf/README.md
new file mode 100644
index 000000000..912548c89
--- /dev/null
+++ b/conf/README.md
@@ -0,0 +1,254 @@
+## Creating/editing templates
+
+You can either read [ONTAP's documentation](https://mysupport.netapp.com/documentation/productlibrary/index.html?productID=60427) or use Harvest's `zapi` tool to explore available APIs and metrics on your cluster. Examples:
+
+```sh
+$ harvest zapi --poller <poller> show apis
+ # will print list of apis that are available
+ # usually apis with the "get-iter" suffix can provide useful metrics
+$ harvest zapi --poller <poller> show attrs --api volume-get-iter
+ # will print the attribute tree of the API
+$ harvest zapi --poller <poller> show data --api volume-get-iter
+ # will print raw data of the API attribute tree
+```
+
+(Replace `<poller>` with the name of a poller that can connect to an ONTAP system.)
+
+This document scope is on two kind of template yamls as below. [Collector templates](#Collector-templates) and [Object templates](#Object-templates)
+
+## Collector templates
+
+Example of collector template `conf/zapi/default.yaml`
+By default Harvest reads from `conf/zapi/default.yaml` (shipped with harvest) and `conf/zapi/custom.yaml` (used to extend `conf/zapi/default.yaml`).
+We can define the configuration file of the collector. If no configuration file is specified, the default configuration file (`conf/zapiperf/default.yaml`) will be used and if the file `conf/zapiperf/custom.yaml` is present, it will be merged to the default one. If we specify our own configuration file for the collector, it can have any name, and it will not be merged.
+
+Examples:
+1. Define a poller that will run the ZapiPerf collector using its default configuration file:
+
+```yaml
+Pollers:
+ jamaica: # name of the poller
+ datacenter: munich
+ addr: 10.10.10.10
+ auth_style: basic_auth
+ username: harvest
+ password: pass
+ collectors:
+ - Zapi # will use conf/zapi/default.yaml and optionally merge with conf/zapi/custom.yaml
+```
+
+2. Define a poller that will run the Zapi collector using a custom configuration file:
+
+```yaml
+Pollers:
+ jamaica: # name of the poller
+ addr: 10.10.10.10
+ auth_style: basic_auth
+ username: harvest
+ password: pass
+ collectors:
+ - ZapiPerf:
+ - limited.yaml # will use conf/zapi/limited.yaml
+ # if more templates are added, they will be merged
+```
+
+### Object Templates
+Object templates contains counter details of an object. Example object template `conf/zapi/cdot/9.8.0/lun.yaml`
+
+Instead of editing one of the existing templates, it's better to extend existing templates. That way, your custom template will not be overwritten when upgrading Harvest. For example, if you want to extend `conf/zapi/cdot/9.8.0/aggr.yaml`, first create a copy (e.g., `conf/zapi/cdot/9.8.0/custom_aggr.yaml`), then add these lines to `conf/zapi/custom.yaml`:
+
+```yaml
+objects:
+ Aggregate: custom_aggr.yaml
+```
+
+After restarting your pollers, `aggr.yaml` and `custom_aggr.yaml` will be merged.
+
+#### 1. Create a new objectTemplate
+
+In this example, we want to collect sensor metrics from the `environment-sensors-get-iter` API. These are the steps that we need to follow:
+
+Create the file `conf/zapi/cdot/9.8.0/sensor.yaml` (optionally replace `9.8.0` with the version of your ONTAP). Add following content:
+
+```yaml
+name: Sensor
+query: environment-sensors-get-iter
+object: sensor
+
+counters:
+ environment-sensors-info:
+ - critical-high-threshold => critical_high
+ - critical-low-threshold => critical_low
+ - ^discrete-sensor-state => discrete_state
+ - ^discrete-sensor-value => discrete_value
+ - ^^node-name => node
+ - ^^sensor-name => sensor
+ - ^sensor-type => type
+ - ^threshold-sensor-state => threshold_state
+ - threshold-sensor-value => threshold_value
+ - ^value-units => unit
+ - ^warning-high-threshold => warning_high
+ - ^warning-low-threshold => warning_low
+
+export_options:
+ include_all_labels: true
+```
+
+#### 2. Enable the new objectTemplate
+
+To enable the new objectTemplate, create `conf/zapi/custom.yaml` with the lines shown below.
+
+```yaml
+objects:
+ Sensor: sensor.yaml
+```
+In the future, if you add more objectTemplates, you can add those in this same file.
+
+#### 3. Extend an existing objectTemplate
+
+In this example, we want to extend existing default objectTemplate `conf/zapi/cdot/9.8.0/lun.yaml` and collect additional metrics from the `lun-get-iter` zapi. These are the steps that one need to follow:
+
+Existing default `conf/zapi/cdot/9.8.0/lun.yaml` as below
+
+```yaml
+name: Lun
+query: lun-get-iter
+object: lun
+
+counters:
+ lun-info:
+ - ^node
+ - ^path
+ - ^qtree
+ - size
+ - size-used
+ - ^state
+ - ^^uuid
+ - ^volume
+ - ^vserver => svm
+
+plugins:
+ - LabelAgent:
+ value_mapping:
+ - status state online `1`
+ # metric label zapi_value rest_value `default_value`
+ value_to_num:
+ - new_status state online online `0`
+ # path is something like "/vol/vol_georg_fcp401/lun401"
+ # we only want lun name, which is 4th element
+ split:
+ - path `/` ,,,lun
+
+export_options:
+ instance_keys:
+ - node
+ - qtree
+ - lun
+ - volume
+ - svm
+ instance_labels:
+ - state
+ ```
+
+To extend this objectTemplate, create `conf/zapi/custom.yaml` with the lines shown below.
+
+```yaml
+objects:
+ Sensor: custom_lun.yaml
+```
+
+Create a new objectTemplate `conf/zapi/cdot/9.8.0/custom_lun.yaml` with the lines shown below.
+
+Use Case:
+1. Add additional counters `multiprotocol-type`, `application`
+2. Configure a new `value_to_num` plugin
+3. Add `application` to instance_keys
+4. Overwrite object name
+
+```yaml
+name: CustomLun
+counters:
+ lun-info:
+ - ^multiprotocol-type
+ - ^application
+
+plugins:
+ - LabelAgent:
+ value_to_num:
+ - new_status state online online `0`
+
+export_options:
+ instance_keys:
+ - application
+ ```
+
+Harvest will merge these templates during poller start and merged template will be as below
+
+```yaml
+name: CustomLun
+query: lun-get-iter
+object: lun
+counters:
+ lun-info:
+ - ^node
+ - ^path
+ - ^qtree
+ - size
+ - size-used
+ - ^state
+ - ^^uuid
+ - ^volume
+ - ^vserver => svm
+ - ^multiprotocol-type
+ - ^application
+plugins:
+ LabelAgent:
+ value_mapping:
+ - status state online `1`
+ value_to_num:
+ - new_status state online online `0`
+ split:
+ - path `/` ,,,lun
+export_options:
+ instance_keys:
+ - node
+ - qtree
+ - lun
+ - volume
+ - svm
+ - application
+```
+
+To view the merged template, you can also run below command to verify the template.
+```sh
+bin/harvest doctor merge --template lun.yaml --with custom_lun.yaml
+```
+
+#### 4. Test your changes and restart pollers
+
+Test your new `Sensor` template with a single poller like this:
+```
+./bin/harvest start <poller> --foreground --verbose --collectors Zapi --objects Sensor
+```
+Replace `<poller>` with the name of one of your ONTAP pollers.
+
+Once you have confirmed that the new template works, restart any already running pollers that you want to pick up the new template(s).
+
+### Check the metrics
+
+If you are using the Prometheus exporter, check the metrics on the HTTP endpoint with `curl` or a web browser. E.g., my poller is exporting its data on port `15001`. Adjust as needed for your exporter.
+
+```
+curl -s 'http://localhost:15001/metrics' | grep sensor_
+
+sensor_value{datacenter="WDRF",cluster="shopfloor",critical_high="3664",node="shopfloor-02",sensor="P3.3V STBY",type="voltage",warning_low="3040",critical_low="2960",threshold_state="normal",unit="mV",warning_high="3568"} 3280
+sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="P1.2V STBY",type="voltage",threshold_state="normal",warning_high="1299",warning_low="1105",critical_low="1086",node="shopfloor-02",critical_high="1319",unit="mV"} 1193
+sensor_value{datacenter="WDRF",cluster="shopfloor",unit="mV",critical_high="15810",critical_low="0",node="shopfloor-02",sensor="P12V STBY",type="voltage",threshold_state="normal"} 11842
+sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="P12V STBY Curr",type="current",threshold_state="normal",unit="mA",critical_high="3182",critical_low="0",node="shopfloor-02"} 748
+sensor_value{datacenter="WDRF",cluster="shopfloor",critical_low="1470",node="shopfloor-02",sensor="Sysfan2 F2 Speed",type="fan",threshold_state="normal",unit="RPM",warning_low="1560"} 2820
+sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="PSU2 Fan1 Speed",type="fan",threshold_state="normal",unit="RPM",warning_low="4600",critical_low="4500",node="shopfloor-01"} 6900
+sensor_value{datacenter="WDRF",cluster="shopfloor",sensor="PSU1 InPwr Monitor",type="unknown",threshold_state="normal",unit="mW",node="shopfloor-01"} 132000
+sensor_value{datacenter="WDRF",cluster="shopfloor",critical_high="58",type="thermal",unit="C",warning_high="53",critical_low="0",node="shopfloor-01",sensor="Bat Temp",threshold_state="normal",warning_low="5"} 24
+sensor_value{datacenter="WDRF",cluster="shopfloor",critical_high="9000",node="shopfloor-01",sensor="Bat Charge Volt",type="voltage",threshold_state="normal",unit="mV",warning_high="8900"} 8200
+sensor_value{datacenter="WDRF",cluster="shopfloor",node="shopfloor-02",sensor="PSU1 InPwr Monitor",type="unknown",threshold_state="normal",unit="mW"} 132000
+```
\ No newline at end of file