Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: fetch constituents when asked from template #2838

Merged
merged 1 commit into from
Apr 17, 2024
Merged

refactor: fetch constituents when asked from template #2838

merged 1 commit into from
Apr 17, 2024

Conversation

Hardikl
Copy link
Contributor

@Hardikl Hardikl commented Apr 17, 2024

Prior the changes:

harvest % ./bin/poller -p sar --promPort 18002  -o Volume -c Rest
2024-04-16T17:37:02+05:30 INF poller/poller.go:221 > Init Poller=sar asup=false confPath=conf config=harvest.yml configPath=harvest.yml cwd=/Users/hardikl/Documents/Harvest/harvest daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=18002 version="harvest version 24.04.16-1 (commit 88070726) (build date 2024-04-16T17:36:37+0530) darwin/amd64"
2024-04-16T17:37:02+05:30 INF poller/poller.go:245 > started in foreground Poller=sar pid=39528
2024-04-16T17:37:03+05:30 INF collector/helpers.go:88 > best-fit template Poller=sar collector=Rest:Volume path=conf/rest/9.12.0/volume.yaml v=9.13.1
2024-04-16T17:37:04+05:30 INF poller/poller.go:363 > Autosupport scheduled. Poller=sar asupSchedule=24h
2024-04-16T17:37:04+05:30 INF poller/poller.go:372 > poller start-up complete Poller=sar
2024-04-16T17:37:04+05:30 INF prometheus/httpd.go:40 > server listen Poller=sar exporter=prometheus1 url=http://:18002/metrics
2024-04-16T17:37:04+05:30 INF poller/poller.go:518 > updated status, up collectors: 1 (of 1), up exporters: 1 (of 1) Poller=sar
2024-04-16T17:37:04+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=337 bytesRx=0 collector=Rest:Volume metrics=0 numCalls=0 pollMs=337 task=counter zBegin=1713269224322
2024-04-16T17:37:10+05:30 INF volume/volume.go:277 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T17:37:11+05:30 INF volume/volume.go:277 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=true Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T17:37:11+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=4478 bytesRx=605997 calcMs=0 collector=Rest:Volume exportMs=14 instances=258 instancesExported=213 metrics=12105 metricsExported=8446 numCalls=7 parseMs=2938 pluginMs=2427 pollMs=6928 zBegin=1713269224659
2024-04-16T17:40:11+05:30 INF volume/volume.go:277 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T17:40:12+05:30 INF volume/volume.go:277 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=true Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T17:40:13+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=6331 bytesRx=602162 calcMs=0 collector=Rest:Volume exportMs=13 instances=258 instancesExported=213 metrics=12105 metricsExported=8446 numCalls=6 parseMs=3379 pluginMs=2002 pollMs=8351 zBegin=1713269404662
2024-04-16T17:43:11+05:30 INF volume/volume.go:277 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T17:43:12+05:30 INF volume/volume.go:277 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=true Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T17:43:13+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=6586 bytesRx=602164 calcMs=0 collector=Rest:Volume exportMs=15 instances=258 instancesExported=213 metrics=12105 metricsExported=8446 numCalls=6 parseMs=2949 pluginMs=2061 pollMs=8661 zBegin=1713269584666

With changes:
when include_constituents: false in rest template

harvest % ./bin/poller -p sar --promPort 18002  -o Volume -c Rest
2024-04-16T18:25:10+05:30 INF poller/poller.go:221 > Init Poller=sar asup=false confPath=conf config=harvest.yml configPath=harvest.yml cwd=/Users/hardikl/Documents/Harvest/harvest daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=18002 version="harvest version 24.04.16-1 (commit 88070726) (build date 2024-04-16T18:25:03+0530) darwin/amd64"
2024-04-16T18:25:10+05:30 INF poller/poller.go:245 > started in foreground Poller=sar pid=43822
2024-04-16T18:25:11+05:30 INF collector/helpers.go:88 > best-fit template Poller=sar collector=Rest:Volume path=conf/rest/9.12.0/volume.yaml v=9.13.1
2024-04-16T18:25:12+05:30 INF poller/poller.go:363 > Autosupport scheduled. Poller=sar asupSchedule=24h
2024-04-16T18:25:12+05:30 INF poller/poller.go:372 > poller start-up complete Poller=sar
2024-04-16T18:25:12+05:30 INF prometheus/httpd.go:40 > server listen Poller=sar exporter=prometheus1 url=http://:18002/metrics
2024-04-16T18:25:12+05:30 INF poller/poller.go:518 > updated status, up collectors: 1 (of 1), up exporters: 1 (of 1) Poller=sar
2024-04-16T18:25:12+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=348 bytesRx=0 collector=Rest:Volume metrics=0 numCalls=0 pollMs=349 task=counter zBegin=1713272112093
2024-04-16T18:25:18+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:25:20+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=5518 bytesRx=596008 calcMs=0 collector=Rest:Volume exportMs=14 instances=258 instancesExported=213 metrics=12105 metricsExported=8446 numCalls=6 parseMs=2945 pluginMs=2092 pollMs=7635 zBegin=1713272112441
2024-04-16T18:28:18+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:28:20+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=6137 bytesRx=592177 calcMs=0 collector=Rest:Volume exportMs=20 instances=258 instancesExported=213 metrics=12105 metricsExported=8446 numCalls=5 parseMs=3039 pluginMs=1681 pollMs=7849 zBegin=1713272292443

when include_constituents: true in rest template

harvest % ./bin/poller -p sar --promPort 18002  -o Volume -c Rest
2024-04-16T18:31:33+05:30 INF poller/poller.go:221 > Init Poller=sar asup=false confPath=conf config=harvest.yml configPath=harvest.yml cwd=/Users/hardikl/Documents/Harvest/harvest daemon=false debug=false homePath= hostname=hardikl-mac-0 logLevel=info logPath=/var/log/harvest/ profiling=0 promPort=18002 version="harvest version 24.04.16-1 (commit 88070726) (build date 2024-04-16T18:25:03+0530) darwin/amd64"
2024-04-16T18:31:33+05:30 INF poller/poller.go:245 > started in foreground Poller=sar pid=45491
2024-04-16T18:31:35+05:30 INF collector/helpers.go:88 > best-fit template Poller=sar collector=Rest:Volume path=conf/rest/9.12.0/volume.yaml v=9.13.1
2024-04-16T18:31:35+05:30 INF poller/poller.go:363 > Autosupport scheduled. Poller=sar asupSchedule=24h
2024-04-16T18:31:35+05:30 INF prometheus/httpd.go:40 > server listen Poller=sar exporter=prometheus1 url=http://:18002/metrics
2024-04-16T18:31:35+05:30 INF poller/poller.go:372 > poller start-up complete Poller=sar
2024-04-16T18:31:35+05:30 INF poller/poller.go:518 > updated status, up collectors: 1 (of 1), up exporters: 1 (of 1) Poller=sar
2024-04-16T18:31:36+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=356 bytesRx=0 collector=Rest:Volume metrics=0 numCalls=0 pollMs=357 task=counter zBegin=1713272495887
2024-04-16T18:31:42+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:31:43+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=true Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:31:43+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=5361 bytesRx=605988 calcMs=0 collector=Rest:Volume exportMs=23 instances=258 instancesExported=259 metrics=12105 metricsExported=10292 numCalls=7 parseMs=2845 pluginMs=2309 pollMs=7700 zBegin=1713272496244
2024-04-16T18:34:42+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:34:44+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=true Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:34:44+05:30 INF collector/collector.go:595 > Collected Poller=sar apiMs=6170 bytesRx=602157 calcMs=0 collector=Rest:Volume exportMs=21 instances=258 instancesExported=259 metrics=12105 metricsExported=10292 numCalls=6 parseMs=3173 pluginMs=2073 pollMs=8267 zBegin=1713272676246
2024-04-16T18:37:42+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=false Poller=sar object=Volume plugin=Rest:Volume
2024-04-16T18:37:44+05:30 INF volume/volume.go:280 > api/storage/volumes?return_records=true&fields=name,svm.name,clone.parent_snapshot.name,clone.split_estimate,anti_ransomware.dry_run_start_time,anti_ransomware.state&is_constituent=true Poller=sar object=Volume plugin=Rest:Volume

@cla-bot cla-bot bot added the cla-signed label Apr 17, 2024
@Hardikl Hardikl linked an issue Apr 17, 2024 that may be closed by this pull request
Improve multiple Rest calls in Volume Rest plugin #2837
Closed
@cgrinds cgrinds merged commit f0d29ae into main Apr 17, 2024
6 checks passed
@cgrinds cgrinds deleted the hl_vol_call branch April 17, 2024 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rahulguptajss rahulguptajss rahulguptajss approved these changes

@cgrinds cgrinds Awaiting requested review from cgrinds cgrinds is a code owner

Assignees
No one assigned
Labels
cla-signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Improve multiple Rest calls in Volume Rest plugin
3 participants
@Hardikl @rahulguptajss @cgrinds