Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check license for CCR before attempting to fetch stats #9003

Merged
merged 8 commits into from
Nov 15, 2018
Merged

Check license for CCR before attempting to fetch stats #9003

merged 8 commits into from
Nov 15, 2018

Conversation

ycombinator
Copy link
Contributor

Resolves #8915.

This PR teaches the Fetch() method of the elasticsearch/ccr metricset to first check if the CCR feature is available per the current Elasticsearch license. If it isn't, the metricset logs+reports an actionable error message every minute.

Before this PR, there was no such check so the call to the CCR stats API would simply fail with a 403 error from Elasticsearch if Elasticsearch wasn't using a Trial or Platinum license. That unhelpful 403 error would get logged+reported every 10s (or whatever period the metricset was configured to).

Before this PR

In the metricbeat logs:

screen shot 2018-11-08 at 4 42 28 pm

In the metricbeat-* index:

screen shot 2018-11-08 at 4 42 38 pm

After this PR

In the metricbeat logs:

screen shot 2018-11-08 at 4 35 19 pm

In the metricbeat-* index:

screen shot 2018-11-08 at 4 37 27 pm

@@ -391,3 +391,24 @@ func getSettingGroup(allSettings common.MapStr, groupKey string) (common.MapStr,

return common.MapStr(v), nil
}

// IsCCRAvailable returns whether CCR is available or not, depending on the current license
func IsCCRAvailable(http *helper.HTTP, resetURI string) (isAvailable bool, currentLicense string, err error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be confusing to understand the difference with IsCCRStatsAvailable. We could make this a generic license checker, and/or we could have an only method to check if CCR stats are available in current license AND version.

Copy link
Contributor Author

@ycombinator ycombinator Nov 9, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like both your ideas, actually. The version check should probably log+report at the same frequency as the license check. So how about I do this:

  • Make a generic version check function in elasticsearch.go. In fact, I think there is one already.
  • Make a generic license check function in elasticsearch.go.
  • Move isCCRAvailable into ccr.go, since it's really only needed by that metricset. This function would check the license and the version, returning a non-empty log message, if the check fails.

WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactoring done in 49c422b.

@ycombinator ycombinator added in progress Pull request is currently in progress. and removed review labels Nov 10, 2018
@ycombinator ycombinator removed the request for review from ruflin November 10, 2018 12:04
@ycombinator
Copy link
Contributor Author

Things aren't quite working correctly after my latest changes so I'm moving this PR out of review for now.

@ycombinator ycombinator requested a review from ruflin November 13, 2018 14:41
@ycombinator ycombinator added review and removed in progress Pull request is currently in progress. labels Nov 13, 2018
@ycombinator
Copy link
Contributor Author

I've now fixed up a couple of minor issues I introduced in the refactoring done in 49c422b. @jsoriano @ruflin this PR is ready for review again. Thanks!

@ycombinator
Copy link
Contributor Author

@jsoriano @ruflin Addressed review feedback. Ready for your 👀 again. Thanks!

metricbeat/module/elasticsearch/ccr/ccr.go Outdated Show resolved Hide resolved
@ycombinator ycombinator merged commit 4e6c1cf into elastic:master Nov 15, 2018
@ycombinator ycombinator deleted the metricbeat-es-ccr-xpack branch November 15, 2018 13:29
ycombinator added a commit that referenced this pull request Nov 15, 2018
…fetch stats (#9095)

Cherry-pick of PR #9003 to 6.x branch. Original message: 

Resolves #8915.

This PR teaches the `Fetch()` method of the `elasticsearch/ccr` metricset to first check if the CCR feature is available per the current Elasticsearch license. If it isn't, the metricset logs+reports an **actionable** error message **every minute**.

Before this PR, there was no such check so the call to the CCR stats API would simply fail with a 403 error from Elasticsearch if Elasticsearch wasn't using a Trial or Platinum license. That unhelpful 403 error would get logged+reported every 10s (or whatever `period` the metricset was configured to).

## Before this PR

### In the `metricbeat` logs:

<img width="839" alt="screen shot 2018-11-08 at 4 42 28 pm" src="https://user-images.githubusercontent.com/51061/48236039-5fa30c80-e375-11e8-8d9d-64c4861fb35d.png">


### In the `metricbeat-*` index:

<img width="1491" alt="screen shot 2018-11-08 at 4 42 38 pm" src="https://user-images.githubusercontent.com/51061/48236036-5b76ef00-e375-11e8-9a21-7cde7620c80b.png">


## After this PR

### In the `metricbeat` logs:

<img width="1236" alt="screen shot 2018-11-08 at 4 35 19 pm" src="https://user-images.githubusercontent.com/51061/48235932-e4415b00-e374-11e8-8b39-a60e4fe87279.png">

### In the `metricbeat-*` index:

<img width="1495" alt="screen shot 2018-11-08 at 4 37 27 pm" src="https://user-images.githubusercontent.com/51061/48235909-d095f480-e374-11e8-9eea-718c0deacafa.png">
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants