Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MutualInformationTests.testBasicScoreProperties fails reproducibly #75561

Closed
pgomulka opened this issue Jul 21, 2021 · 3 comments
Closed

MutualInformationTests.testBasicScoreProperties fails reproducibly #75561

pgomulka opened this issue Jul 21, 2021 · 3 comments
Assignees
@benwtrent
Labels
:Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test-failure Triaged test failures from CI

Comments

@pgomulka
Copy link
Contributor

Build scan:
https://gradle-enterprise.elastic.co/s/5aulfo3ieqjn6

Repro line:
./gradlew ':server:test' --tests "org.elasticsearch.search.aggregations.bucket.terms.heuristic.MutualInformationTests.testBasicScoreProperties" -Dtests.seed=2036BFB6F863ADFE -Dtests.locale=fr-CA -Dtests.timezone=Australia/Darwin -Druntime.java=16 -Dtests.fips.enabled=true
fails for that seed only

Reproduces locally?:
yes
Applicable branches:
7.x
Failure history:

Failure excerpt:

06:39:44 org.elasticsearch.search.aggregations.bucket.terms.heuristic.MutualInformationTests > testBasicScoreProperties FAILED
06:39:44     java.lang.AssertionError: 
06:39:44     Expected: a value equal to or greater than <0.0>
06:39:44          but: <-0.46758255981716007> was less than <0.0>
06:39:44         at __randomizedtesting.SeedInfo.seed([2036BFB6F863ADFE:904D11FBE699F9BE]:0)
06:39:44         at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
06:39:44         at org.junit.Assert.assertThat(Assert.java:956)
06:39:44         at org.junit.Assert.assertThat(Assert.java:923)
06:39:44         at org.elasticsearch.search.aggregations.bucket.terms.AbstractSignificanceHeuristicTests.testBasicScoreProperties(AbstractSignificanceHeuristicTests.java:150)

fails since 15th july
https://build-stats.elastic.co/app/kibana#/discover?_g=(refreshInterval:(pause:!t,value:0),time:(from:now-30d,mode:quick,to:now))&_a=(columns:!(process.status),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:b646ed00-7efc-11e8-bf69-63c8ef516157,key:process.status,negate:!f,params:(query:FAILURE,type:phrase),type:phrase,value:FAILURE),query:(match:(process.status:(query:FAILURE,type:phrase))))),index:b646ed00-7efc-11e8-bf69-63c8ef516157,interval:auto,query:(language:lucene,query:'MutualInformationTests%20testBasicScoreProperties'),sort:!(process.time-start,desc))
@pgomulka pgomulka added >test-failure Triaged test failures from CI :Analytics/Aggregations Aggregations labels Jul 21, 2021
@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jul 21, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@pgomulka
Copy link
Contributor Author

more failures https://gradle-enterprise.elastic.co/s/offdtwuj6rhwo
also failing for ChiSquareTests
the bug might be in AbstractSignificanceHeuristicTests.testBasicScoreProperties

@benwtrent
Copy link
Member

This will be fixed with: #75452

benwtrent added a commit that referenced this issue Jul 21, 2021
@benwtrent
Significant terms test refactor for extendability (#75452)
9b88db7 
The original PR #75264 made some test mistakes

NXY Significant term heuristics have additional values that need to be set when testing
basicScore properties.

Additionally, previous refactor kept the abstract test class in a package that other plugins
don't have access to.

closes #75442, #75561
elasticsearchmachine pushed a commit that referenced this issue Jul 21, 2021
@benwtrent
Significant terms test refactor for extendability (#75452) (#75593)
890c665 
The original PR #75264 made some test mistakes

NXY Significant term heuristics have additional values that need to be set when testing
basicScore properties.

Additionally, previous refactor kept the abstract test class in a package that other plugins
don't have access to.

closes #75442, #75561
ywangd pushed a commit to ywangd/elasticsearch that referenced this issue Jul 30, 2021
@benwtrent @ywangd
Significant terms test refactor for extendability (elastic#75452)
d6fb1c1 
The original PR elastic#75264 made some test mistakes

NXY Significant term heuristics have additional values that need to be set when testing
basicScore properties.

Additionally, previous refactor kept the abstract test class in a package that other plugins
don't have access to.

closes elastic#75442, elastic#75561
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@benwtrent benwtrent

Labels
:Analytics/Aggregations Aggregations Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@benwtrent @pgomulka @elasticmachine