Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Lucene 8 upgrade checklist #32899

Closed
17 of 18 tasks
jimczi opened this issue Aug 16, 2018 · 1 comment · Fixed by #33310
Closed
17 of 18 tasks

Lucene 8 upgrade checklist #32899

jimczi opened this issue Aug 16, 2018 · 1 comment · Fixed by #33310
Labels
>non-issue :Search/Search Search-related issues that do not fall into other categories >upgrade

Comments

@jimczi
Copy link
Contributor

jimczi commented Aug 16, 2018 •
edited
Loading

This is a meta issue to track the progress of upgrading master to Lucene 8. The branch can be found here. Below is the list of things to be changed/removed for the upgrade:

  • Levenstein distance renaming.
  • TermContext renaming (TermStates).
  • Collectors should implement scoreMode instead of needsScore. (@jpountz)
  • IndexSearcher#createNormalizedWeight has been removed. (@jpountz)
  • StandardFilter removal. (@jimczi)
  • EarlyTerminatingSortingCollector removal. (@jpountz)
  • BoostingQuery removal (replaced with FunctionScoreQuery#boostByQuery). (@jimczi)
  • Similarity#SimWeight removal. (@jpountz)
  • BasicModel removals (after effects, BE, D and P). (@jpountz)
  • Version#LUCENE_6* removal. (@jimczi Remove unsupported Version.V_5_* #32937)
  • TopDocs#totalHits rewriting.
  • TopDocs#computeMaxScore replacement
  • Explanation#getValue() is a Number instead of a float

Test failures

  • PercolateQueryBuilder#createMultiDocumentSearcher, PercolatorQuerySearchIT: @jpountz
  • FunctionScoreTests#testSimpleWeightedFunction: produces negative scores (Forbid negative values for "weight" in Function Score Query #31927)
  • ScriptedSimilarityTests#testBasics, ScriptedSimilarityTests#testInit: score is called on non-scoring clauses (to get the max score, see ImpactsDISI)
  • FieldSubsetReaderTests#testFieldNames, FieldSubsetReaderTests#testFieldNamesMissing, FieldSubsetReaderTests#testFieldThreeFields: FieldNamesTerms return invalid statistics (e.g. -1) Security for _field_names field should not override field statistics #33261
  • TransportTwoNodesSearchIT#testDfsQueryThenFetch, TransportTwoNodesSearchIT#testDfsQueryThenFetchWithSort: new explanation format and message

\cc @elastic/es-distributed to sync up.
@colings86 colings86 added the :Search/Search Search-related issues that do not fall into other categories label Aug 16, 2018
@jasontedor
Copy link
Member

Pinging @dnhatn

@jimczi jimczi mentioned this issue Aug 31, 2018
Merged
jimczi added a commit that referenced this issue Sep 6, 2018
@jimczi
Upgrade to a Lucene 8 snapshot (#33310)
7ad71f9 
The main benefit of the upgrade for users is the search optimization for top scored documents when the total hit count is not needed. However this optimization is not activated in this change, there is another issue opened to discuss how it should be integrated smoothly.
Some comments about the change:
* Tests that can produce negative scores have been adapted but we need to forbid them completely: #33309

Closes #32899
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>non-issue :Search/Search Search-related issues that do not fall into other categories >upgrade
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Upgrade to a Lucene 8 snapshot elastic/elasticsearch
3 participants
@colings86 @jasontedor @jimczi