Skip to content

Commit

Permalink
MB-31358: Estimation of memory needed for query with highlighting
Browse files Browse the repository at this point in the history
+ Check this test ..
```
func TestEstimate(t *testing.T) {
    q := []byte(`{"from":8996,"highlight":{},"query":{"bool":false,"field":"free_parking"},"size":405}`)
    var sr *SearchRequest
    err := json.Unmarshal(q, &sr)
    if err != nil {
        t.Fatal(err)
    }

    x := MemoryNeededForSearchResult(sr)
    fmt.Println(x)
}
```

+ For the query, the MemoryNeededForSearchResult API claims that it
  needs .. 6363433784 bytes (that's over 6GB!).
+ Fixing the size calculation estimate when the query requests for
  "fields" to be returned or "highlighting", basing the number of
  results returned on the collector's PreAllocSizeSkipCap.
  • Loading branch information
abhinavdangeti committed Feb 5, 2021
1 parent 2522dc2 commit 3468f89
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions search.go
Original file line number Diff line number Diff line change
Expand Up @@ -600,11 +600,11 @@ func MemoryNeededForSearchResult(req *SearchRequest) uint64 {
estimate += len(req.Facets) * fr.Size()
}

// highlighting, store
// overhead from highlighting, store
var d document.Document
if len(req.Fields) > 0 || req.Highlight != nil {
for i := 0; i < (req.Size + req.From); i++ {
estimate += (req.Size + req.From) * d.Size()
for i := 0; i < numDocMatches; i++ {
estimate += numDocMatches * d.Size()
}
}

Expand Down

0 comments on commit 3468f89

Please sign in to comment.