Full text search results list includes entries without matching keyword

[prismriverrr]

Sometimes, when searching for certain text, there are no matches under a specific entry, but results list still appears. As shown in the image, the dictionary being searched is Contemporary Chinese Dictionary.

Version: 25.04.0.7f9a23f at 2025-02-19T06:18:38Z
Qt 6.7.2 MSVC 194234436
windows winnt 10.0.26100 x86_64-little_endian-llp64
Flags: ZIM EPWING OPENCC

[prismriverrr]

I just found that if the keyword is enclosed in quotation marks, this issue won't occur. So my question is, how does program perform matching without using quotation marks?

[prismriverrr]

for comparison：

[shenlebantongying]

AFAIK, wthout quotes, a CJK sentence will be segmented into 2-grams (n-gram, bigram, 2 characters as a unit).

I only find this book's section 3.5 has a simple description https://plg.uwaterloo.ca/~ir/ir/book/03-tokens-and-terms.pdf

On xapian's side, the thing is FLAG_CJK_NGRAM https://xapian.org/docs/apidoc/html/classXapian_1_1QueryParser.html

[prismriverrr]

AFAIK, wthout quotes, a CJK sentence will be segmented into 2-grams (n-gram, bigram, 2 characters as a unit).

So in this case, is 清代文 equivalent to 清代 AND 文? But results are different.

[shenlebantongying]

I am not certain, but I think it is 清代 AND 代文

[prismriverrr]

I am not certain, but I think it is 清代 AND 代文

That makes sense, but currently there seem to be some issues with text highlighting and match counting?

[xiaoyifang]

the number only count the first entry

[prismriverrr]

the number only count the first entry

Will it be changed to count all entries in the future? I think this is more reasonable, as it allows for better positioning of all matches using the previous and next buttons.

[xiaoyifang]

have no simple way to fix this.

[prismriverrr]

Sometimes headword cannot be found, but definition can.

Chinese-English Wiktionary dictionary stardict.tar.gz