Contemplate how to handle words with non-Chinese characters

[danielat998]

No description provided.

[james-s-w-clark]

Could try a rule:
If (string unicode isn't han) {
Try looking up substring+next character(s) (which may be Chinese)
}

[danielat998]

@james-clark-5 A valid idea. But we will have to consider where in the code this is done, as I have a suspicion we may extract non-Chinese characters earlier in the process. Also, there exist words that start with non-Chinese characters

[james-s-w-clark]

What examples do we have? Should help get the ball rolling.
AA制 (to go 50/50)
.......

[danielat998]

That's the example that came to mind too, though if you look at the beginning of the file, you will find more examples. I think the first step would be to find out if there are any words that *end* with non Hanzi characters, as this might make the implementation a little (though not much) more complicated.

…

On 22 September 2017 at 04:16, IdiosApps ***@***.***> wrote: What examples do we have? Should help get the ball rolling. AA制 (to go 50/50) ....... — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOmv5AFAcwBBAMiCPmqlYkZEyYDul118ks5skyargaJpZM4OBl4j> .

[james-s-w-clark]

I guess we'll be able to find about potential words ending with non-hanzi characters by reusing the recursive "try-all-character-combinations" code. Would also bring up words starting with non-hanzi characters.

[danielat998]

Finding them would be very easy, something like: for Word word in (the set of words){ for(char c in word.toCharArray()){ if (c is not Unicode.HAN){ System.out.println(word.getSimplifiedChinese); continue; } } }

…

On 22 September 2017 at 10:18, IdiosApps ***@***.***> wrote: I guess we'll be able to find about potential words ending with non-hanzi characters by reusing the recursive "try-all-character-combinations" code. Would also bring up words starting with non-hanzi characters. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AOmv5Ly3HkHC2otRLwUrafDGc86EOIXJks5sk3t9gaJpZM4OBl4j> .

[danielt998]

less cedict_ts.u8 | cut -d ' ' -f 2 |grep '[A-Z]|[A-Z]'
edit:less cedict_ts.u8 | cut -d ' ' -f 2 |grep '[A-Z]|[a-z]'
3C 3P 3Q A AA制 AB制 ACG A咖 A圈儿 A片 A菜 A货 B B型超声 B超 CP C盘 C罗 DNA鉴定 E仔 G友 G弦裤 G点 H桥 K人 K仔 K他命 K房 K书 K歌 K粉 K线 K线图 M巾 N挡 OK绷 OK镜 OS O型腿 P P图 P挡 P民 Q T TA T字帐 T字裤 T恤 T裇 USB手指 USB记忆棒 U凸内裤 U型枕 U形转弯 U盘 V沟 X光 三C 三K党 三P 事儿B 来M 傻B 傻X 动L 卡拉OK 哆啦A梦 唱K 大V 巴比Q 拉K 柯P 牛B 异维A酸 装B 阿Q 阿Q正传 齐B小短裙 齐B短裙

[james-s-w-clark]

I'm curious what this would pull up:
less cedict_ts.u8 | cut -d ' ' -f 2 |grep '[a-z]|[a-z]'

[james-s-w-clark]

I'm curious what this would pull up:
less cedict_ts.u8 | cut -d ' ' -f 2 |grep '[0-9]|[0-9]'

[danielt998]

圈a
and

21三体综合症
3C
3P
3Q
502胶
美国51区

and bear in mind there are entries with other characters too, such as this:
□ □ [ging1] /uptight/obstinate/to awkwardly force oneself to do sth/(Taiwanese, POJ pr. [gēng], often written as ㄍㄧㄥ, no generally accepted hanzi form)/