Bug: Win11 Emoji Picker does not accept most kb inputs when kanata is running and the emoji search/filter input is focused

[riotrah]

Describe the bug

Using my capslock/hyper-esque config with Kanata via the hook, invoking the Win11 emoji picker by any means (eg Win+;) results in a mouse-accessible picker appearing, but one whose normal keyboard-based operations stop functioning if the focus is in the search input. Anything done to move the focus out of the search input onto any other UI element restores normal behavior until the search is focused again.

Said input is the first thing focused upon appearance, and is always refocused when non-editing/navigational shortcuts (arrows, enter, escape, etc) are entered. This can make it appear as tho the kb inputs generally do not work, and not due to focus behavior. I actually realized this as I tried to add an ironic emoji to this bug report ha.

Version
1.0.7

Relevant kanata configs

config

;; Comments are prefixed by double-semicolon. A single semicolon is parsed as the
;; keyboard key. Comments are ignored for the configuration file.
;;
;; This configuration language is Lisp-like. If you're unfamiliar with Lisp,
;; don't be alarmed. The maintainer jtroo is also unfamiliar with Lisp. You
;; don't need to know Lisp in-depth to be able to configure kanata.
;;
;; If you follow along with the examples, you should be fine. Kanata should
;; also hopefully have helpful error messages in case something goes wrong.
;; If you need help, you are welcome to ask.

(defcfg
  ;; If you are using Windows, you do not need the linux-dev line below, but
  ;; you still need a defcfg entry. You can leave the linux-dev entry and it
  ;; will be ignored or you can delete it if desired.
  ;;
  ;; Your keyboard device may differ from this. I believe /dev/input/by-id/
  ;; is preferable; I recall reading that it's less likely to change names on you,
  ;; but I didn't find any keyboard device in there in my VM.
  linux-dev /dev/input/by-path/platform-i8042-serio-0-event-kbd

  process-unmapped-keys yes
)

;; Only one defsrc is allowed.
;;
;; defsrc defines the keys that will be intercepted by kanata. The order of the
;; keys matches with deflayer declarations and all deflayer declarations must
;; have the same number of keys as defsrc. Any keys not listed in defsrc will
;; be passed straight to the operating system.
(defsrc
  esc  f1   f2   f3   f4   f5   f6   f7   f8   f9   f10  f11  f12 
  grv  1    2    3    4    5    6    7    8    9    0    -    =    bspc
  tab  q    w    e    r    t    y    u    i    o    p    [    ]    \
  caps a    s    d    f    g    h    j    k    l    ;    '    ret
  lsft z    x    c    v    b    n    m    ,    .    /    rsft
  lctl lmet lalt           spc            ralt rmet rctl
)

;; The first layer defined is the layer that will be active by default when
;; kanata starts up. This layer is the standard QWERTY layout except for the
;; backtick/grave key (@grl) which is an alias for a tap-hold key.
;;
;; There are currently a maximum of 25 layers allowed.
(deflayer qwerty
  esc  f1   f2   f3   f4   f5   f6   f7   f8   f9   f10  f11  f12 
  @grl  1    2    3    4    5    6    7    8    9    0    -    =    bspc
  tab   q    w    e    r    t    y    u    i    o    p    [    ]    \
  @caps a    s    d    f    g    h    j    k    l    ;    '    ret
  lsft  z    x    c    v    b    n    m    ,    .    /    rsft
  lctl  lmet lalt           spc            ralt rmet rctl
)

;; defalias is used to declare a shortcut for a more complicated action to keep
;; the deflayer declarations clean and aligned. The alignment in deflayers is not
;; necessary, but is strongly recommended for ease of understanding visually.
;;
;; Aliases are referred to by `@<alias_name>`.
(defalias
  ;; tap: backtick (grave), hold: toggle layer-switching layer while held
  grl (tap-hold 200 200 grv (layer-toggle layers))

  ;; layer-switch changes the base layer.
  qwr (layer-switch qwerty)

  ;; tap for capslk, hold for lctl
  caps (tap-hold-press 200 5000 esc (layer-toggle hyper))
)

;; The `lrld` action stands for "live reload". This will re-parse everything
;; except for defcfg. I.e. in Linux, you cannot live reload and switch keyboard
;; devices for the time being.
;;
;; The keys 1 and 2 switch the base layer to qwerty and dvorak respectively.
;;
;; Apart from the layer switching and live reload, all other keys are the
;; underscore _ which means "transparent". Transparent means the base layer
;; behaviour is used when pressing that key.
(deflayer layers
  lrld _    _    _    _    _    _    _    _    _    _    _    _
  _    @qwr _    _    _    _    _    _    _    _    _    _    _    _
  _    _    _    _    _    _    _    _    _    _    _    _    _    _
  _    _    _    _    _    _    _    _    _    _    _    _    _
  _    _    _    _    _    _    _    _    _    _    _    _
  _    _    _              _              _    _    _
)

(deflayer caps
  caps _    _    _    _    _    _    _    _    _    _    _    _
  _    _    _    _    _    _    _    _    _    _    _    _    _    _
  _    _    _    _    _    _    _    pgdn pgup _    home _    _    _
  _    _    _    _    _    _    lft  down up   rght _    _    _
  _    _    _    _    _    _    end  _    _    _    _    _
  _    _    _              _              _    _    _
)

(deflayer hyper
  caps        S-C-M-A-f1  S-C-M-A-f2  S-C-M-A-f3  S-C-M-A-f4  S-C-M-A-f5  S-C-M-A-f6  S-C-M-A-f7  S-C-M-A-f8  S-C-M-A-f9  S-C-M-A-f10 S-C-M-A-f11 S-C-M-A-f12 
  S-C-M-A-grv S-C-M-A-1   S-C-M-A-2   S-C-M-A-3   S-C-M-A-4   S-C-M-A-5   S-C-M-A-6   S-C-M-A-7   S-C-M-A-8   S-C-M-A-9   S-C-M-A-0   S-C-M-A--   S-C-M-A-=   S-C-M-A-bspc
  S-C-M-A-tab S-C-M-A-q   S-C-M-A-w   S-C-M-A-e   S-C-M-A-r   S-C-M-A-t   S-C-M-A-y   pgdn        pgup        S-C-M-A-o   home        S-C-M-A-[   S-C-M-A-]   S-C-M-A-\
  XX          S-C-M-A-a   S-C-M-A-s   S-C-M-A-d   S-C-M-A-f   S-C-M-A-g   lft         down        up          rght        S-C-M-A-;   S-C-M-A-'   S-C-M-A-ret
  lsft        S-C-M-A-z   S-C-M-A-x   S-C-M-A-c   S-C-M-A-v   S-C-M-A-b   end         S-C-M-A-m   S-C-M-A-,   S-C-M-A-.   S-C-M-A-/   rsft
  lctl        lmet        lalt                                ret                                 ralt        rmet        rctl
)

To reproduce

1. "Use" that config
2. "Use" llhook
3. start kanata
4. (optional?) find some input field somewhere in the os or an app or website lol and focus it
5. invoke emoji picker, any means. example is win+;
6. try:
   7. pressing enter immediately, which should insert whatever default/recently used emoji is prehighlighted
   8. using any capslock+vimkey bindings from that config (or equivalent mapping of anything => arrows), try to move around in the picker mouse-lessly
   9. pressing escape which should quit the picker
   10. pressing any other alphanumeric-ish keys, which should start searching for those chars
7. then:
   1. via mouse, clicking on any emoji, which will insert it
   2. then trying the above "assertions", minus the search one, which should all work so long as the search input is never focused again
8. then:
   4. focus the search again either arrow key navigating to it, clicking into it, or typing any characters, which will bork things again
   5. You get the idea

Expected behavior

It's all there above

Operating system

OS: Windows 11 Pro x86_64
Kernel: 10.0.22621

Additional context
Add any other context about the problem here.

[jtroo]

My findings on this issue are that on my Windows 11 machine, the emoji picker does not work at all with kanata (llhook mechanism), while it works fine on Windows 10. I think it may be because of step 8., which seems to happen automatically (search is focused by default).

I'd be curious to know if kmonad has the same issue, since both use llhook keyboard injection, and this sounds like an issue at the OS side.

[jtroo]

The issue below exists in PowerToys Keyboard Manager, which also uses SendInput and hook mechanisms to remap keys. Seems like it's a Windows issue.

microsoft/PowerToys#14328

[felipecrp]

Thank you for the feedback. I'm having the same problem with windows 11.

Apparently, the problem happens only with the search box. After selecting an emoji with the mouse, I can navigate using arrow keys, but not search for the emoji.

[felipecrp]

My findings on this issue are that on my Windows 11 machine, the emoji picker does not work at all with kanata (llhook mechanism), while it works fine on Windows 10. I think it may be because of step 8., which seems to happen automatically (search is focused by default).

I'd be curious to know if kmonad has the same issue, since both use llhook keyboard injection, and this sounds like an issue at the OS side.

I just tested, and the problem also happens using kmonad.

[felipecrp]

I was reading the related issue in the power toys project, and apparently, in their case, the keyboard still works, but not the configured mappings. Thus, I have an idea.

What do you think about trying to detect such cases where kanata can't handle the input because of the OS bug and suppress the program mapping just for those cases?

@jtroo Do you think it is feasible?

[jtroo]

I was reading the related issue in the power toys project, and apparently, in their case, the keyboard still works, but not the configured mappings. Thus, I have an idea.

What do you think about trying to detect such cases where kanata can't handle the input because of the OS bug and suppress the program mapping just for those cases?

@jtroo Do you think it is feasible?

I don't know of any easy ways to detect that the key outputs are working properly, so for now we'll say it's infeasible to do automatically. One could do it manually though, by using the live reload prev/next actions to go between your normal config and a minimal one.

[felipecrp]

Thank you for the return and insight. Your idea worked. For instance, I will share a minimal configuration that others can use to cycle and use the emoji picker with kanata.

(defsrc
  caps 
)

(deflayer default
  lrld-next
)

[conventoangelo]

🚀⌨️🖱️Typed these emojis with kanata on. I was shocked that it started working randomly. Might have been fixed on the OS side. Keyboard inputs now accepted. Tested on Windows 11 23H2 OS Build 22631.4391.

[riotrah]

1. I too suddenly noticed this - it works flawlessly. I wonder if the powertoys version of this issue was the impetus
2. To that end, should we update docs/platform-known-issues.adoc's item about this?

...

3. (dumb aside: vaguely cool to see both jtroo & LGUG2Z on the same reaction above, my fave windows oss desktop devs!)