Support Flash by rerouting keyboard focus

[1ec5]

Flash plugin support has been by far the most elusive feature request I’ve gotten. The core issue is that Firefox doesn’t allow chrome or content scripts to capture keyboard and mouse events targeted at plugins – they go directly to the plugin process and never bubble up. For Silverlight, we’re able to get around this restriction using Silverlight’s legacy JavaScript API. But Flash has no general-purpose JavaScript API; the ExternalInterface API only allows the animation to communicate with the webpage via an interface explicitly defined by the animation’s author.

At one point there was a package that Java developers could bundle with their applets to imbue the applets with Vietnamese IME functionality. A similar approach would be possible with Flash, with an ActionScript library, but it’s far from certain that any Flash developer would be interested in embedding such a library, even if Vietnamese webmasters are comfortable embedding avim.js and other scripts into webpages.

Maybe one way to get around this situation is to steal keyboard focus whenever it goes to a Flash plugin and route it to an anonymous element inside the <object> element. An event handler attached to the anonymous element would forward all mouse events directly to the plugin but only forward keyboard events after allowing AVIM to process them. AVIM would process a keystroke by synthesizing copy-paste events, just like it does in Google Docs. In principle, this approach would work for any plugin at all, and we could even “fix” bug 78414 for AVIM users by retargeting certain key combinations to the surrounding webpage instead.

This is all very hand-wavy and totally untested at the moment. How will Flash react to these synthetic events? (Even the DOM events that this anonymous element forwards may not look like the raw events the plugin normally receives.) Do certain animations get throttled based on the plugin truly receiving focus?

Of course, there would be little net benefit to implementing this workaround for Flash once Shumway supports TextInput (mozilla/shumway#1590) or for Adobe Reader once pdf.js supports AcroForms input (mozilla/pdf.js#1459).

[1ec5]

The developer of Viettyping.js and VinaMoziE (like AVIM but for the ancient Mozilla Application Suite) long ago released a Flash component called Viettyping for Flash that works very similar to the Java package I described above. But to my knowledge, it has hardly ever been used anywhere, proving my point about needing a solution that doesn’t rely on Flash authors.

[1ec5]

The first challenge would be to get notified whenever a plugin gets focus. The focusin event isn’t supported by Firefox, but it should be possible to capture focus events.

[1ec5]

Yep, we can capture focus events.

Another hurdle is that AVIM won’t be able to tell whether a TextInput control is focused when the keypress event comes in. (Edit menu items act on the page even when a Flash animation is focused.) We’ll need something like AdBlock Plus’s “index tab” to use as a toggle for AVIM on specific Flash animations.

[1ec5]

Out-of-process plugins doesn’t seem to respond to synthetic KeyEvents targeted at them. Oh well. :-(