Asyncify whitelist and blacklist support

[kripken]

A slight complication here is that we need function names for the lists. This extends the current symbol-map mechanism to keep around function names in the wasm until the very end of the optimization pipeline. (There are still risks with inlining and duplicate function elimination changing things, though, which can't be solved without adding some new special function annotation.)

This depends on WebAssembly/binaryen#2218 for tests to pass.

cc @Keno @caiiiycuk @gabrielcuvillier - if you're curious to try this out, let me know how it goes!

[caiiiycuk]

@kripken I tried new WHITELIST feature of asyncify. I used emscripten/incoming, binaryen/incoming and emsdk/clang-incoming-64bin (9.0). Hope this envronment config is valid, because emcc told me

shared:WARNING: LLVM version appears incorrect (seeing "9.0", expected "10.0")

many times.

Also I have those warnings, which seems strange:

warning: Asyncify whitelist contained a non-existing function name: _main ($_main)
warning: Asyncify whitelist contained a non-existing function name: __Z10SHELL_Initv ($__Z10SHELL_Initv)
warning: Asyncify whitelist contained a non-existing function name: __Z12DOS_ReadFiletPhPtb ($__Z12DOS_ReadFiletPhPtb)
warning: Asyncify whitelist contained a non-existing function name: __Z17DOSBOX_RunMachinev ($__Z17DOSBOX_RunMachinev)
warning: Asyncify whitelist contained a non-existing function name: __Z19CALLBACK_RunRealInth ($__Z19CALLBACK_RunRealInth)
warning: Asyncify whitelist contained a non-existing function name: __Z19CALLBACK_RunRealFartt ($__Z19CALLBACK_RunRealFartt)
warning: Asyncify whitelist contained a non-existing function name: __Z13CALLBACK_Idlev ($__Z13CALLBACK_Idlev)
warning: Asyncify whitelist contained a non-existing function name: __ZL11Normal_Loopv ($__ZL11Normal_Loopv)
warning: Asyncify whitelist contained a non-existing function name: __ZL13DOS_21Handlerv ($__ZL13DOS_21Handlerv)
warning: Asyncify whitelist contained a non-existing function name: __ZL16PROGRAMS_Handlerv ($__ZL16PROGRAMS_Handlerv)
warning: Asyncify whitelist contained a non-existing function name: __ZN10device_CON4ReadEPhPt ($__ZN10device_CON4ReadEPhPt)
warning: Asyncify whitelist contained a non-existing function name: __ZN10device_CONC2Ev ($__ZN10device_CONC2Ev)
warning: Asyncify whitelist contained a non-existing function name: __ZN10DOS_Device4ReadEPhPt ($__ZN10DOS_Device4ReadEPhPt)
warning: Asyncify whitelist contained a non-existing function name: __ZN3MEM3RunEv ($__ZN3MEM3RunEv)
warning: Asyncify whitelist contained a non-existing function name: __ZN5INTRO3RunEv ($__ZN5INTRO3RunEv)
warning: Asyncify whitelist contained a non-existing function name: __ZN4BOOT3RunEv ($__ZN4BOOT3RunEv)
warning: Asyncify whitelist contained a non-existing function name: __ZN6Config7StartUpEv ($__ZN6Config7StartUpEv)
warning: Asyncify whitelist contained a non-existing function name: __ZN6Config4InitEv ($__ZN6Config4InitEv)
warning: Asyncify whitelist contained a non-existing function name: __ZN7Section11ExecuteInitEb ($__ZN7Section11ExecuteInitEb)
warning: Asyncify whitelist contained a non-existing function name: __ZL11GUI_StartUpP7Section ($__ZL11GUI_StartUpP7Section)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell12InputCommandEPc ($__ZN9DOS_Shell12InputCommandEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell3RunEv ($__ZN9DOS_Shell3RunEv)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell7ExecuteEPcS0_ ($__ZN9DOS_Shell7ExecuteEPcS0_)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell9DoCommandEPc ($__ZN9DOS_Shell9DoCommandEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell9ParseLineEPc ($__ZN9DOS_Shell9ParseLineEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell8CMD_HELPEPc ($__ZN9DOS_Shell8CMD_HELPEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell9CMD_PAUSEEPc ($__ZN9DOS_Shell9CMD_PAUSEEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell10CMD_CHOICEEPc ($__ZN9DOS_Shell10CMD_CHOICEEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell7CMD_DIREPc ($__ZN9DOS_Shell7CMD_DIREPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell8CMD_TIMEEPc ($__ZN9DOS_Shell8CMD_TIMEEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell7CMD_CLSEPc ($__ZN9DOS_Shell7CMD_CLSEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell8CMD_DATEEPc ($__ZN9DOS_Shell8CMD_DATEEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZN9DOS_Shell8CMD_COPYEPc ($__ZN9DOS_Shell8CMD_COPYEPc)
warning: Asyncify whitelist contained a non-existing function name: __ZL14Reboot_Handlerv ($__ZL14Reboot_Handlerv)
warning: Asyncify whitelist contained a non-existing function name: __Z22DOS_LoadKeyboardLayoutPKciS0_ ($__Z22DOS_LoadKeyboardLayoutPKciS0_)
warning: Asyncify whitelist contained a non-existing function name: __Z23DOS_KeyboardLayout_InitP7Section ($__Z23DOS_KeyboardLayout_InitP7Section)
warning: Asyncify whitelist contained a non-existing function name: __Z24DOS_SwitchKeyboardLayoutPKcRi ($__Z24DOS_SwitchKeyboardLayoutPKcRi)
warning: Asyncify whitelist contained a non-existing function name: __ZN15keyboard_layout18read_codepage_fileEPKci ($__ZN15keyboard_layout18read_codepage_fileEPKci)
warning: Asyncify whitelist contained a non-existing function name: __ZN15keyboard_layout22switch_keyboard_layoutEPKcRPS_Ri ($__ZN15keyboard_layout22switch_keyboard_layoutEPKcRPS_Ri)
warning: Asyncify whitelist contained a non-existing function name: __ZN18DOS_KeyboardLayoutC2EP7Section ($__ZN18DOS_KeyboardLayoutC2EP7Section)
warning: Asyncify whitelist contained a non-existing function name: __ZN4KEYB3RunEv ($__ZN4KEYB3RunEv)
warning: Asyncify whitelist contained a non-existing function name: __Z12PIC_RunQueuev ($__Z12PIC_RunQueuev)
warning: Asyncify whitelist contained a non-existing function name: __Z15MAPPER_RunEventj ($__Z15MAPPER_RunEventj)
warning: Asyncify whitelist contained a non-existing function name: __Z18MAPPER_RunInternalv ($__Z18MAPPER_RunInternalv)
warning: Asyncify whitelist contained a non-existing function name: __ZL10Pause_Loopv ($__ZL10Pause_Loopv)
warning: Asyncify whitelist contained a non-existing function name: __ZN4WGET3RunEv ($__ZN4WGET3RunEv)

In runtime I have this error

VM31:8 extracting: 'dosbox.conf', size: 23, mtime: 1419884700
VM31:8 DOSBox version 6.22.30 (c335ebdccf281f383f2c215b69f1598e)
VM31:8 Copyright 2002-2015 DOSBox Team, published under GNU GPL.
VM31:8 ---
VM31:8 CONFIG:Loading primary settings from config file /home/web_user/.dosbox/dosbox-jsdos.conf
VM31:8 invalid state: 1
VM31:8 invalid state: 1
abort @ VM31:8
handleSleep @ VM31:8
_emscripten_sleep @ VM31:8
wasm-function[2686] @ wasm-007fc4e2-2686:1563 (GUI_StartUp\28Section*\29)
wasm-function[3010] @ wasm-007fc4e2-3010:29 (Config::Init\28\29)
wasm-function[1733] @ wasm-007fc4e2-1733:1201 (main)
ret.(anonymous function) @ VM31:8
Module._main @ VM31:8
callMain @ VM31:8
main @ js-dos-module.ts:176
fs.extract.then @ index.html:21
Promise.then (async)
Dos.ready @ index.html:20
(anonymous) @ js-dos.ts:92
Promise.then (async)
e.ready @ js-dos.ts:91
(anonymous) @ index.html:19

Also I little bit worried that symbols file now contain demangled names, it's breaking change for tools such bug reporters.

Seems I need to demangle whitelist file to use it with this feature, is it correct? (I replaced "_main" with "main" and it remove one warning).

[caiiiycuk]

After passing function names with c++filt -_, I have only this warnings

warning: Asyncify whitelist contained a non-existing function name: DOS_ReadFile(unsigned short ($DOS_ReadFile\28unsigned\20short)
warning: Asyncify whitelist contained a non-existing function name: 
"DOS_ReadFile(unsigned short ($\0a\22DOS_ReadFile\28unsigned\20short)
warning: Asyncify whitelist contained a non-existing function name:  unsigned char* ($\20unsigned\20char*)
warning: Asyncify whitelist contained a non-existing function name:  unsigned short* ($\20unsigned\20short*)
warning: Asyncify whitelist contained a non-existing function name:  bool) ($\20bool\29)
warning: Asyncify whitelist contained a non-existing function name: CALLBACK_RunRealFar(unsigned short ($CALLBACK_RunRealFar\28unsigned\20short)
warning: Asyncify whitelist contained a non-existing function name: 
"CALLBACK_RunRealFar(unsigned short ($\0a\22CALLBACK_RunRealFar\28unsigned\20short)
warning: Asyncify whitelist contained a non-existing function name:  unsigned short) ($\20unsigned\20short\29)
warning: Asyncify whitelist contained a non-existing function name: device_CON::Read(unsigned char* ($device_CON::Read\28unsigned\20char*)
warning: Asyncify whitelist contained a non-existing function name: 
"device_CON::Read(unsigned char* ($\0a\22device_CON::Read\28unsigned\20char*)
warning: Asyncify whitelist contained a non-existing function name:  unsigned short*) ($\20unsigned\20short*\29)
warning: Asyncify whitelist contained a non-existing function name: device_CON::device_CON() ($device_CON::device_CON\28\29)
warning: Asyncify whitelist contained a non-existing function name: DOS_Device::Read(unsigned char* ($DOS_Device::Read\28unsigned\20char*)
warning: Asyncify whitelist contained a non-existing function name: 
"DOS_Device::Read(unsigned char* ($\0a\22DOS_Device::Read\28unsigned\20char*)
warning: Asyncify whitelist contained a non-existing function name:  unsigned short*) ($\20unsigned\20short*\29)
warning: Asyncify whitelist contained a non-existing function name: Config::StartUp() ($Config::StartUp\28\29)
warning: Asyncify whitelist contained a non-existing function name: DOS_Shell::Execute(char* ($DOS_Shell::Execute\28char*)
warning: Asyncify whitelist contained a non-existing function name: 
"DOS_Shell::Execute(char* ($\0a\22DOS_Shell::Execute\28char*)
warning: Asyncify whitelist contained a non-existing function name:  char*) ($\20char*\29)
warning: Asyncify whitelist contained a non-existing function name: DOS_LoadKeyboardLayout(char const* ($DOS_LoadKeyboardLayout\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name: 
"DOS_LoadKeyboardLayout(char const* ($\0a\22DOS_LoadKeyboardLayout\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name:  int ($\20int)
warning: Asyncify whitelist contained a non-existing function name:  char const*) ($\20char\20const*\29)
warning: Asyncify whitelist contained a non-existing function name: DOS_SwitchKeyboardLayout(char const* ($DOS_SwitchKeyboardLayout\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name: 
"DOS_SwitchKeyboardLayout(char const* ($\0a\22DOS_SwitchKeyboardLayout\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name:  int&) ($\20int&\29)
warning: Asyncify whitelist contained a non-existing function name: keyboard_layout::read_codepage_file(char const* ($keyboard_layout::read_codepage_file\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name: 
"keyboard_layout::read_codepage_file(char const* ($\0a\22keyboard_layout::read_codepage_file\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name:  int) ($\20int\29)
warning: Asyncify whitelist contained a non-existing function name: keyboard_layout::switch_keyboard_layout(char const* ($keyboard_layout::switch_keyboard_layout\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name: 
"keyboard_layout::switch_keyboard_layout(char const* ($\0a\22keyboard_layout::switch_keyboard_layout\28char\20const*)
warning: Asyncify whitelist contained a non-existing function name:  keyboard_layout*& ($\20keyboard_layout*&)
warning: Asyncify whitelist contained a non-existing function name:  int&) ($\20int&\29)

I think parser didn't catch function names like DOS_ReadFile(unsigned short, unsigned char*, unsigned short*, bool) guess should be DOS_ReadFile\28unsigned\20short\2c\20unsigned\20char*\2c\20unsigned\20short*\2c\20bool\29, btw build still not work with this error:

Uncaught RuntimeError: function signature mismatch
    at wasm-function[489]:1103 (DOS_Shell::DoCommand\28char*\29)
    at wasm-function[354]:273 (GFX_CaptureMouse\28\29)
    at wasm-function[1224]:816 (DOS_Shell::Run\28\29)
    at wasm-function[2248]:1589 (SHELL_Init\28\29)
    at wasm-function[1733]:3535 (main)
    at Object.ret.(anonymous function) [as eb] (eval at success (http://127.0.0.1:8080/js-dos.js:1:22154), <anonymous>:8:184773)
    at eval (eval at success (http://127.0.0.1:8080/js-dos.js:1:22154), <anonymous>:8:185993)
    at eval (eval at success (http://127.0.0.1:8080/js-dos.js:1:22154), <anonymous>:8:97176)

[caiiiycuk]

Nope, even I used exact name from symbols:
DOS_ReadFile\28unsigned\20short\2c\20unsigned\20char*\2c\20unsigned\20short*\2c\20bool\29
I still have this warning:

warning: Asyncify whitelist contained a non-existing function name: DOS_ReadFile\28unsigned\20short\2c\20unsigned\20char*\2c\20unsigned\20short*\2c\20bool\29 ($DOS_ReadFile\5c28unsigned\5c20short\5c2c\5c20unsigned\5c20char*\5c2c\5c20unsigned\5c20short*\5c2c\5c20bool\5c29)

[kripken]

@caiiiycuk yeah, this is tricky - the wasm backend emits "human-readable" names, not normal mangled ones. I added some notes in settings.js about this.

I'm surprised by your second comment though. Perhaps c++filt doesn't do the exact right thing? Building the wasm with --profiling (so names are kept in the final binary) and inspecting the binary itself might help.