Float16 type support

[BenjaminPoulain]

The current spec defines the tensor types "float32", "int32" and "8 bits quantized".

We should consider "float16" for a few reasons:

• Certain hardware are significantly slower handling float32 compared to float16. This includes but is not limited to GPUs.
• Storing the intermediate values in float16 can be beneficial to reduce memory footprint and memory bandwidth. This is true even if all the operations are done on float32 in the FPUs.
• Supporting constant tensors in float16 would allow clients to half the size of the model's parameters. This can reduce models size in memory and reduce network bandwidth when loading the page.

Implementations without native float16 support should be allowed to use float32 internally. This would help maximizing performance on float32 hardware while keeping things simple for the authors.

[huningxin]

Thanks for proposing this @BenjaminPoulain . Adding "float16" support makes sense to me.

[anssiko]

Resolution from https://www.w3.org/2020/01/09-webmachinelearning-minutes.html#x04

Add "float16" to OperandType enum in WebNN API and define a way for the API to respond when float16 is not natively supported

@huningxin feel free to craft a PR for the group to review. Thanks!

[wchao1115]

Agreed with @BenjaminPoulain on the stated benefits of supporting float16 in the API. Models with float16 weights and features have become increasingly popular in real-world use cases especially on the GPU e.g. Nvidia TensorCore in the Pascal or newer generation is specifically float16-based. Float16 is also a primary native data type on most DSP-based AI hardware chips e.g. Intel Movidius.

That said, in absence of native support in the underlying hardware, internal simulation such as up-casting the data type on the fly could lead to severe performance penalty and sometimes unexpected computation errors. The API should be designed such that it could fail the operation with unsupported error to allow the client an opportunity to gracefully fallback either by switching to a different target computing device, or to a model variation that simply processes float32 weights and feature sets.

[huningxin]

@huningxin feel free to craft a PR for the group to review. Thanks!

Put together #35. @BenjaminPoulain @wchao1115 @anssiko , please take a look. Thanks.

[BenjaminPoulain]

Closing. @huningxin resolved the issue in #35.