Tensorflow Lite

[george-ai]

Are there any future plans to use tensorflow lite?

[MatthewScholefield]

Tensorflow lite doesn't currently directly support lstm layers. It sounds like it might be possible by manually implementing it, but we won't be looking into it until we've found an example of someone else doing this successfully online.

[goddade]

https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/examples/lstm/g3doc
Can this help?

[andreselizondo-adestech]

Do we have any news on this? It'd be great to have support for this on precise-runner (for lightweight applications).
Training wouldn't need to be done on TF Lite.

[andreselizondo-adestech]

Hello all!
I've forked the project and updated it to use TF 2.2.0 (currently using 2.2.0rc2 as official release is yet to come) and Keras 2.3.1
You can find the fork here: https://github.com/andreselizondo-adestech/mycroft-precise

A few things to note:
- Removed all pocketsphynx dependencies and scripts.
- TF 2.x doesn't expose graphs/sessions (only if using tf.compat.v1). This fork makes things work without explicitly sharing sessions between TF and Keras.
- TF 2.1 doesn't support GRU layers correctly for tflite. TF 2.2 is minimum requirement.
- The only scripts I tested for compatibility are the ones I use:
  - precise-train (train keras model)
  - precise-convert (convert keras model to tflite)
  - precise-test (test keras or tflite models)
  - precise-listen (listen on keras or tflite models)
- Added a TFLiteRunner that allows using tflite models with the previous scripts.
- Added my custom training parameters for my model on scripts/train.py
- I couldn't get TF 2.x to support TensorBoard. Decided it was not important.

I'm no expert on TF/Keras usage, so I might've butchered some things.
However, I believe this is a good starting point for resolving this issue.
Let me know if I can be of further assistance!

[krisgesling]

Eltocino just flagged this TFlite accelerator library as a possibility:
https://blog.tensorflow.org/2020/07/accelerating-tensorflow-lite-xnnpack-integration.html

Posting it here so I don't forget it, and in case anyone's interested in trying it out.