TFRT

Description

• New runtime aimed to replace the existing TensorFlow runtime.
• Responsible for efficient execution of kernels – low-level device-specific primitives – on target hardware.
• Last update in Feb 2022 via blog. No update since then.

Device support (as of 2022)

• CPU - yes
• TPU - Yes
• Cloud TPU - In progress
• GPU - In progress

Usage (as of 2022)

• "Deployed broadly inside Google for training and inference"
• Pixel phones - Being explored.
• Vertex AI - Available as a preview (https://cloud.google.com/vertex-ai/docs/predictions/optimized-tensorflow-runtime)
• Waymo - Being integrated.

Resources

• TFRT: A new TensorFlow runtime - https://blog.tensorflow.org/2020/04/tfrt-new-tensorflow-runtime.html
• TFRT: Progress update - https://blog.tensorflow.org/2022/02/tfrt-progress-update.html

See also

• TensorFlow
• TVM
• MLIR
• oneDNN | Eigen
• IREE