Google

About

• American multinational technology company that specializes in Internet-related services and products

Small CPU models usage in Ads team.

Small CPU models that are embedded in application servers, invoked within the application process instead of via RPC/REST calls, remain critical for some of Google’s business workloads from users like Ads.

Source: https://blog.tensorflow.org/2022/02/tfrt-progress-update.html

ML Inference

Name	Program	Size (MB) (Avg/Max)	Programs	Std Dev	Range
MLP0	-	580/2500	27	±17	1-93
MLP1	RankBrain	90/NA	5	±0.3	1-5
CNN0	AlphaZero	60/454	1	-	1
CNN1	Image Classification Model	120/680	6	±10	1-34
RNN0	Transformer encoder + LSTM decoder	1300/1300	13	±3	1-29
RNN1	Wave RNN	120/400	1	-	-
BERT0	-	3000/3000	9	±2	1-14
BERT1	DeepRank	90/N.A	5	±0.3	1-5

Percentage usage

Type	% use in 2016	% use in 2020
MLP	61%	25%
CNN	5%	18%
RNN	0%	29%
BERT	0%	28%

ML software

• TensorFlow | XLA
• MLIR

ML hardware

• TPU

ML models

• BERT T5

See also

• Google | Meta | Amazon | Microsoft | Apple
• Intel | Arm | AMD