-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy path7b.log
15 lines (15 loc) · 1.47 KB
/
7b.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[2024-01-29 01:50:10,113] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Namespace(model='meta-llama/Llama-2-7b-hf', T=100, B=1, P=32)
[2024-01-29 01:50:54,082] [INFO] [logging.py:96:log_dist] [Rank -1] DeepSpeed info: version=0.12.6, git-hash=unknown, git-branch=unknown
[2024-01-29 01:50:54,085] [INFO] [logging.py:96:log_dist] [Rank -1] quantize_bits = 8 mlp_extra_grouping = False, quantize_groups = 1
Length :32, inference time:0.02243281126022339
[2024-01-29 01:51:03,065] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Namespace(model='meta-llama/Llama-2-7b-hf', T=100, B=1, P=64)
[2024-01-29 01:51:47,033] [INFO] [logging.py:96:log_dist] [Rank -1] DeepSpeed info: version=0.12.6, git-hash=unknown, git-branch=unknown
[2024-01-29 01:51:47,036] [INFO] [logging.py:96:log_dist] [Rank -1] quantize_bits = 8 mlp_extra_grouping = False, quantize_groups = 1
Length :64, inference time:0.024571387767791747
[2024-01-29 01:51:56,366] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
Namespace(model='meta-llama/Llama-2-7b-hf', T=100, B=1, P=128)
[2024-01-29 01:52:40,221] [INFO] [logging.py:96:log_dist] [Rank -1] DeepSpeed info: version=0.12.6, git-hash=unknown, git-branch=unknown
[2024-01-29 01:52:40,224] [INFO] [logging.py:96:log_dist] [Rank -1] quantize_bits = 8 mlp_extra_grouping = False, quantize_groups = 1
Length :128, inference time:0.02776776075363159