Skip to content
This repository has been archived by the owner on Oct 11, 2024. It is now read-only.

dhuangnm triggered nightly on refs/heads/main #128

dhuangnm triggered nightly on refs/heads/main

dhuangnm triggered nightly on refs/heads/main #128

Manually triggered May 14, 2024 19:40
Status Failure
Total duration 11h 49m 5s
Artifacts 6

nightly.yml

on: workflow_dispatch
BUILD-TEST  /  ...  /  BENCHMARK
5h 30m
BUILD-TEST / BENCHMARK / BENCHMARK
BUILD-TEST  /  ...  /  TEST
4h 2m
BUILD-TEST / TEST-SOLO / TEST
BUILD-TEST  /  ...  /  TEST
5h 5m
BUILD-TEST / TEST-MULTI / TEST
BUILD-TEST  /  ...  /  TEST-ACCURACY-SMOKE
8m 5s
BUILD-TEST / TEST-ACCURACY-SMOKE / TEST-ACCURACY-SMOKE
BUILD-TEST  /  ...  /  TEST-ACCURACY-FULL
BUILD-TEST / TEST-ACCURACY-FULL / TEST-ACCURACY-FULL
BUILD-TEST  /  ...  /  BENCHMARK_REPORT
20s
BUILD-TEST / BENCHMARK / BENCHMARK_REPORT
BUILD-TEST  /  ...  /  PUBLISH
21s
BUILD-TEST / PUBLISH / PUBLISH
Fit to window
Zoom out
Zoom in

Annotations

1 error and 1 warning
BUILD-TEST / BENCHMARK / BENCHMARK_REPORT
# :warning: **Performance Alert** :warning: Possible performance regression was detected for benchmark **'smaller_is_better'**. Benchmark result of this commit is worse than the previous benchmark result exceeding threshold `1.10`. | Benchmark suite | Current: 3a2545670126854a4a685edd889fe68f2fe250c3 | Previous: d485d3e5c9721b27cd0fe345d062fabc038049a1 | Ratio | |-|-|-|-| | `{"name": "mean_ttft_ms", "description": "VLLM Serving - 2:4 Sparse\nmodel - neuralmagic/OpenHermes-2.5-Mistral-7B-pruned2.4\nmax-model-len - 4096\nsparsity - semi_structured_sparse_w16a16\nbenchmark_serving {\n \"nr-qps-pair_\": \"1500,5\",\n \"dataset\": \"sharegpt\"\n}", "gpu_description": "NVIDIA A10G x 1", "vllm_version": "0.2.0", "python_version": "3.10.12 (main, May 10 2024, 13:42:25) [GCC 9.4.0]", "torch_version": "2.3.0+cu121"}` | `284.8694550293367` ms | `253.06591161863494` ms | `1.13` | This comment was automatically generated by [workflow](https://github.com/neuralmagic/nm-vllm/actions?query=workflow%3ANightly) using [github-action-benchmark](https://github.com/marketplace/actions/continuous-benchmark). Comment was generated at https://github.com/neuralmagic/nm-vllm/commit/3a2545670126854a4a685edd889fe68f2fe250c3#commitcomment-142020198
BUILD-TEST / BENCHMARK / BENCHMARK_REPORT
Performance alert! Previous value was 253.06591161863494 and current value is 284.8694550293367. It is 1.1256729648307158x worse than previous exceeding a ratio threshold 1.1

Artifacts

Produced during runtime
Name Size
3.10.12-nm-vllm-0.2.0.tar.gz Expired
534 KB
9085220879-aws-avx2-32G-a10g-24G Expired
124 KB
cc-vllm-html-aws-avx2-192G-4-a10g-96G Expired
2.23 MB
cc-vllm-html-aws-avx2-32G-a10g-24G Expired
2.23 MB
gh_action_benchmark_jsons-9085220879-aws-avx2-32G-a10g-24G Expired
28.4 KB
nm_vllm-0.2.0-cp310-cp310-manylinux_2_17_x86_64.whl Expired
103 MB