MINOR: [Release] Update versions for 16.1.0
conbench-apache-arrow / Conbench performance report
failed
Jun 3, 2024 in 0s
Found 82 regressions
Thanks for your patience. Conbench analyzed the 6 benchmarking runs that have been run so far on PR commit 7dd1d34.
Benchmarks with performance regressions
There were 15 possible performance regressions, according to the lookback z-score method.
Benchmarks with regressions:
-
Pull Request Run on
arm64-t4g-linux-computeat 2024-05-27 11:49:41Z -
Pull Request Run on
arm64-m6g-linux-computeat 2024-05-27 11:39:34Ztpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-17, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-14, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-05, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-18, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-07, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-10, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-08, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-20, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-19, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-22, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-09, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-04, scale_factor=10tpch(R) with engine=arrow, format=parquet, language=R, memory_map=False, query_id=TPCH-12, scale_factor=1
All benchmark runs analyzed:
- Pull Request Run on
ursa-thinkcentre-m75qat 2024-05-27 11:41:32Z - Pull Request Run on
ec2-t3-xlarge-us-east-2at 2024-05-27 11:49:43Z - Pull Request Run on
ec2-m5-4xlarge-us-east-2at 2024-05-27 11:30:24Z - Pull Request Run on
test-mac-armat 2024-05-27 11:34:34Z - Pull Request Run on
arm64-t4g-linux-computeat 2024-05-27 11:49:41Z - Pull Request Run on
arm64-m6g-linux-computeat 2024-05-27 11:39:34Z
Unstable benchmarks with performance regressions
The following benchmark results indicate a possible performance regression, but are known to sometimes produce false positives when applying the lookback z-score analysis.
-
Pull Request Run on
ursa-thinkcentre-m75qat 2024-05-27 11:41:32ZArrayArrayKernel(C++) with params=<AddChecked, Int64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkSerialTaskGroup(C++) with params=task_cost:10000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmarkExecuteScalarExpressionBaseline(C++) with params=/rows_per_batch:100000/real_time/threads:1, source=cpp-micro, suite=arrow-acero-expression-benchmarkArrayArrayKernel(C++) with params=<AddChecked, UInt64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBatchToTensorSimple(C++) with params=/size:524288/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkShortVectorInsert(C++) with params=<STATIC_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBatchToTensorSimple(C++) with params=/size:524288/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkConstructFromEmptyStdVector(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBatchToTensorSimple(C++) with params=/size:524288/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkMoveEmptyVector(C++) with params=<STATIC_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBatchToTensorSimple(C++) with params=/size:524288/num_columns:3, source=cpp-micro, suite=arrow-tensor-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::REQUIRED, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkUnaryOp(C++) with source=cpp-micro, suite=arrow-decimal-benchmarkErrorSchemeExceptionNoInline(C++) with source=cpp-micro, suite=arrow-type-benchmarkArrayArrayKernel(C++) with params=<SubtractChecked, UInt64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::REQUIRED, Compression::SNAPPY>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkSerialTaskGroup(C++) with params=task_cost:1000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmarkThreadedTaskGroup(C++) with params=threads:2/task_cost:100000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::REPEATED, Compression::SNAPPY>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkShortVectorPushBack(C++) with params=<STATIC_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmarkSerialTaskGroup(C++) with params=task_cost:100000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmarkThreadedTaskGroup(C++) with params=threads:4/task_cost:100000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::OPTIONAL, Compression::ZSTD>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkShortVectorInsert(C++) with params=<SMALL_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmarkBitmapUInt64Reader(C++) with params=8192, source=cpp-micro, suite=arrow-bit-util-benchmarkShortVectorPushBack(C++) with params=<SMALL_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkErrorSchemeBoolNoInline(C++) with source=cpp-micro, suite=arrow-type-benchmarkArrayArrayKernel(C++) with params=<SubtractChecked, Int64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkExecuteScalarExpressionBaseline(C++) with params=/rows_per_batch:10000/real_time/threads:1, source=cpp-micro, suite=arrow-acero-expression-benchmarkThreadedTaskGroup(C++) with params=threads:1/task_cost:100000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmarkShortVectorInsertAtEnd(C++) with params=<SMALL_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkArrayArrayKernel(C++) with params=<MultiplyChecked, UInt64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkConstructFromEmptyStdVector(C++) with params=<SMALL_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkArrayArrayKernel(C++) with params=<MultiplyChecked, Int64Type>/size:524288/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkBM_ReadInt64Column(C++) with params=<Repetition::OPTIONAL, Compression::SNAPPY>/65536/1024, source=cpp-micro, suite=parquet-column-io-benchmarkThreadedTaskGroup(C++) with params=threads:1/task_cost:10000/real_time, source=cpp-micro, suite=arrow-thread-pool-benchmark
-
Pull Request Run on
test-mac-armat 2024-05-27 11:34:34ZTouchArea(C++) with params=size:4096/real_time/threads:1, source=cpp-micro, suite=arrow-memory-pool-benchmarkReadCachedFile(C++) with params=num_cols:512/is_partial:1/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkShortVectorInsertAtEnd(C++) with params=<STATIC_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkGreaterArrayArrayString(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkAllocateTouchDeallocate(C++) with params=/size:65536/real_time/threads:4, source=cpp-micro, suite=arrow-memory-pool-benchmarkValidateTinyAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkValidateSmallAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkGreaterArrayScalarString(C++) with params=65536/1, source=cpp-micro, suite=arrow-compute-scalar-compare-benchmarkArrayScalarKernel(C++) with params=<AddChecked, Int64Type>/size:4194304/inverse_null_proportion:0, source=cpp-micro, suite=arrow-compute-scalar-arithmetic-benchmarkReadCachedFile(C++) with params=num_cols:64/is_partial:1/real_time, source=cpp-micro, suite=arrow-ipc-read-write-benchmarkTouchArea(C++) with params=size:4096/real_time/threads:4, source=cpp-micro, suite=arrow-memory-pool-benchmarkRandomKernelSystem(C++) with params=65536, source=cpp-micro, suite=arrow-compute-scalar-random-benchmarkAllocateTouchDeallocate(C++) with params=/size:1048576/real_time/threads:1, source=cpp-micro, suite=arrow-memory-pool-benchmarkTouchArea(C++) with params=size:65536/real_time/threads:2, source=cpp-micro, suite=arrow-memory-pool-benchmarkHexParsing(C++) with params=, source=cpp-micro, suite=arrow-value-parsing-benchmarkTouchArea(C++) with params=size:65536/real_time/threads:1, source=cpp-micro, suite=arrow-memory-pool-benchmarkConstructFromEmptyStdVector(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkCopyEmptyVector(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmark
-
Pull Request Run on
arm64-t4g-linux-computeat 2024-05-27 11:49:41ZShortVectorInsertAtEnd(C++) with params=<STATIC_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkValidateSmallAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkCopyShortVector(C++) with params=<SMALL_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmarkConstructFromEmptyStdVector(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkShortVectorInsertAtEnd(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkShortVectorPushBack(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkCopyShortVector(C++) with params=<SMALL_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkCopyShortVector(C++) with params=<STATIC_VECTOR(std::shared_ptr)>, source=cpp-micro, suite=arrow-small-vector-benchmarkShortVectorPushBack(C++) with params=<STATIC_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkShortVectorInsertAtEnd(C++) with params=<SMALL_VECTOR(int)>, source=cpp-micro, suite=arrow-small-vector-benchmarkValidateLargeAscii(C++) with source=cpp-micro, suite=arrow-utf8-util-benchmarkConstructFromShortStdVector(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmarkShortVectorPushBack(C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmark
Details
This report was generated using the lookback z-score method with a z-score threshold of 30.0.
Loading