Support arrays of zeros in Vec's __rust_alloc_zeroed optimization

[scottmcm]

I happened to notice in https://users.rust-lang.org/t/any-advantage-of-box-u64-16-16-16-over-vec-u64/73500/3?u=scottmcm that the calloc optimization wasn't applying to vectors-of-arrays, so here's the easy fix for that.

[rust-highfive]

r? @joshtriplett

(rust-highfive has picked a reviewer for you, use r? to override)

[Dylan-DPC]

r? @Mark-Simulacrum

[Mark-Simulacrum]

Hm. The current IsZero implementations were all really cheap -- basically a single cmp -- while scanning an array to check it's all zeroes is comparatively expensive. That said, I think we don't currently use it for anything except the calloc optimization, and there I think it is likely to be a win to know up front you're allocating a set of zeroes.

I suppose with the possible exception where the array is long and total number of elements is low, so the extra pass over the array ends up being more expensive than the knowledge the time saved with memset(0) over memcpy. But probably that case is quite rare.

[Mark-Simulacrum]

r=me if you agree with that rationale, otherwise we can discuss more.

[nbdd0121]

Yeah, I think this'll be too expensive for large array. It might make sense to do this only when its size is small enough, e.g. containable in a cache line.

[Mark-Simulacrum]

Part of my worry is that due to various processor optimizations and e.g. the ability to calloc by directly asking the kernel for zeroed pages (in theory), there's a really hard balance to strike for this kind of thing. So for short arrays it definitely seems reasonable -- where the cost of checking is quite low. I guess it's purely an optimization though, so should be easy to make that happen.

[nbdd0121]

The issue is that if the array is a chain of zeros followed by a non-zero byte, then we spend all time checking it without using the calloc path.

[scottmcm]

Yes, the trait isn't even mem::IsZero, it's alloc::vec::IsZero, because it's only used for this.

Basically, the odds of a particularly long only-non-zero-at-the-end array getting passed to vec![x; n] seem low enough to me that it's ok as-is. And we can always tweak the check later if needed.

@bors r=Mark-Simulacrum

(It's not obvious where the threshold for stopping would be if we did make one, because I suspect that the things passed to vec! are often literals, where even something like vec![[0_u8; 4096]; n] is completely fine because the check would be const-folded.)

Oh, and in the long-term I look forward to having something from safe transmute so we'll be able to just have a Vec::zeroed(n) that's safe by using those traits to ensure sure values are safe to construct, so that people will be able to get this directly and obviously, rather than needing these tricks internally. At that point restricting it to shorter arrays would make more sense to me, since with runtime values it's less obviously useful to have this check.

[bors]

📌 Commit 8034c45 has been approved by Mark-Simulacrum

[nbdd0121]

We could have different optimization for literals/values visible to optimizer. I am not convinced that a runtime check is a good one.

[nbdd0121]

(It's not obvious where the threshold for stopping would be if we did make one, because I suspect that the things passed to vec! are often literals, where even something like vec![[0_u8; 4096]; n] is completely fine because the check would be const-folded.)

This doesn't seem true: https://godbolt.org/z/KcP1sjEj3

[bors]

⌛ Testing commit 8034c45 with merge bf61143...

[bors]

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing bf61143 to master...

[rust-timer]

Finished benchmarking commit (bf61143): comparison url.

Summary: This benchmark run did not return any relevant results.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

[ChayimFriedman2]

(It's not obvious where the threshold for stopping would be if we did make one, because I suspect that the things passed to vec! are often literals, where even something like vec![[0_u8; 4096]; n] is completely fine because the check would be const-folded.)

This doesn't seem true: https://godbolt.org/z/KcP1sjEj3

Test shows that current LLVM threshold is up to 98 elements (including), so it may make sense to limit N?

[nbdd0121]

The value I got is 49 elements.

[scottmcm]

@ChayimFriedman2 I also got 49 -- but see #96596 where I've already limited this check.

[ChayimFriedman2]

The value I got is 49 elements.

I used -Copt-level=3, you probably used -O which is opt level 2.