Tracking Issue for secure random data generation in std

[joboet]

Feature gate: #![feature(random)]

This is a tracking issue for secure random data generation support in std.

Central to this feature are the Random and RandomSource traits inside core::random. The Random trait defines a method to create a new random value of the implementing type from random bytes generated by a RandomSource. std also exposes the platform's secure random number generator via the DefaultRandomSource type which can be conveniently access via the random::random function.

Public API

// core::random

pub trait RandomSource {
    fn fill_bytes(&mut self, bytes: &mut [u8]);
}

pub trait Random {
    fn random(source: &mut (impl RandomSource + ?Sized)) -> Self;
}

impl Random for bool { ... }
impl Random for /* all integer types */ { ... }

// std::random (additionally)

pub struct DefaultRandomSource;

impl RandomSource for DefaultRandomSource { ... }

pub fn random<T: Random>() -> T { ... }

Steps / History

• ACP: Simple secure random number generation libs-team#393
• Implementation: std: implement the random feature (alternative version) #129201
• Final comment period (FCP)¹
• Stabilization PR

Unresolved Questions

• Naming: the ACP used gen_bytes and DefaultRng, the implementation PR uses fill_bytes and DefaultRandomSource (see arguments pro gen_bytes and pro fill_bytes)
• Concerns listed at Simple secure random number generation libs-team#393 (comment) should be addressed

Footnotes

1. https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

[newpavlov]

Disclaimer: I am one of the getrandom developers.

I think it's important for RandomSource methods to properly return potential errors. Getting randomness is an IO operation and it may fail. In some context it's important to process such errors instead of panicking. The error may be either io::Error or something like getrandom::Error (i.e. a thin wrapper around NonZeroU32).

It may be worth to add the following methods to RandomSource:

• fill_bytes which works with uninitialized buffers, e.g. based on BorrowedBuf. Yes, zeroization of buffers is usually a very small cost compared to a syscall, but it still goes against the zero-cost spirit.
• Generation of u32 and u64. Some platforms support direct generation of such values (e.g. RDRAND, WASI, etc.). Going through fill_bytes will be a bit less efficient in such cases.
• Methods for potentially "insecure" generation of random values, but which are less prone to blocking. The HashMap seeding is the most obvious use-case for this.

It's also not clear whether it's allowed to overwrite the default RandomSource supplied with std similarly to GlobalAlloc.

[joboet]

Would you consider rust-lang/libs-team#159 to be a better solution? That one used the Read trait to fulfil everything you mention.

[newpavlov]

No, I don't think it's an appropriate solution. Firstly, it relies on io::Error, while IIUC you intend for this API to be available in core. Secondly, it does not provide methods for generation of u32 and u64. As I wrote, going through the byte interface is not always efficient. Finally, most of io::Read methods are not relevant here.

For the last point I guess we could define a separate DefaultInsecureRandomSource type.

[bstrie]

Firstly, it relies on io::Error, while IIUC you intend for this API to be available in core.

I don't think this needs to be a blocker. IMO a lot of std::io should be moved into core--not the OS-specific implementations obviously, but all the cross-platform things like type definitions, same as what happened with core::net.

[wwylele]

fn random(source: &mut (impl RandomSource + ?Sized)) -> Self;

Would it be better to make this explicit with generics? Like fn random<R: RandomSource + ?Sized>(source: &mut R) -> Self;. This gives user the ability to specify the type name when needed.

[ericlagergren]

I think it's important for RandomSource methods to properly return potential errors.

I (mostly) disagree. CSPRNGs should almost never fail. When they do, users are almost never not qualified to diagnose the problem.

For example: golang/go#66821

A compromise is something like this:

trait RandomSource {
    type Error;
    fn fill_bytes(...) {
        self.try_fill_bytes(...).unwrap();
    }
    fn try_fill_bytes(...) -> Result<..., Self::Error>
}

This allows most CSPRNGs to use Error = Infallible, but still has support for weird HSMs, etc.

[newpavlov]

@ericlagergren
I've assumed this trait is for a "system" RNG, which will work together with a #[global_allocator]-like way to register implementation. I don't think that we need a general RNG trait in std/core as I wrote in this comment.

As for design of fallible RNG traits, see the new rand_core crate.

[dhardy]

pub trait Random {
    fn random(source: &mut (impl RandomSource + ?Sized)) -> Self;
}

This trait (and the topic of random value generation) should be removed from this discussion entirely in my opinion, focussing only on "secure random data generation" as in the title. Why: because (1) provision of secure random data is an important topic by itself (with many users only wanting a byte slice and with methods like from_ne_bytes already providing safe conversion) and (2) because random value generation is a whole other topic (including uniform-ranged samples and much more).

Disclaimer: I am one of the rand developers. rand originally had a similar trait which got removed; the closest surviving equivalent is StandardUniform.

[hanna-kruppe]

@newpavlov

It's also not clear whether it's allowed to overwrite the default RandomSource supplied with std similarly to GlobalAlloc.

Overriding the default source in an application that already has one from linking std seems questionable. It's not that I can't imagine any use case for it, but the established pattern for such overrides allows any crate in the dependency graph to do it (it's only an error if you link two such crates), instead of putting the leaf binary/cdylib/staticlib artifact in charge. As you articulated in the context of getrandom, that's a security risk for applications. So a RandomSource equivalent should probably be more restrictive in who can override it, but that's not the existing pattern. It also doesn't seem to fit with the proposed generalization of that pattern via "externally implementable functions" (rust-lang/rfcs#3632) -- if that RFC is accepted, any new API surface should use it instead of adding new one-off mechanisms.

If overriding the std source isn't supported, then it could work the same way as #[panic_handler]: you must supply an implementation if you don't link std, but if you do link std then supplying your own is an error. This would still be extremely useful. Currently, every crate that's (optionally) no_std and needs some randomness, most commonly for seeding Hashers, has to cobble together some sub-par ad-hoc solution to try and get some entropy from somewhere. There's a bunch of partial solutions that are better than nothing (const-random, taking addresses of global/local variables and praying that there's some ASLR, a global counter when atomics are available, cfg-gated access to target-specific sources like CPU cycle counters or x86 RDRAND) but:

1. How well this works ends up highly platform-specific, in particular none of them work well for wasm32-unknown-unknown and wasm32v1-none targets .
2. Applications that have access to a better source of entropy and (directly or transitively) use such libraries don't have a good way to enumerate them and make all of them use the better source.

This wouldn't be a problem if the entire ecosystem could agree to always delegate this problem to on one specific crate (version) with appropriate hooks, like getrandom, but evidently that's not happening. Putting this capability into core (or a new no_std sysroot crate, comparable to alloc) has a better chance of solving this coordination problem. Well, at least eventually, once everyone's MSRV has caught up.

Edit: almost forgot that even std::collections::Hash{Map,Set} depend on having a source of random seeds. A way to supply such a source without linking std could help with moving those types to alloc, although as #27242 (comment) points out, it's not backwards compatible to make such a source mandatory for no_std + alloc applications.

[newpavlov]

@hanna-kruppe

Overriding the default source in an application that already has one from linking std seems questionable.

There is a number of reasons to allow overriding:

1. An alternative interface may be more efficient than the default one (e.g. reading the RNDR register vs doing syscall)
2. It may help reduce binary size and eliminate potentially problematic fallback paths (e.g. if you know that you do not need the file fallback on Linux)
3. In some cases it's useful to eliminate non-deterministic inputs (testing, fuzzing)

So a RandomSource equivalent should probably be more restrictive in who can override it, but that's not the existing pattern.

Yes. How about following the getrandom path and allow override only when a special configuration flag is passed to the compiler?

Either way, overriding is probably can be left for later. I think we both agree that we need a way to expose "system" entropy source in std and a way to define this source for std-less targets.

It also doesn't seem to fit with the proposed generalization of that pattern via "externally implementable functions" -- if that RFC is accepted, any new API surface should use it instead of adding new one-off mechanisms.

I agree that ideally we need a unified approach for this kind of problem. I made a similar proposal once upon a time.

But I think it fits fine? Targets with std could implicitly use std_random_impl crate for "external implementation" of the getrandom-like functions and users will be able to override it in application crates if necessary.

How well this works ends up highly platform-specific, in particular none of them work well for wasm32-unknown-unknown and wasm32v1-none targets .

I believe that having std for wasm32-unknown-unknown was a big mistake in the first place and the wasm32v1-none target is a good step in the direction of amending it. So I hope we will not give too much attention to its special circumstances.

This wouldn't be a problem if the entire ecosystem could agree to always delegate this problem to on one specific crate (version) with appropriate hooks, like getrandom, but evidently that's not happening.

Well, it has happened, sort of. getrandom is reasonably popular in the ecosystem even after excluding rand users.

The problem is that std already effectively includes its variant of getrandom for HashMap seeding and people reasonably want to get access to that. And I think problem of getting "system" entropy is fundamental enough for having it in std (well, not in the std per se, let's say in the sysroot crate set).

A way to supply such a source without linking std could help with moving those types to alloc, although as #27242 (comment) points out, it's not backwards compatible to make such a source mandatory for no_std + alloc applications.

Can we add yet another sysroot crate for HashMap which will depend on both alloc and the hypothetical "system entropy" crate?

[hanna-kruppe]

But I think it fits fine? Targets with std could implicitly use std_random_impl crate for "external implementation" of the getrandom-like functions and users will be able to override it in application crates if necessary.

The RFC (and the competing ones I've looked at) only supports a default implementation in the crate that "declares" the externally-implementable thing. If that crate isn't std, then an implementation from std would not count as "default" but conflict with any other definition. So we'd need another special carve-out for std (the very thing we'd want to avoid by adding a general language feature), or the language feature needs to become much more general to support overrideable default implementations from another source.

I believe that having std for wasm32-unknown-unknown was a big mistake in the first place and the wasm32v1-none target is a good step in the direction of amending it. So I hope we will not give too much attention to its special circumstances.

I was specifically talking about no_std libraries, for which the two targets are basically equivalent. Both don't have any source of entropy implied by the target tuple (instruction set, OS, env, etc.), and if you want to add one it'll have to involve whatever application-specific interface the wasm module has with its host.

Well, it has happened, sort of. getrandom is reasonably popular in the ecosystem even after excluding rand users.

Not to point any fingers but a counter example that's fresh on my mind because I looked at its code recently is foldhash. As another example, ahash only uses getrandom optionally (though it's on by default). If you're only using ahash indirectly through another library that disables the feature, then it's not gonna use getrandom unless you happen to notice this and add a direct dependency to enable the feature. In that case there is a solution, at least, but it's still not discoverable.

Can we add yet another sysroot crate for HashMap which will depend on both alloc and the hypothetical "system entropy" crate?

Possibly, but people may object to a proliferation of sysroot crates so let's hope there's a better solution.

[newpavlov]

or the language feature needs to become much more general to support overrideable default implementations from another source.

Yes, it's not as if RFC is a technical specification which must be followed word-by-word. There is a number of cases where the original RFC vision has somewhat changed during implementation stages. If anything, I would say it's an oversight/deficiency of the RFC to not cover cases like this.

foldhash

If a crate aims to minimize its number of dependencies as far as possible even at the cost of code quality and security, it obviously will not depend on getrandom, despite it being the de facto standard for getting system entropy. I think most people will agree that hacks like this have a strong smell. The same applies to fastrand (amusingly, it still uses getrandom for Web WASM) and other "partial solutions" listed by you. As you can notice, both hack their way into using std to get system entropy and either use a fixed seed or pile even more hacks when std is not available.

[hanna-kruppe]

As I said, I have no intention of pointing fingers at any crates. They have to navigate tricky trade-offs and complexities due to Rust's standard library (as a whole, not just std) not yet providing any entropy access. Let's keep this issue focused on changing that.

[workingjubilee]

The Random trait seems weakly motivated in terms of coupling it to RandomSource, as its design seems like it will be a much more hotly contested space, and it is (mostly) unrelated to RandomSource.

[theemathas]

Note that @dhardy (maintainer of rand) wrote some criticism of this at rust-lang/libs-team#393 (comment)

[abgros]

Could Random be implemented for arrays? That way, we could write something like:

let random_array: [u64; 100] = random();

which is a lot more convenient than using the fill_bytes() method.

[sorairolake]

I think it would be useful to have a data type like rand::rngs::mock::StepRng to represent a source of randomness. This is useful for testing whether Random for an arbitrary data type T is implemented as expected.

[dhardy]

I think it would be useful to have a data type like rand::rngs::mock::StepRng to represent a source of randomness. This is useful for testing whether Random for an arbitrary data type T is implemented as expected.

I disagree that it is useful in testing implementations of traits like Random, since there should not be any implied restrictions on such a distribution except that outputs are uniformly distributed, given that the outputs of the random source are truly random. Testing the distribution of output values requires statistical tools, e.g. the KS test as we have here.

[hanna-kruppe]

As far as I know there’s no plan to make the RandomSource trait sealed, so other implementations can be provided by third party crates without having to debate whether it’s a good idea for std to include them.

[jstarks]

It seems like one might regret adding Random later--for some times (e.g., u32), there's a natural definition and implementation that produces a uniform distribution, but it happens to correspond to using RandomSource with something like zerocopy/bytemuck/the safe transmute effort. I.e., there's already a reasonable solution for these types.

For more complicated types (e.g., f32 or (T, U)), Random is under-defined. It is unclear what distribution is expected, and without implementations on tuples or a derive for enums/structs, there's really no "reference implementation" for third-party code to model.

It seems better to leave this out rather than leave it in this poorly specified state. Just stabilize RandomSource.

Although... even RandomSource is lacking, in that it doesn't support writing to an uninitialized buffer... Would it be better to wait until this problem is solved in Read (#78485) before committing to this new interface? Adding BorrowedBuf support later will be a headache.

[hanna-kruppe]

Would it be better to wait until this problem is solved in Read (#78485) before committing to this new interface?

I really don't think so. The current interface works perfectly fine, it's just slightly suboptimal in some use cases (fewer than in the context of general byte-centric I/O). Providing only a Read::read_buf-style API would be strictly worse for many common use cases, in that it would be more complicated to use for no benefit. Usually you either want to generate one conveniently-sized integer at a time (probably should have a dedicated trait method for performance reasons), or you want to fill a small fixed-size buffer completely (e.g., cryptographic key material). Scenarios where the cost of buffer initialization is significant, e.g. preparing a large ephemeral buffer but only filling a small part of it before discarding it, are rare.

So I think it's pretty clear that the method for filling a &mut [u8] should exist even if io::Read had already arrived at a stable solution for reading into uninitialized buffers. Adding the latter after stabilizing the former does pose some challenges, but io::Read already has to solve similar challenges -- more of them, actually, because it also has to work well for several layers of I/O adapters, while PRNGs are rarely composed in a way that stresses the performance of data flowing through several layers of composition.

[Kixunil]

@hanna-kruppe

Perhaps something like this would work:

// Users don't need to concern themselves with this type.
pub strut OutBuf([MaybeUninit<u8>]);

impl OutBuf {
    // various methods to construct it and write to it, no method to read from it
}

// Users don't need to worry about implementing this, just use it
pub unsafe trait AsOutBuf {
    type Init: ?Sized;

    fn as_out_buf(&mut self) -> &mut OutBuf;
    // Only allowed if the entire buffer was overwritten.
    unsafe fn assume_init(&mut self) -> &mut Self::Init;
}

impl AsOutBuf for [u8] { type Init = [u8]; ... }
impl AsOutBuf for [MaybeUninit<u8>] { type Init = [u8]; ... }
impl<const N: usize> AsOutBuf for [u8; N] { type Init = [u8; N]; ... }
impl<const N: usize> AsOutBuf for [MaybeUninit<u8>; N] { type Init = [u8; N]; ... }

// Users can pass in any byte slice, byte array or their uninitialized counterparts
// The entire value is guaranteed to get overwritten, so if using the returned value is undesirable due to borrowing issues the users can simply use the original value safely, if it was an initialized type to begin with or they may just soundly call `assume_init` on it.
pub fn fill_random_bytes<T: AsOutBuf>(buf: &mut T) -> &mut T::Init { ... }

This way it already supports all reasonable cases and people are not forced into using uninitialized API. They can write let mut buf = [0; 32]; fill_random_bytes(&mut buf); exactly as if they were using the simple API or calling Read::read_exact.

[hanna-kruppe]

That helps the simple call sites remain simple, but it has several downsides:

1. The more complicated signature makes it less obvious that it can be used like that for simple cases, both for users who aren't reading the rustdoc page from start to finish, and also to the compiler's type inference in some cases.
2. The reliance on generics means RandomSource is no longer dyn-compatible. This could be fixed by gating the AsOutBuf-using methods on Self: Sized and providing other methods that are dyn-compatible, but at that point you might as well provide two concrete methods for the initialized and possibly-uninitialized case.
3. The new abstractions compete with Borrowed{Buf,Cursor} and other proposed APIs for "writing into a possibly-uninitialized buffer". std should minimize the number of subtly different and incompatible abstractions for the same problem space, especially if it's something third party libraries would want to use as vocabulary type/trait. Going in this direction would give more reasons to block stabilization of secure random data generation on other APIs that have been in limbo for a long time.
4. Even if we ignore the previous point and focus only on the narrow use case of RandomSource, there's considerable design space to be explored there. For example, a downside of the specific approach you outlined is that it forces every RandomSource implementation to use unsafe internally while also requiring every caller who wants to use the MaybeUninit capabilities to use another bit of unsafe justified by "the implementation is correct" (and we'd have to make RandomSource unsafe to implement to make this sound). In contrast, the more complicated interface of Borrowed{Buf,Cursor} provides a bunch of useful functionality as safe APIs.

[jstarks]

The only thing that rand_fill_buffer is "intended for" is the PRNG initialization, or other similar task like starting up a HashMap.

The ACP says this is for "secure" random number generation. I take that to mean this interface is for cases where you need a truly unpredictable value, e.g., an encryption key or some kind of secret token. This is something only the platform can provide, these days, especially if you want protection against VM snapshots and process forks and things like that.

There are really no design decisions here (other than this &mut [u8] vs BorrowedBuf thing)--in particular, you really don't want to use a kernel source to seed a user-mode PRNG for cryptographic use cases, please never do this. You just want to use the primitive the vendor provides.

And so it seems like a good idea to expose this from std, so that users are most likely to get the "right" implementation for their platform by default.

But echoing some comments above, it does feel like this proposal is attempting to support usage for randomized algorithms as well. But there are already well-supported crates in the ecosystem for this, so it's not obvious that this really needs to be in std at this time.

And really the ACP is kind of two-faced about this, as well. It talks about secure random numbers but then uses a bunch of randomized-algorithm like tasks as motivating examples.

But anyway, I see that basically my objections mirror the various rand experts here, so +1 to those.

[Lokathor]

Uh, right? I mean I agree with you that a programmer should not use a normal PRNG for cryptography. That is absolutely not what I was suggesting.

[ericlagergren]

The only thing that rand_fill_buffer is "intended for" is the PRNG initialization, or other similar task like starting up a HashMap.

It's normal (and expected) to read cryptographic keys straight from the system CSPRNG.

[Lokathor]

To me that would fall under "similar task", yeah.

[sorairolake]

The necessity of the Random trait is debated, but I think it's OK to implement this trait for at least the following types:

• Primitive integer types and bool
• Arrays and tuples of the above values
• NonZero<T>, Saturating<T> and Wrapping<T>

Even if we are to implement this trait for floating-point types, string types, or char, I think sufficient discussion would be needed in advance.

I also think it might be a good idea to add the Sealed trait to the trait bound of this trait. Because we can use the Distribution trait of the rand crate for this purpose. If the rand crate does not exist or if the random module is intended to replace the rand crate, it would make sense to make it possible to implement it in external crates, but since this is not the case, I don't think there is any need to do so.

If we want an implementation of this trait for floating-point types, we can do so by defining struct Double(f64) in external crates and implementing this trait for struct Double(f64). But I don't think it's necessary to allow this with this trait provided by the standard library. I think it would be better to be guaranteed that this trait is only implemented for types that are less controversial or well-discussed.

[oxalica]

The necessity of the Random trait is debated, but I think it's OK to implement this trait for at least the following types:
* Primitive integer types and bool
* Arrays and tuples of the above values
* NonZero<T>, Saturating<T> and Wrapping<T>

I don't think so. If there is a limited list, there will be more and more questions about "why not also Y". People can also underestimate the complexity here, like, how can you generate a uniform NonZero<u8> in a constant time upper bound? It's not as easy as you think. Same thing applies to char. Floating point types make it even worse as they cannot be "uniform" and we'll have different distribution on different types.

If all we want is to have a way to get random CSPRNG bits from OS, we should stick on getting bytes, and let downstream to implement these "shortcut traits and methods" for their various requirements (someone thinks random() % range is good enough, but someone does not).

[abgros]

People can also underestimate the complexity here, like, how can you generate a uniform NonZero<u8> in a constant time upper bound? It's not as easy as you think.

"Not as easy you think" is an understatement. The task is literally impossible, because your RNG could continuously generate zeros for an arbitrary amount of time. In this case your only choice is to map zero to some value, thus making the distribution no longer uniform.

Floating point types make it even worse as they cannot be "uniform" and we'll have different distribution on different types.

You could generate a uniform random float by just choosing random bits, it's just that such an operation is completely useless in practice (you would get a bunch of big numbers, tiny numbers, and NaNs). People generally want a mathematical distribution like the standard uniform or normal distributions which I think is out of scope of this issue.

Generating a random char is equally useless because you'll get mostly undefined characters as well a few Chinese characters and maybe some weird stuff like U+2069 POP DIRECTIONAL ISOLATE.

I would recommend implementing Random for any type in which the number of "useful" possibilities is some power of two because that way it can be done in a zero-cost way.

[sorairolake]

I removed the implementation of Random for NonZero<T> from #136733.

[ericlagergren]

"Not as easy you think" is an understatement. The task is literally impossible, because your RNG could continuously generate zeros for an arbitrary amount of time.

"An arbitrary amount of time" is a bit of an overstatement. The probability of a CSPRNG generating N consecutive zero bits is 1/(2^N). Rejection sampling is probably fast enough, but you could also over sample to prevent more than one call to the CSPRNG. Either way, you know what the realistic upper bound is.

In this case your only choice is to map zero to some value, thus making the distribution no longer uniform.

Or sample extra bits.

[Kixunil]

Regarding errors, this is a simple approach:

pub trait RandomSource {
    // CSPRNG impls can simply set this to Infallible
    type Error;
    // ignoring uninit for simplicity
    fn fill_bytes(&mut self, bytes: &mut [u8]) -> Result<(), Self::Error>;

    /// Returns an adaptor converting all the errors.
    ///
    /// This is useful when one needs to unify multiple different errors (e.g. by boxing them) so that the trait can be used in dynamic dispatch.
    fn map_err<E, F: FnMut(Self::Error) -> E>(self, f: F) -> MapErr<Self, F> where Self: Sized { /* ... */ }
}

The great thing about this is std can simply return io::Error with no risk of affecting no_std.

[dhardy]

In light of the proposed "quick and dirty" solution, I want to emphasize Artyom's point, which I've also made elsewhere:

Tying the interface to io::Error can be too restrictive. Firstly, some platforms "guarantee" that they do not return errors on entropy generation.

CSPRNGs should not fail, and more implementations are moving in this direction. Returning an error like io::Error means that callers have to:

@ericlagergren this is why in rand_core we now have both the fallible TryRngCore and the infallible RngCore. The only fallible RNG we support directly (i.e. which is not an infallible RNG) is OsRng, the system interface.

I just don't like having several ways of doing the same thing. And free-standing functions have a certain (C) smell in my opinion. It's like in addition to Vec::new() we had a "convenience" functions like vec() or vec_from_slice(buf: &[T]).

@newpavlov I'm saying we definitely want a fallible interface for the system random source. Whether we also want to support other potentially fallible random sources in std I don't know, but I suspect we can keep it simple and not do this...

... except, we may want both secure and potentially-insecure external random sources. hash_map::RandomState already uses the latter to avoid blocking if somehow it is used in an early boot environment.

So this is a design decision: use a simple try_fill_bytes method or use something like TryRngCore over two sources:

• SecureRandomSource
• InsecureRandomSource — same as above except that it doesn't block in early boot / when the OS does not yet have enough entropy for secure byte output

The necessity of the Random trait is debated

@sorairolake can we just not talk about this for now? Or make a new issue for it?

The same goes for infallible random sources. We may want a trait like rand_core::RngCore for that (depending on whether or not we decide to include random functionality aimed at the end user within std). But it is a different problem to the above problem of getting random data from the OS through a potentially fallible API, precisely because (as @ericlagergren pointed out) we want an infallible interface for the latter.

[Kixunil]

this is why in rand_core we now have both the fallible TryRngCore and the infallible RngCore.

Why two traits instead of setting the associated error type to infallible/io::Error depending on the source?

[dhardy]

this is why in rand_core we now have both the fallible TryRngCore and the infallible RngCore.

Why two traits instead of setting the associated error type to infallible/io::Error depending on the source?

Do you want to write fn foo<R: TryRngCore<Error = Infallible>>(rng: &mut R) and then unwrap() all results (while knowing that the unwrap() can never panic)?

Much simpler to write fn foo<R: RngCore>(rng: &mut R).

Other than this (and the dyn equivalent where Error must be specified)... yes, it would be possible.

[newpavlov]

I'm saying we definitely want a fallible interface for the system random source.

I agree. In my comment I wrote that the free standing function may simply panic on potential errors to side-step the error type issue. Right now HashMap already panics in such cases, same with ThreadRng, so it may be acceptable. Users who would like to properly process potential errors would use the fallible RNG traits.

As I wrote above, I would prefer to have at least 3 sources exposed in std: SecureRandomSource, InsecureRandomSource, and ThreadRng. Though I think the first two are better named as SysRng and InsecureSysRng. Sys is used instead of Os because in future these sources may be exposed on no_std targets with user-provided implementations and "OS" would be confusing in such contexts.

As for the trait design, one potential alternative to what we have today in rand_core could be this:

pub trait TryRng {
    type Error: core::error::Error = Infallible;
    fn try_next_u32(&mut self) -> Result<u32, Self::Error>;
    fn try_next_u64(&mut self) -> Result<u64, Self::Error>;
    fn try_fill_bytes(&mut self, buf: &mut [u8]) -> Result<(), Self::Error>;
}

pub trait Rng: TryRng<Error = Infallible> {
    fn next_u32(&mut self) -> u32;
    fn next_u64(&mut self) -> u64;
    fn fill_bytes(&mut self, buf: &mut [u8]);
}

impl<T: TryRng<Error = Infallible>> Rng for T {
    fn next_u32(&mut self) -> u32 {
        let Ok(val) = self.try_next_u32();
        val
    }
    fn next_u64(&mut self) -> u64 {
        let Ok(val) = self.try_next_u64();
        val
    }
    fn fill_bytes(&mut self, buf: &mut [u8]) {
        let Ok(()) = self.try_fill_bytes(buf);
    }
}

In other words, all RNGs would implement the TryRng trait while Rng will be a simple extension trait. Alternatively, the traits could be named as Rng and InfallibleRng respectively.

[Kixunil]

Do you want to write fn foo<R: TryRngCore<Error = Infallible>>(rng: &mut R)

I most likely want to just pass the error to the caller and keep my API flexible. If this is for some reason not suitable, I'd be fine with that, and once trait aliases are a thing you can have your cake and eat it too.

and then unwrap() all results (while knowing that the unwrap() can never panic)?

No, in today's stable Rust I write .unwrap_or_else(|never| match never {}) to convey "never fails". There's an unstable method into_ok on Result perhaps this could motivate stabilizing it. You could also have the fallible method called try_fill_bytes and a helper fn fill_bytes(&mut self, buf: &mut [u8]) where Self::Error = ! (though I'm not sure if that bound is trivially expressible).

Much simpler to write fn foo<R: RngCore>(rng: &mut R).

Yes but also it's less flexible. You are committing to not support fallible RNGs. Whether it's a problem or not depends on specific case.

Other than this (and the dyn equivalent where Error must be specified)... yes, it would be possible.

That's good to hear, so would you agree that perhaps with trait aliases and stable Result::into_ok or a helper method this would've been viable option for std?

@newpavlov I think that trait alias might be cleaner but you probably do have a point that infallible RNG implies fallible with Error = Infallible and I think most RNGs will be infallible.

[dhardy]

That's good to hear, so would you agree that perhaps with trait aliases and stable Result::into_ok or a helper method this would've been viable option for std?

What applications are we talking about? For a system random source that is only used for a few low-level applications (like seeding a local PRNG), requiring users be pedantic is fine. But if we're talking about replacing rand::rngs::ThreadRng for general usage, then no, this is too verbose. @newpavlov's proposal is better for that.

ThreadRng

I don't know if we want to call it that, but yes, we might want that too. I was assuming this would be added later, but it may make sense to add now too (but possibly stabilise later). The main concern I have is what implementation would make the most sense: something complex like rand::rngs::ThreadRng or something much simpler like OsRng (or possibly just a cache over OsRng)... but this decision should be made later (in a new issue) in my opinion.

[jstarks]

I'm saying we definitely want a fallible interface for the system random source.

Why?

[briansmith]

Not all system RNGs are infallible. Windows < 10 RNG isn't infallible. Most OS RNGs are not infallible, in fact--especially ones that get FIPS-ified. Anybody can unwrap() an error if they want an infallible interface. Not everybody can use an API that abort()s or panics, as some coding standards do not allow any abort() or (reachable) panic at all.

(It is true that people designing system RNGs should aim for them to be as close to infallible as they can get.)

BTW, is the "system" RNG really the system RNG, or is it sometimes a libc-provided thing, like is common in BSD? I would love to see an interface that skips completely over libc whenever practical.

[jstarks]

Windows < 10 RNG isn't infallible.

Is it actually fallible, or is it just not documented to be infallible?

My view is that the failure mode of thinking you got cryptographic random data in a buffer but you didn't is so bad that we should not support that possibility in std. If you are running in an environment where you can't abort the process on critical errors, then just don't use this interface, just like you can't use Box::new.

[programmerjake]

• InsecureRandomSource — same as above except that it doesn't block in early boot / when the OS does not yet have enough entropy for secure byte output

I think we specifically need it to also possibly be deterministic (or have an alternative interface for hashmaps), since we may want our program to be 100% deterministic for things like synchronously stepping game state, reproducibility (e.g. so everyone agrees on the output of a particular transaction in a cryptocurrency, or for reproducible builds), and similar. this is an explicit goal of WASI's random interface, where a deterministic WASM engine will only implement insecure random and always returns the same sequence every time it's run (well, technically it just provides a constant seed value and lets the WASM code build a RNG from that).

[ericlagergren]

Not all system RNGs are infallible. Windows < 10 RNG isn't infallible. Most OS RNGs are not infallible, in fact--especially ones that get FIPS-ified. Anybody can unwrap() an error if they want an infallible interface. Not everybody can use an API that abort()s or panics, as some coding standards do not allow any abort() or (reachable) panic at all.

(It is true that people designing system RNGs should aim for them to be as close to infallible as they can get.)

The problem is that users will fall back to insecure RNGs, which is catastrophic. And if your CSPRNG fails—then what? Outside of very niche situations there isn't anything you can do about it.

BTW, is the "system" RNG really the system RNG, or is it sometimes a libc-provided thing, like is common in BSD? I would love to see an interface that skips completely over libc whenever practical.

Very much in agreement with this.

[briansmith]

Is it actually fallible, or is it just not documented to be infallible?

For people subject to strict (external) requirements, it's a distinction without a difference because we can't rely on undocumented behavior without a lot of work. My understanding is that Linux getrandom() fails in cases where it shouldn't because some distros (not mainstream ones) patch it. getrandom() also frequently fails because of sandboxing issues.

[jstarks]

For people subject to strict (external) requirements, it's a distinction without a difference because we can't rely on undocumented behavior without a lot of work.

Sure. But I work for Microsoft. I can probably get documentation corrected if it's really a blocker. I got ProcessPrng documented a few years ago after some cajoling.

If there are issues with other OSes, well, I can't help with that :).

[briansmith]

Sure. But I work for Microsoft. I can probably get documentation corrected if it's really a blocker. I got ProcessPrng documented a few years ago after some cajoling.

Yes, that would be great, esp for Windows 7 and Windows 10. I would also love better documentation for SystemPrng for the case where I really want to bypass any userspace PRNG.

[jstarks]

I would also love better documentation for SystemPrng for the case where I really want to bypass any userspace PRNG.

There is no documented facility to bypass userspace PRNG in Windows. I'm not sure why you'd want to, given that it already has rekey logic--it should be no more vulnerable to VM snapshots or similar than the kernel RNG.

[sorairolake]

@dhardy

@sorairolake can we just not talk about this for now? Or make a new issue for it?

Yes, I don't need to talk about it for now.

[Kixunil]

My view is that the failure mode of thinking you got cryptographic random data in a buffer but you didn't is so bad that we should not support that possibility in std.

Unused Result already produces warnings but funnily, this can be enforced more strongly by providing uninit interface only.

[workingjubilee]

My view is that the failure mode of thinking you got cryptographic random data in a buffer but you didn't is so bad that we should not support that possibility in std. If you are running in an environment where you can't abort the process on critical errors, then just don't use this interface, just like you can't use Box::new.

There is an alternative: a function that returns a Result but in a by-value array.

Of course then you'll have people screaming about how it's "inefficient" because they have to copy a few bytes once, despite the entire point being that you would use it to obtain a seed and not repeatedly hammer it to create large quantities of entropy.

[Kixunil]

Of course then you'll have people screaming about how it's "inefficient" because they have to copy a few bytes once, despite the entire point being that you would use it to obtain a seed and not repeatedly hammer it to create large quantities of entropy.

There are ways to mitigate it or solve it completely (at the cost of very weird API). If you take in MaybeUninit slice/array and return &mut [u8] (or array) the cost is already down to needing a register to return the pointer and perhaps some mov instructions. There could also be the public function with #[inline] and that signature calling into an internal function followed by assume_init - after inlining the optimizer sees it's the same pointer and doesn't waste registers. Or if you don't want to rely on inlining, there's a crazy trick with returning ZST Token<'a> with invariant lifetime which is required to access the bytes in the buffer. But I think that's simply too much.