C-based pypi package

[ThomasWaldmann]

It would be cool to have a C-based pypi package for the blake3 bindings.

I've read the ticket on the Python issue tracker about hashlib inclusion, but that was closed / rejected (for now) - what a pity.

I guess there are some projects out there (like borgbackup, see borgbackup/borg#45 (comment) ) that would like to add blake3, but not introduce a dependency on the rust toolchain. The latter might be no big issue if you target modern linux dists, but if you try to support a lot of platforms, this at least creates a lot of uncertainties (that block adoption) if not real issues (if one adopts and then stumbles into them).

Not supporting internal multithreading is no issue in our case because if we implement MT, we need to do that on a higher level anyway. The GIL should be freed though while computing a hash, so it doesn't block other stuff for no reason.

[ddelange]

Hi @ThomasWaldmann 👋

fyi, blake3 v1.0 has been released with a stable API, and with lots of wheels to cover a modern audience.

If you're missing a specific wheel there, please give a shout. I don't think the reference C implementation was ever intended for production.

Hope that helps!

[ThomasWaldmann]

Thanks for the pointer. Yeah, that's quite a lot already.

Missing OSes:

• *BSD
• Solaris derivates, like e.g. OpenIndiana

Not sure about the situation on windows. Would such a windows wheel also work e.g. on cygwin?

[ThomasWaldmann]

Another data point:

Some years passed since I opened this ticket and availability of and dependency on rust have changed.

E.g. I recently had to add rust toolchain to my *BSD platform testing machines, because a borg master branch dependency (not blake3) required it. Also, looks like rust is available on OpenIndiana.

[ddelange]

the most relevant issue I found is pydantic/pydantic-core#773 (comment)

PEP 600 has been designed to be "future-proof" and does not enforce specific symbols and a specific distro to build. It only states that a wheel tagged manylinux_x_y shall work on any distro based on glibc>=x.y. PEP 656 added musllinux_x_y tags for musl>=x.y

ref https://github.com/pypa/manylinux

concretely, the blake3 CPython wheels will install on any linux distro that is based on either glibc>=2.17 or musl>=1.1. but you're right that doesn't cover BSD's libc nor solaris' libc

regarding cygwin I think this comment explains why you don't see cygwin wheels in the wild. my bet would be that modern users will either be installing the windows wheels natively (using CPython compiled for windows), or installing the linux wheels inside the WSL (https://superuser.com/questions/1360748).

[oconnor663]

I don't think the reference C implementation was ever intended for production.

The main purpose of the C implementation was as a proof-of-concept for including BLAKE3 in the Python standard library (which already has BLAKE2). If I remember correctly, they ultimately decided they didn't want to take it until there was OpenSSL support.