ToyBox

[jart]

One thing I always found peculiar is that Bazel is theoretically hermetically sealed, but I can still run programs from /bin like it's not a thing, and the fact that they'll be different on different platforms.

Has the Bazel team considered simply shipping Bazel with BusyBox baked in so shell scripts will be guaranteed to work the exact same way across platforms? It's a few megabytes in size and provides a lightweight implementation of all the core unix commands.

Yes... we'll have to give up some of the more exotic features of GNU coreutils. But it'll be genuinely cross platform. After all, you guys are surely doing something similar to this for the Windows port. And to be honest, the Almquist Shell (which is /bin/sh on Ubuntu) does a pretty good job these days supporting nearly all the bash syntax.

[laszlocsomor]

That's a noteworthy idea, thanks! However, while playing around a bit with BusyBox in a VM, I found it supports some of the bash constructs we use (e.g. [[ and ]]) but not others (e.g. ${FUNCNAME[0]} and arrays in general), and since we use these extensively in shell tests, I don't think BusyBox's shell is a viable alternative for us currently.

That said, I don't know if we considered using BusyBox before, @damienmg, @dslomov, @kchodorow ?

On Windows we currently use MSYS2 which provides bash and many if not all common GNU coreutils.

[damienmg]

We never considered it but @philwo was suggesting in his design doc to be able to mount base image in the sandbox, maybe that should be part of that plan.

[dslomov]

On WIndows, I am seriously considering BusyBox but that can only happen after the launcher does not depend on msys.

[pcj]

+1

[jart]

It's a shame that a lot of the shell scripts have been written to use bash features, because ash/dash is 4x faster and uses very limited resources. I personally always try to write my shell scripts to be ash compatible, even though on Google Ubuntu workstations that's difficult, because they symlink sh to bash for some reason.

[damienmg]

/cc @aehlig our resident expert on killing bash-ism :)

[jart]

Actually it turns out BusyBox is GPL. We would have to use ToyBox.

[jart]

I say this at the risk of embarrassing myself, but I've changed my mind regarding the Almquist Shell.

I sat down and tried to write a nontrivial script for a repository_rule() that is ash compatible. I discovered that this is not easy to do without sacrificing correctness.

• The lack of set -o pipefail or $PIPESTATUS forced me into the position of having to either be really clever or use a lot of temporary files to properly catch errors.
• The lack of arrays made it impossible for me to store flags in memory without breaking support for input with spaces.

So my personal preference at this point would most likely be following the Google Shell Style Guide which forbids everything except bash. I guess it evolved to be that way for a reason.

Note: I'm using ash 0.5.7-4ubuntu1

[jart]

I'll also say that I used to write Bourne Shell scripts when I was younger. As in, the Bourne Shell written by Steven Bourne with C source code that looked like ALGOL 68. I figured, oh Almquist, how hard could it possibly be? Except I didn't take into consideration that I had spent the last four years in a Google shell scripting paradise where everything is guaranteed to be a working version of bash. I imagine the same is true for many others. It's probably a better use of eng time if we can guarantee bash.

[ulfjack]

Chrome has gone the other way and completely disallowed shell scripts everywhere, instead forcing everything to use Python.

[jart]

That would be wise too @ulfjack. I was watching @damienmg speak at the Dart Developer Summit and I thought he said something particularly profound about how Skylark is a restrictive subset language that ensures the developer doesn't do things that introduce build badness. Maybe that's the right direction.