archive/tar: mode header field contains incorrect bits

[Larsjep]

When using the FileInfoHeader to create a tar file header the mode field contains incorrect bits.

According to the GNU tar spec:
https://www.gnu.org/software/tar/manual/html_node/Standard.html
and POSIX:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/tar.h.html

The mode field should only contain the permission bits (bit 0-11)

But the FileInfoHeader also set bit 15 for regular files and bit 14 for directories.
And properly also other bits for other types of files.

What version of Go are you using (go version)?

go version go1.8.1 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/lje/work/golang"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build824361864=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
PKG_CONFIG="pkg-config"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"

[dsnet]

I looked through the history of archive/tar and it seems that these were added in https://golang.org/cl/5796073 with the comment that these came from "the tar spec", but neither of the two links in the package doc matches the mode constants currently used in Go.

I looked at GNU's implementation of tar and they do not have any bit modes defined larger than 07777.
I looked at BSD's implementation of tar and they do have bit modes defined larger than 07777. When writing a tar file, they always mask with 07777, so it never writes the larger bit modes, but seems to be able to understand them when reading.

I believe there is sufficient evidence to remove setting these high bits when using FileInfoHeader. However, I believe the Reader and Writer should continue handling higher-order bits if a user specifically sets them.

Just wondering, how were the setting of these high-order bits affecting your use-case?

[Larsjep]

I discovered the problem while investigating a problem related to this issue in docker.py
docker/docker-py#998

I found out that the root cause was that these mode bit was set. When docker compares files from different tar archives, in order to determine if a cache version can be used, it uses all the bits.

Of course this issue can be fixed by changing the docker implementation, but I think the best solution is to fix it in archive/tar.

[gopherbot]

CL https://golang.org/cl/42093 mentions this issue.