x/crypto/ssh: Dial function with ClientConfig Timeout got stuck

[lindmin]

What version of Go are you using (go version)?

go version go1.16.1 linux/amd64

Does this issue reproduce with the latest release?

I dont know.

What operating system and processor architecture are you using (go env)?

go env Output

O111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bapt/.cache/go-build"
GOENV="/home/bapt/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/bapt/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/bapt/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.1"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build576072964=/tmp/go-build -gno-record-gcc-switches"

What did you do?

trying to connect to ssh server that is currently not available with a timeout of 5 minutes.

What did you expect to see?

timeout of the connection attempt after the 5 minutes delay is exceeded.

What did you see instead?

the goroutine got stuck.

here an extract of the core dump:

goroutine 342 [IO wait, 196 minutes]:
internal/poll.runtime_pollWait(0x7f88d35586f0, 0x72, 0xffffffffffffffff)
	/usr/local/go/src/runtime/netpoll.go:222 +0x55
internal/poll.(*pollDesc).wait(0xc0003fbf98, 0x72, 0x0, 0x1, 0xffffffffffffffff)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0x45
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0xc0003fbf80, 0xc0004da30a, 0x1, 0x1, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:166 +0x1d5
net.(*netFD).Read(0xc0003fbf80, 0xc0004da30a, 0x1, 0x1, 0x0, 0x0, 0x8)
	/usr/local/go/src/net/fd_posix.go:55 +0x4f
net.(*conn).Read(0xc000508098, 0xc0004da30a, 0x1, 0x1, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:183 +0x91
io.ReadAtLeast(0xbf4e60, 0xc000508098, 0xc0004da30a, 0x1, 0x1, 0x1, 0xc0003fbf80, 0xc000510150, 0xc)
	/usr/local/go/src/io/io.go:328 +0x87
io.ReadFull(...)
	/usr/local/go/src/io/io.go:347
golang.org/x/crypto/ssh.readVersion(0xbf4e60, 0xc000508098, 0xc000508098, 0xbf4e60, 0xc000508098, 0x0, 0x0)
	/go/pkg/mod/golang.org/x/[email protected]/ssh/transport.go:317 +0x105
golang.org/x/crypto/ssh.exchangeVersions(0x7f88d3599e18, 0xc000508098, 0xc0004da300, 0xa, 0xa, 0xc000682100, 0x4, 0xc000402800, 0x0, 0xc00062bfe0)
	/go/pkg/mod/golang.org/x/[email protected]/ssh/transport.go:301 +0x15d
golang.org/x/crypto/ssh.(*connection).clientHandshake(0xc000682100, 0xc000680bd0, 0x2a, 0xc0003f6680, 0x0, 0x0)
	/go/pkg/mod/golang.org/x/[email protected]/ssh/client.go:100 +0xf2
golang.org/x/crypto/ssh.NewClientConn(0xc06048, 0xc000508098, 0xc000680bd0, 0x2a, 0xc000416f70, 0xc06048, 0xc000508098, 0x0, 0x0, 0xc000680bd0, ...)
	/go/pkg/mod/golang.org/x/[email protected]/ssh/client.go:83 +0xf8
golang.org/x/crypto/ssh.Dial(0xb33f34, 0x3, 0xc000680bd0, 0x2a, 0xc000416f70, 0xb5ae65, 0x2, 0xc000680bd0)
	/go/pkg/mod/golang.org/x/[email protected]/ssh/client.go:177 +0xb8
personal/worker/executors/executionexecutor.NewSSHExecutor.func1(0xc000279a8b, 0x27, 0x16, 0xc000416f70, 0xc00025d460, 0x0, 0x21, 0x10)

In my case, the ssh port of the server was accessible (telnet OK). But, due to cpu overload, the ssh connection (with ssh command) did not succeed.

In the core dump, it seems it got stuck in the readVersion function.

I think the Timeout property of ssh.ClientConfig pass to the Dial function should be use un the function NewClientConn to prevent this problem.

PS: it is pretty hard to reproduce on my side...

Lindmin.

[davecheney]

You didn't show us how to reproduce the issue. Can you please provide a runnable program that shows the problem. Thank you.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)