additional kex algorithms

[mpet]

Hi
This again add support for kex algorithms:

ecdh-sha2-nistp256
ecdh-sha2-nistp384
ecdh-sha2-nistp521
curve25519-sha256

I have tested all but curve25519-sha256 vs Mina.

I tried to keep it as small as possible. Changes taken from connectbot/sshlib.
I also added some more tests from the same project.

@kuisathaverat I would appreciate if you could do a new check of code.

//mike

[jvz]

This PR looks good to me from a code standpoint. I haven't manually tested this, however.

[mpet]

@jvz

rfc8332 enabled

ssh-dss used for Mina in KexManager.verifySignature()

rsa-sha2-256 used for libssh in KexManager.verifySignature()

Caused by: java.io.IOException: Peer sent wrong signature format
	at com.trilead.ssh2.signature.RSAKeyAlgorithm.decodeSignature(RSAKeyAlgorithm.java:71)
	at com.trilead.ssh2.transport.KexManager.verifySignature(KexManager.java:409)
	at com.trilead.ssh2.transport.KexManager.handleMessage(KexManager.java:671)
	at com.trilead.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:790)
	at com.trilead.ssh2.transport.TransportManager$1.run(TransportManager.java:502)

Public key

Sun RSA public key, 2048 bits
  modulus: 24247994701388530985777874400375834166851168539718260762048203164605437776096320696475128500914043328563334993089615828868976641819889953559375831203517411728220021554958175578958518511241198590332408332822434908128048995032985247538084278757682407106185175350494603932101978596444154007217201318721448069013773378485483977957797219137788870714168001383376978609111302513411200180638471272414779931090516432769864984381287463266050292206124775831716656643873116427835139480465883968282667070080850542771363566882056116655280129893585903557956453119780352142516195528623182365943823079911520036863689881913648549544089
  public exponent: 65537

rfc8332 disabled

ssh-dss used for Mina im KexManager.verifySignature()

ssh-rsa used for libssh in in KexManager.verifySignature() then works ok but tests are failing.

@kuisathaverat I saw you had reverted this before. Did you experience something similar? Solved and how?

[jvz]

Ideally, we'd like to both support RSA SHA-2 as well as deprecating or removing RSA SHA-1 as that's what OpenSSH has been recommending for several months (possibly a couple years by now) as they'll be removing support in a future release (it's currently disabled by default at least). I believe the code added here about extension negotiation may be necessary to ensure the SSH server supports RSA SHA-2 as my previous attempt at supporting this would simply send the SHA-2 signature first and then retry with the SHA-1 signature if it failed, though some SSH servers may be overly paranoid and reject the second attempt (it is allowed by the RFC after all, though it's a shitty response).

[kuisathaverat]

I saw you had reverted this before. Did you experience something similar? Solved and how?

yep, every time we add a new algorithm, those tests have a list of algorithms in order, it should be the same that is returned by the method getPreferredServerHostkeyAlgorithmOrder, and the only way to have the list is to inspect the value is running the tests in debug mode.

[kuisathaverat]

in the meantime, I am checking a prototype of trilead-api-plugin and ssh-slaves-plugin, I have to check why the keys encrypted are not working but all the rest is working.

The only key type does not wor is the RSA (old format ssh-rsa)

[mpet]

@jvz I can try to add extension negotiation if we cannot live with this commit as is.
Which server did you test on ? I only know Apache Mina that has support for https://tools.ietf.org/html/rfc8332
We really need this commit for our testing of libssh server.

@kuisathaverat how shall we proceed? I need 3 algorithms for testing of libssh server.

[mpet]

@jvz I checked and it seems to be quite some changes in to get sha 2 working but I will give it a try.

[mpet]

@jvz I checked code and I cannot see that ext info is used for providing info for SHA2. I can see this for HostKey verification.

if (RSASHA1Verify.ID_SSH_RSA.equals(serverHostKeyAlgorithm) ||
			RSASHA256Verify.ID_RSA_SHA_2_256.equals(serverHostKeyAlgorithm) ||
			RSASHA512Verify.ID_RSA_SHA_2_512.equals(serverHostKeyAlgorithm))
		{
			remoteKey = RSASHA1Verify.get().decodePublicKey(serverHostKey);
		}

So it seems SHA1 is used.
I think we need to to a separate submission for sha2. @kuisathaverat what do you think?

[jvz]

Oops, sorry, I was referring to the SHA-2 support I had introduced in an earlier commit that we had to revert. I had tested that out with OpenSSH, though we still had regression reports (possible that OpenSSH fixed something itself in newer versions to accept that).

In that code, the RSA public key is still the same regardless if it's using SHA-1 or SHA-2; the hash is only used for host key verification (while signatures are usually used for authentication). So that code snippet is correct at least, @mpet.

[mpet]

@jvz so my PR is ok if I fix the test?

[jvz]

From my point of view, yes. Ivan is the maintainer, though, so he'll confirm and merge this when he can. I'm on the Jenkins Security team, so I tend to review or audit cryptography and other security changes in Jenkins things, even when we're not the maintainers of the code (it is used in Jenkins after all!).

[mpet]

@jvz ok. @kuisathaverat what is your opinion?

[mpet]

@kuisathaverat could you make a delivery?

[kuisathaverat]

I will test it this weekend, if you need a binary you can use the incremental https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/trilead-ssh2/build-217-jenkins-27-rc201.14ce3b77aeaa/

[mpet]

@kuisathaverat thanks! Greatly appreciated.

[kuisathaverat]

I've tested that does not break anything, but I did not test if the support for the new kex works, I have to prepare some agents with those kex only enabled.
The pre-release of the trilead-api plugin with these changes can be downloaded from https://repo.jenkins-ci.org/incrementals/org/jenkins-ci/plugins/trilead-api/1.0.13-rc47.df7c5d8a2be7/

Along this week I will try to finish my tests.

[kuisathaverat]

It was quicker than expected, the curve25519-sha256 seems does not work, it does not appear in the list of algorithms

ssh-agent-curve25519-sha256     | debug1: Forked child 1021.
ssh-agent-curve25519-sha256     | debug1: Set /proc/self/oom_score_adj to 0
ssh-agent-curve25519-sha256     | debug1: rexec start in 5 out 5 newsock 5 pipe 7 sock 8
ssh-agent-curve25519-sha256     | debug1: inetd sockets after dupping: 4, 4
ssh-agent-curve25519-sha256     | Connection from 172.19.0.2 port 41248 on 172.19.0.4 port 22 rdomain ""
ssh-agent-curve25519-sha256     | debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
ssh-agent-curve25519-sha256     | debug1: Remote protocol version 2.0, remote software version TrileadSSH2Java_213
ssh-agent-curve25519-sha256     | debug1: no match: TrileadSSH2Java_213
ssh-agent-curve25519-sha256     | debug1: permanently_set_uid: 105/65534 [preauth]
ssh-agent-curve25519-sha256     | debug1: list_hostkey_types: rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519 [preauth]
ssh-agent-curve25519-sha256     | debug1: SSH2_MSG_KEXINIT sent [preauth]
ssh-agent-curve25519-sha256     | debug1: SSH2_MSG_KEXINIT received [preauth]
ssh-agent-curve25519-sha256     | debug1: kex: algorithm: (no match) [preauth]
ssh-agent-curve25519-sha256     | Unable to negotiate with 172.19.0.2 port 41248: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521 [preauth]
ssh-agent-curve25519-sha256     | debug1: do_cleanup [preauth]
ssh-agent-curve25519-sha256     | debug1: monitor_read_log: child log fd closed
ssh-agent-curve25519-sha256     | debug1: do_cleanup
ssh-agent-curve25519-sha256     | debug1: Killing privsep child 1022
ssh-agent-curve25519-sha256     | debug1: audit_event: unhandled event 12
ssh-agent-curve25519-sha256     | debug1: main_sigchld_handler: Child exited

This is my test environment https://github.com/kuisathaverat/jenkins-issues/tree/master/kex-algorithms

[mpet]

@kuisathaverat I fixed the list issue. Could you have a look again? Also which code formatter do you use for project?

[mpet]

@kuisathaverat could you check last commit. Or tell me how to test it?

[mpet]

@kuisathaverat could you look into it now?

[kuisathaverat]

during the week I do not have too much time, Friday I have time for it. If you want to test it you only have to checkout my test environment and make make start, then you can go to http://localhost:8080, login (admin:password), and check all the agents are connected.

https://github.com/kuisathaverat/jenkins-issues/tree/master/kex-algorithms

[mpet]

@kuisathaverat where do I set to use my PR in the testenv?

[kuisathaverat]

you have to replace the trilead-api plugin (https://github.com/kuisathaverat/jenkins-issues/tree/master/kex-algorithms/jenkins/jenkins_home/plugins) with the incremental generated by jenkinsci/trilead-api-plugin#20 this file is loaded on you Jenkins instance when you create the Docker container (make clean start)

If you make changes on this PR, a new incremental would be generated, you have to follow the incrementals link in the GitHub checks and take the incremental number

then you have to modify the incremental version and compile again the trilead-api plugin https://github.com/jenkinsci/trilead-api-plugin/pull/20/files

[mpet]

@kuisathaverat I failed to do it. I create a PR of jenkinsci/trilead-api-plugin#21 but it fails.
Maybe it is best that you assist me and I can do it next time.

[kuisathaverat]

The latest changes fix the issue with the curve25519-sha256 kex, but breaks the rsa-sha2-256 and rsa-sha2-512 key negotiation

ssh-agent-rsa-512               | debug1: inetd sockets after dupping: 4, 4
ssh-agent-rsa-512               | Connection from 192.168.80.3 port 53682 on 192.168.80.8 port 22 rdomain ""
ssh-agent-rsa-512               | debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
ssh-agent-rsa-512               | debug1: Remote protocol version 2.0, remote software version TrileadSSH2Java_213
ssh-agent-rsa-512               | debug1: no match: TrileadSSH2Java_213
ssh-agent-rsa-512               | debug1: permanently_set_uid: 105/65534 [preauth]
ssh-agent-rsa-512               | debug1: list_hostkey_types: rsa-sha2-512 [preauth]
ssh-agent-rsa-512               | debug1: SSH2_MSG_KEXINIT sent [preauth]
ssh-agent-rsa-512               | debug1: SSH2_MSG_KEXINIT received [preauth]
ssh-agent-rsa-512               | debug1: kex: algorithm: diffie-hellman-group-exchange-sha256 [preauth]
ssh-agent-rsa-512               | debug1: kex: host key algorithm: (no match) [preauth]
ssh-agent-rsa-512               | Unable to negotiate with 192.168.80.3 port 53682: no matching host key type found. Their offer: ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-rsa,ssh-dss [preauth]
ssh-agent-rsa-512               | debug1: do_cleanup [preauth]
ssh-agent-rsa-512               | debug1: monitor_read_log: child log fd closed
ssh-agent-rsa-512               | debug1: do_cleanup
ssh-agent-rsa-512               | debug1: Killing privsep child 90
ssh-agent-rsa-512               | debug1: audit_event: unhandled event 12
ssh-agent-rsa-512               | debug1: main_sigchld_handler: Child exited

ssh-agent-rsa-256               | debug1: inetd sockets after dupping: 4, 4
ssh-agent-rsa-256               | Connection from 192.168.80.3 port 59600 on 192.168.80.4 port 22 rdomain ""
ssh-agent-rsa-256               | debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.1
ssh-agent-rsa-256               | debug1: Remote protocol version 2.0, remote software version TrileadSSH2Java_213
ssh-agent-rsa-256               | debug1: no match: TrileadSSH2Java_213
ssh-agent-rsa-256               | debug1: permanently_set_uid: 105/65534 [preauth]
ssh-agent-rsa-256               | debug1: list_hostkey_types: rsa-sha2-256 [preauth]
ssh-agent-rsa-256               | debug1: SSH2_MSG_KEXINIT sent [preauth]
ssh-agent-rsa-256               | debug1: SSH2_MSG_KEXINIT received [preauth]
ssh-agent-rsa-256               | debug1: kex: algorithm: diffie-hellman-group-exchange-sha256 [preauth]
ssh-agent-rsa-256               | debug1: kex: host key algorithm: (no match) [preauth]
ssh-agent-rsa-256               | Unable to negotiate with 192.168.80.3 port 59600: no matching host key type found. Their offer: ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-rsa,ssh-dss [preauth]
ssh-agent-rsa-256               | debug1: do_cleanup [preauth]
ssh-agent-rsa-256               | debug1: monitor_read_log: child log fd closed
ssh-agent-rsa-256               | debug1: do_cleanup
ssh-agent-rsa-256               | debug1: Killing privsep child 96
ssh-agent-rsa-256               | debug1: audit_event: unhandled event 12
ssh-agent-rsa-256               | debug1: main_sigchld_handler: Child exited

[Elisedlund-ericsson]

but breaks the rsa-sha2-256 and rsa-sha2-512 key negotiation

why would they work? (have they ever worked?)
Latest trilead release supports: [ssh-ed25519, ecdsa-sha2-nistp521, ecdsa-sha2-nistp384, ecdsa-sha2-nistp256, ssh-rsa, ssh-dss]
same as "Their offer: ssh-ed25519,ecdsa-sha2-nistp521,ecdsa-sha2-nistp384,ecdsa-sha2-nistp256,ssh-rsa,ssh-dss"

This pullrequest is to add kex algorithms and does not state that it adds any host key algorithms.

[mpet]

@jvz I have tried rsa-sha2-512 for sshlib 0.9.5 as peer. This is what happens:

In verifySignature() we can see that:

algorithm.getKeyFormat().equals(kxs.np.server_host_key_algo)

kxs.np.server_host_key_algo = rsa-sha2-512

PublicKey publicKey = algorithm.decodePublicKey(hostkey);

The PublicKey is:
Sun RSA public key, 2048 bits
OID: 1.2.840.113549.1.1.1

We try to decode the signature from peer.

byte[] signature = algorithm.decodeSignature(sig);

RSAKeyAlgorithm class we do decodeSignature:

 if (!sig_format.equals(getKeyFormat())) {

sig_format is 'ssh-rsa' but should be 'rsa-sha2-512'

So this means the signature for peer is not rsa-sha-512

So the signature in response from peer:

PacketKexDHReply dhr = new PacketKexDHReply(msg, 0, msglen);

contains a signature using 'ssh-rsa'

How is it intended to work?

[mpet]

@kuisathaverat could you make yet another check? i got it working with our product under test.

[jvz]

Ha, I love this RFC because you, just like me, have been confused by what exactly the damn RFC is modifying. The key format is still called "ssh-rsa" because it's just the RSA key; the only change here is for signatures which follow the sha2 format. This same problem has popped up in other SSH libraries during development of the same feature. An ssh-rsa signature is still sha1, but an ssh-rsa host key or authentication key will remain the same for sha2.

[jvz]

Relevant RFC: https://tools.ietf.org/html/rfc8332

Note that there might be a bug (or missing feature?) with some SSH implementations related to trying to use the SHA-2 variant first before falling back to trying again with SHA-1. If there's some extension negotiation info that can help reduce the unnecessary encoding attempts, that might be what you're seeing. It's been a while since I was deep in that code (the latest SSH-related changes I've worked on elsewhere (apache/mina-sshd#177) were related to curve25519-sha256 and curve448-sha512 support similar to this PR).

A potentially related commit in Mina that might help you figure this out:

• apache/mina-sshd@b9ddc14

[mpet]

@kuisathaverat any time to check again? would be nice with a README on how to test :-)

[mpet]

@kuisathaverat this is a friendly bump about running tests to verify ;-)

[jvz]

Thanks for handling this, Ivan!