[SSHD-1231] Public key authentication: wrong signature algorithm used (ed25519 key with ssh-rsa signature)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0, 2.8.0
• Fix Version/s: 2.9.0
• External issue URL: https://bugs.eclipse.org/bugs/show_bug.cgi?id=577545
• Assignee: Thomas Wolf
• Reporter: Thomas Wolf
• Created: 01/Dec/21 23:08
• Updated: 17/Jun/22 10:09
• Resolved: 20/Dec/21 12:42

Description

See Eclipse bug 577545. The following scenario

1. Client tries authenticating with a wrong RSA key with signature rsa-sha2-512
2. Server rejects the authentication attempt
3. Client tries the next (correct) key (an ed25519 key), but unfortunately with the wrong signature algorithm (ssh-rsa)

authentication fails with

Exception in thread "main" org.apache.sshd.common.SshException: DefaultAuthFuture[ssh-connection]: Failed (InvalidKeyException) to execute: Supplied key (net.i2p.crypto.eddsa.EdDSAPrivateKey) is not a RSAPrivateKey instance
	at org.apache.sshd.common.future.AbstractSshFuture.lambda$verifyResult$1(AbstractSshFuture.java:131)
	at org.apache.sshd.common.future.AbstractSshFuture.formatExceptionMessage(AbstractSshFuture.java:185)
	at org.apache.sshd.common.future.AbstractSshFuture.verifyResult(AbstractSshFuture.java:130)
	at org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:39)
	at org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:32)
	at org.apache.sshd.common.future.VerifiableFuture.verify(VerifiableFuture.java:43)
	at ch.paranor.thomas.TestClient.testAuth(TestClient.java:44)
	at ch.paranor.thomas.TestClient.main(TestClient.java:58)
Caused by: java.security.InvalidKeyException: Supplied key (net.i2p.crypto.eddsa.EdDSAPrivateKey) is not a RSAPrivateKey instance
	at org.bouncycastle.jcajce.provider.asymmetric.rsa.DigestSignatureSpi.engineInitSign(Unknown Source)
	at java.security.Signature$Delegate.engineInitSign(Signature.java:1177)
	at java.security.Signature.initSign(Signature.java:530)
	at org.apache.sshd.common.signature.AbstractSignature.initSigner(AbstractSignature.java:104)
	at org.apache.sshd.client.auth.pubkey.KeyPairIdentity.sign(KeyPairIdentity.java:81)
	at org.apache.sshd.client.auth.pubkey.UserAuthPublicKey.appendSignature(UserAuthPublicKey.java:363)
	at org.apache.sshd.client.auth.pubkey.UserAuthPublicKey.processAuthDataRequest(UserAuthPublicKey.java:333)

This exception is thrown in the client if the server is an OpenSSH server. In step 3 above, the following occurs:

• Clients sends SSH_MSG_USERAUTH_REQUEST signature type=ssh-rsa, pubkey=ed25519_key, hasSig=false.
• OpenSSH server checks pubkey against authorized keys and finds a match; replies SSH_MSG_USERAUTH_PK_OK ssh-rsa, ed25519_key.
• Client tries to build a ssh-rsa signature with the ed25519 key and gets the exception.

With an Apache MINA sshd server, the exception occurs on the server side:

• Clients sends SSH_MSG_USERAUTH_REQUEST signature type=ssh-rsa, pubkey=ed25519_key, hasSig=false.
• Apache MINA sshd server initializes a signature verifier with ssh-rsa and the ed25519 key, gets the exception, and replies SSH_MSG_USERAUTH_FAILURE.
• Client skips this key and tries the next one, if any.

Work-arounds:

• Place RSA keys last in the sequence of keys to be tried
• Or ensure only actually working keys are used