[JAMES-3741] SSL: sporadic failure of new connection failure under load in 3.7.x

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.7.0
• Fix Version/s: master
• Component/s: IMAPServer, POP3Server, SMTPServer
• Assignee: Unassigned
• Reporter: Benoit Tellier
• Created: 04/Apr/22 08:14
• Updated: 03/Jun/24 15:03
• Resolved: 03/Jun/24 15:03

Description

Exception seen on 3.7.x... Context performance tests with several new SSL connections opened per seconds (high concurrency).

21:34:28.460 [WARN ] o.a.j.i.n.ImapChannelUpstreamHandler - Error while processing imap request
javax.crypto.BadPaddingException: Insufficient buffer remaining for AEAD cipher fragment (2). Needs to be more than tag size (16)
	at java.base/sun.security.ssl.SSLCipher$T13GcmReadCipherGenerator$GcmReadCipher.decrypt(SSLCipher.java:1894)
	at java.base/sun.security.ssl.SSLEngineInputRecord.decodeInputRecord(SSLEngineInputRecord.java:240)
	at java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:197)
	at java.base/sun.security.ssl.SSLEngineInputRecord.decode(SSLEngineInputRecord.java:160)
	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:111)
	... 24 common frames omitted

Can be reliably reproduced by opening many new SSL connections concurrently:

    @Nested
    class Toto {
        IMAPServer imapServer;
        int port;

        @BeforeEach
        void setup() throws Exception {
            HierarchicalConfiguration<ImmutableNode> config = ConfigLoader.getConfig(ClassLoaderUtils.getSystemResourceAsSharedStream("imapSSL.xml"));
            imapServer = createImapServer(config);
            port = imapServer.getListenAddresses().get(0).getPort();
        }

        @AfterEach
        void tearDown() {
            if (imapServer != null) {
                imapServer.destroy();
            }
        }

        @Test
        void test() throws Exception {
            ConcurrentTestRunner.builder()
                .operation((a, b) -> {
                    IMAPSClient imapsClient = imapsImplicitClient(port);
                    final boolean capability = imapsClient.capability();
                    assertThat(capability).isTrue();
                    final boolean close = imapsClient.close();
                })
                .threadCount(10)
                .operationCount(3000)
                .runSuccessfullyWithin(Duration.ofMinutes(10));
        }

        private IMAPSClient imapsImplicitClient(int port) throws Exception {
            IMAPSClient client = new IMAPSClient(true, BogusSslContextFactory.getClientContext());
            client.setTrustManager(BogusTrustManagerFactory.getTrustManagers()[0]);
            client.connect("127.0.0.1", port);
            return client;
        }
    }

and `imapSSL.xml` being:

<imapserver enabled="true">
    <jmxName>imapserver</jmxName>
    <bind>0.0.0.0:9993</bind>
    <tls socketTLS="true" startTLS="false">
        <privateKey>private.key</privateKey>
        <certificates>certs.self-signed.csr</certificates>
        <secret>123456</secret>
    </tls>
    <auth>
        <plainAuthEnabled>true</plainAuthEnabled>
        <requireSSL>true</requireSSL>
    </auth>
</imapserver>

Interestingly enough the Netty4 migration post 3.7.x fixed the issue.

Thus it will be fixed in later releases yet it seemed interesting to me to document the issue.

I propose to add a non-regression test on master.

Issue Links

• links to: GitHub Pull Request #951