[NIFI-7034] Connection leak with JMSConsumer and JMSPublisher

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.11.0
• Component/s: Extensions
• Environment: Discovered against ActiveMQ.
• Flags: Important
• Assignee: Gardella Juan Pablo
• Reporter: Gardella Juan Pablo
• Created: 16/Jan/20 14:50
• Updated: 14/Feb/24 16:09
• Resolved: 19/Jan/20 17:09

Description

JMS connections are not closed in case of a failure. Discovered against ActiveMQ, but it applies to other JMS servers.

The problem happens when an exception is raised and the worker is marked as invalid. The current code discards the worker before closing it properly. Below the details.

Details

Any exception happening to a ConsumerJMS or PublisherJMS marks the worker as invalid. After that, the worker is discarded (the worker object reference is never cleaned). Below the snipped code of the issue:

AbstractJMSProcessor

        } finally {
            //in case of exception during worker's connection (consumer or publisher),
            //an appropriate service is responsible to invalidate the worker.
            //if worker is not valid anymore, don't put it back into a pool, try to rebuild it first, or discard.
            //this will be helpful in a situation, when JNDI has changed, or JMS server is not available
            //and reconnection is required.
            if (worker == null || !worker.isValid()){
                getLogger().debug("Worker is invalid. Will try re-create... ");
                final JMSConnectionFactoryProviderDefinition cfProvider = context.getProperty(CF_SERVICE).asControllerService(JMSConnectionFactoryProviderDefinition.class);
                try {
                    // Safe to cast. Method #buildTargetResource(ProcessContext context) sets only CachingConnectionFactory
                    CachingConnectionFactory currentCF = (CachingConnectionFactory)worker.jmsTemplate.getConnectionFactory();
                    cfProvider.resetConnectionFactory(currentCF.getTargetConnectionFactory());
                    worker = buildTargetResource(context);
                }catch(Exception e) {
                    getLogger().error("Failed to rebuild:  " + cfProvider);
                    worker = null;
                }
            }

Before discard the worker, it should be cleaned all resources associated with it. The proper solution is to call worker.shutdown() and then discard it.

Issue Links

• fixes: NIFI-6391 ConsumeJMS/PublishJMS - Excessive connections created when get/publish operations continually fail
• links to: GitHub Pull Request #4001
• links to: GitHub Pull Request #4002