Details
-
Improvement
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
1.6.0, 1.8.0, 1.7.1, 1.10.0, 1.9.2, 1.11.4
-
None
-
Patch
Description
Below an scenario to reproduce the non optimize usage of JMS resources. Suppose it is required to publish 1 message to the destination D using PublishJMS. The message is a flow file in the processor input queue.
It is important to know that internally the processor is using CachingConnectionFactory to reuse objects and a worker to be able to use in thread safe manner. For JMS publishers, the default configuration is to cache connections, sessions (only 1) and message producers.
Preconditions
- Flowfile has either jms_destination or jms_replyTo attribute defined. Due to NIFI-7561, it should contain the word queue or topic. Also notice jms_destination should be ignored, as suggested at NIFI-7564. That will limit the scenario only when jms_replyTo attribute is defined.
- For simplicity, the processor is the first time it processes messages.
Scenario
- Processor picks the message. The worker is created.
- Connection C1 and session S1 are created. The Message M1_S1 is created and MessageProducer MP_S1 created too. Required to deliver first message at JMSPublisher#publish.
- S1 and C1 are stored in CachingConnectionFactory. The caching connection factory is created at AbstractJMSProcessor.java#L208.
- An attempt to create a new connection and a new session are requested to the connection factory to create destination defined in the header jms_destination at JMSPublisher.java#L131. Notice the connection C1 is reused although S1 is not reused (it is required to check internal logic in CachingConnectionFactory to understand why not). A new session S2 is created and stored in the CachingConnectionFactory as the new cached session.
- Message is published and S1 and MP_S1 are closed. As S1 is not in the cache, it is physically closed and MP_S1.
- At this point of time, the cached objects are C1, S2. Ideally, all resources should be reused.
The scenario if it is applied to N consecutive messages create a lot of sessions and message producers.
We found this issue by adding an Interceptor to an Apache ActiveMQ v5.x broker to detect the optimal usage of resources. For example, only one message producer per connection. In below scenario we will be created N producers for the same connection. Also in a Nifi flow that connects a ConsumeJMS with a PublishJMS. Notice ConsumeJMS populates by default jms_destination flowfile attribute which, if it is not removed, it is processed by PublishJMS processor (by solving NIFI-7564 should not happen any more).
