Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-10281

ProducerTemplate sending to variable URI's fills-up the heap

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: 2.16.3
    • Fix Version/s: None
    • Component/s: camel-ftp
    • Labels:
    • Environment:

      JDK 8, JBoss EAP7, WAR deployment with SpringFramework hosting CAMEL context and routes, dynamic FTP sender to ad hoc (S)FTP destinations using ProducerTemplate

    • Estimated Complexity:
      Unknown
    • Flags:
      Important

      Description

      We have build a connector (WAR deployment) in XML DSL reading from a JMS queue, using numerous custom processors, and sending messages to SFTP destination. The URI is built dynamically and usually varies (file name, destination directory and server, etc) with each file being transmitted.
      We conducted soak tests and the JVM becomes unresponsive after a few hours. The HEAP increases steadily and rapidly to several Gbytes until we undeploy the WAR at which time the GC can recover memory.
      We discovered that the memory actually fills up with dynamic endpoint instances. If we trace producerTemplate.getCurrentCacheSize(), the number increases by 1 with every file being transmitted. The ProducerTemplate is supposed to obey a max 1000 cache size default but ignores this boundary and any other max cache size setting.
      We can also trace this increasing count but this time up to 1000 max (+ number of static endpoints) via getContext().getEndpointMap().size(). If we try compensating with getContext().removeEndpoint(...) as obtained from the previous map, the remove() generates no error, yet the producer template cache size does not reduce its ever increasing count.
      On the other hand, if we are careful to always guarantee that the CAMEL URI is identical with every transmission and that all dynamic parameters are actually mapped on the exchange header properties instead of CAMEL options, then the count of dynamic endpoints remains at one from the start.
      Whoever builds URIs and not knowing this is at risk of breaking production after a few hours or a few days according to traffic intensity.

      Links to CAMEL-2558, CAMEL-3827, CAMEL-7965

        Attachments

          Activity

            People

            • Assignee:
              davsclaus Claus Ibsen
              Reporter:
              bh@artofe.biz bernard HAUZEUR
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: