Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-7663

SFTP consumers cannot be triggered by Quartz2 scheduler with JDBC trigger persistence: NotSerializableException

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.13.1
    • Fix Version/s: 2.14.0
    • Labels:
      None
    • Environment:

      Quartz 2.2.1 with JDBC JobStore

    • Estimated Complexity:
      Unknown

      Description

      I configured a setup with an SFTP consumer like this in Spring DSL:

      <from uri="sftp:user@host/some.directory?privateKeyUri=classpath:FOO.key&amp;delete=true&amp;scheduler=#q2scheduler" />
      

      Note: q2scheduler refers to an instance of QuartzScheduledPollConsumerScheduler that I set have up with my Cron pattern.

      Starting up the Camel context fails with a NotSerializableException as soon as I switch Quartz from a RAMJobStore to a JDBCJobStore.

      The following exception gets thrown:

      org.quartz.JobPersistenceException: Couldn't store job: Unable to serialize JobDataMap for insertion into database because the value of property 'task' is not serializable: org.apache.camel.component.file.remote.SftpConsumer [See nested exception: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'task' is not serializable: org.apache.camel.component.file.remote.SftpConsumer]
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1115)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1062)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787)
      	at org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058)
      	at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886)
      	at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249)
      	at org.apache.camel.pollconsumer.quartz2.QuartzScheduledPollConsumerScheduler.doStart(QuartzScheduledPollConsumerScheduler.java:174)
      	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
      	at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:74)
      	at org.apache.camel.impl.ScheduledPollConsumer.doStart(ScheduledPollConsumer.java:499)
      	at org.apache.camel.component.file.GenericFileConsumer.doStart(GenericFileConsumer.java:640)
      	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
      	at org.apache.camel.impl.DefaultCamelContext.startService(DefaultCamelContext.java:2042)
      	at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRouteConsumers(DefaultCamelContext.java:2336)
      	at org.apache.camel.impl.DefaultCamelContext.doStartRouteConsumers(DefaultCamelContext.java:2272)
      	at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:2202)
      	at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:1981)
      	at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:1851)
      	at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:1683)
      	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
      	at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:1651)
      	at org.apache.camel.component.servletlistener.CamelServletContextListener.contextInitialized(CamelServletContextListener.java:179)
                     Truncated WebLogic classes....
      Caused by: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'task' is not serializable: org.apache.camel.component.file.remote.SftpConsumer
      	at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.serializeJobData(StdJDBCDelegate.java:3083)
      	at org.quartz.impl.jdbcjobstore.oracle.OracleDelegate.insertJobDetail(OracleDelegate.java:154)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1112)
      	... 65 more
      

      That implies, that a File consumer will fail in such a scenario as well, not only an SFTP consumer.
      Given that the documentation http://camel.apache.org/file2.html advertises Quartz2 support for polling scheduling, we were expecting that this really works.

      For us, this is kind of tragic, as we wanted to use Quartz2's clustering capabilities to make the SFTP consumer fit for use in a (Weblogic) cluster.
      Looking at GenericFileConsumer, it seems at least non-trivial to make all the fields in SftpConsumer's class hierarchy serializable.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                davsclaus Claus Ibsen
                Reporter:
                steffen.ryll Steffen Ryll
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: