[CAMEL-7663] SFTP consumers cannot be triggered by Quartz2 scheduler with JDBC trigger persistence: NotSerializableException

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.13.1
• Fix Version/s: 2.14.0
• Component/s: camel-core, camel-ftp, camel-quartz2

Environment

Quartz 2.2.1 with JDBC JobStore

• Estimated Complexity: Unknown
• Assignee: Claus Ibsen
• Reporter: Steffen Ryll
• Created: 05/Aug/14 17:13
• Updated: 12/Sep/14 17:18
• Resolved: 25/Aug/14 08:01

Description

I configured a setup with an SFTP consumer like this in Spring DSL:

<from uri="sftp:user@host/some.directory?privateKeyUri=classpath:FOO.key&amp;delete=true&amp;scheduler=#q2scheduler" />

Note: q2scheduler refers to an instance of QuartzScheduledPollConsumerScheduler that I set have up with my Cron pattern.

Starting up the Camel context fails with a NotSerializableException as soon as I switch Quartz from a RAMJobStore to a JDBCJobStore.

The following exception gets thrown:

org.quartz.JobPersistenceException: Couldn't store job: Unable to serialize JobDataMap for insertion into database because the value of property 'task' is not serializable: org.apache.camel.component.file.remote.SftpConsumer [See nested exception: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'task' is not serializable: org.apache.camel.component.file.remote.SftpConsumer]
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1115)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1062)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787)
	at org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058)
	at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886)
	at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249)
	at org.apache.camel.pollconsumer.quartz2.QuartzScheduledPollConsumerScheduler.doStart(QuartzScheduledPollConsumerScheduler.java:174)
	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
	at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:74)
	at org.apache.camel.impl.ScheduledPollConsumer.doStart(ScheduledPollConsumer.java:499)
	at org.apache.camel.component.file.GenericFileConsumer.doStart(GenericFileConsumer.java:640)
	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
	at org.apache.camel.impl.DefaultCamelContext.startService(DefaultCamelContext.java:2042)
	at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRouteConsumers(DefaultCamelContext.java:2336)
	at org.apache.camel.impl.DefaultCamelContext.doStartRouteConsumers(DefaultCamelContext.java:2272)
	at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:2202)
	at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:1981)
	at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:1851)
	at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:1683)
	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
	at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:1651)
	at org.apache.camel.component.servletlistener.CamelServletContextListener.contextInitialized(CamelServletContextListener.java:179)
               Truncated WebLogic classes....
Caused by: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'task' is not serializable: org.apache.camel.component.file.remote.SftpConsumer
	at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.serializeJobData(StdJDBCDelegate.java:3083)
	at org.quartz.impl.jdbcjobstore.oracle.OracleDelegate.insertJobDetail(OracleDelegate.java:154)
	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJob(JobStoreSupport.java:1112)
	... 65 more

That implies, that a File consumer will fail in such a scenario as well, not only an SFTP consumer.
Given that the documentation http://camel.apache.org/file2.html advertises Quartz2 support for polling scheduling, we were expecting that this really works.

For us, this is kind of tragic, as we wanted to use Quartz2's clustering capabilities to make the SFTP consumer fit for use in a (Weblogic) cluster.
Looking at GenericFileConsumer, it seems at least non-trivial to make all the fields in SftpConsumer's class hierarchy serializable.

Issue Links

• is required by: CAMEL-7809 Quartz PollConsumerScheduler in a cluster tries to create duplicate triggers, fails
• relates to: CAMEL-7809 Quartz PollConsumerScheduler in a cluster tries to create duplicate triggers, fails