Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-7809

Quartz PollConsumerScheduler in a cluster tries to create duplicate triggers, fails

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.13.1
    • 2.18.4, 2.19.0
    • camel-quartz2
    • None
    • identical CamelContexts deployed to each node in a cluster
      Quartz 2.2.1 with JDBC JobStore and synchronization via DB

    • Patch Available
    • Unknown

    Description

      I tested the patch developed in CAMEL-7663 and ran into another issue that prevents the CamelContext from starting up. The setup is the same as described in CAMEL-7663.
      Upon startup I get following exception from Quartz on at least one cluster node:

      13:23:36|INFO |component.quartz2.QuartzComponent|[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'|Shutting down scheduler. (will wait for all jobs to complete first.)
      13:23:36|INFO |quartz.core.QuartzScheduler|[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'|Scheduler IPP_Integration_Scheduler-emerald-integration-web_$_fra...1410348212901 shutting down.
      13:23:36|INFO |quartz.core.QuartzScheduler|[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'|Scheduler IPP_Integration_Scheduler-emerald-integration-web_$_fra...1410348212901 paused.
      13:23:36|INFO |quartz.core.QuartzScheduler|[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'|Scheduler unregistered from name 'quartz:type=QuartzScheduler,name=IPP_Integration_Scheduler-emerald-integration-web,instance=fra...1410348212901' in the local MBeanServer.
      2014-09-10-13:23:36|INFO |quartz.core.QuartzScheduler|[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'|Scheduler IPP_Integration_Scheduler-emerald-integration-web_$_fra...1410348212901 shutdown complete.
      13:23:36|ERROR|component.servletlistener.CamelServletContextListener|[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'|Error starting CamelContext.
      org.quartz.ObjectAlreadyExistsException: Unable to store Trigger with name: 'scheduler.fax.in' and group: 'QuartzScheduledPollConsumerScheduler', because one already exists with this identification.
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeTrigger(JobStoreSupport.java:1179)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport$2.executeVoid(JobStoreSupport.java:1063)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3703)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport$VoidTransactionCallback.execute(JobStoreSupport.java:3701)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.executeInNonManagedTXLock(JobStoreSupport.java:3787)
      	at org.quartz.impl.jdbcjobstore.JobStoreTX.executeInLock(JobStoreTX.java:93)
      	at org.quartz.impl.jdbcjobstore.JobStoreSupport.storeJobAndTrigger(JobStoreSupport.java:1058)
      	at org.quartz.core.QuartzScheduler.scheduleJob(QuartzScheduler.java:886)
      	at org.quartz.impl.StdScheduler.scheduleJob(StdScheduler.java:249)
      	at org.apache.camel.pollconsumer.quartz2.QuartzScheduledPollConsumerScheduler.doStart(QuartzScheduledPollConsumerScheduler.java:187)
      	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
      	at org.apache.camel.util.ServiceHelper.startService(ServiceHelper.java:74)
      	at org.apache.camel.impl.ScheduledPollConsumer.doStart(ScheduledPollConsumer.java:499)
      	at org.apache.camel.component.file.GenericFileConsumer.doStart(GenericFileConsumer.java:640)
      	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
      	at org.apache.camel.impl.DefaultCamelContext.startService(DefaultCamelContext.java:2042)
      	at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRouteConsumers(DefaultCamelContext.java:2336)
      	at org.apache.camel.impl.DefaultCamelContext.doStartRouteConsumers(DefaultCamelContext.java:2272)
      	at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:2202)
      	at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:1981)
      	at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:1851)
      	at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:1683)
      	at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
      	at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:1651)
      	at org.apache.camel.component.servletlistener.CamelServletContextListener.contextInitialized(CamelServletContextListener.java:179)
      	at weblogic.servlet.internal.EventsManager$FireContextListenerAction.run(EventsManager.java:661)
      	at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
      	at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
      	at weblogic.servlet.provider.WlsSubjectHandle.run(WlsSubjectHandle.java:57)
      	at weblogic.servlet.internal.EventsManager.executeContextListener(EventsManager.java:241)
      	at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:198)
      	at weblogic.servlet.internal.EventsManager.notifyContextCreatedEvent(EventsManager.java:183)
      	at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1783)
      	at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:2807)
      	at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1661)
      	at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:822)
      	at weblogic.application.internal.ExtensibleModuleWrapper$StartStateChange.next(ExtensibleModuleWrapper.java:360)
      	at weblogic.application.internal.ExtensibleModuleWrapper$StartStateChange.next(ExtensibleModuleWrapper.java:356)
      	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
      	at weblogic.application.internal.ExtensibleModuleWrapper.start(ExtensibleModuleWrapper.java:138)
      	at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:124)
      	at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:213)
      	at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:208)
      	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
      	at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:70)
      	at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:24)
      	at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:729)
      	at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
      	at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:258)
      	at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:61)
      	at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:165)
      	at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:80)
      	at weblogic.deploy.internal.targetserver.operations.AbstractOperation.activate(AbstractOperation.java:586)
      	at weblogic.deploy.internal.targetserver.operations.ActivateOperation.activateDeployment(ActivateOperation.java:148)
      	at weblogic.deploy.internal.targetserver.operations.ActivateOperation.doCommit(ActivateOperation.java:114)
      	at weblogic.deploy.internal.targetserver.operations.StartOperation.doCommit(StartOperation.java:151)
      	at weblogic.deploy.internal.targetserver.operations.AbstractOperation.commit(AbstractOperation.java:339)
      	at weblogic.deploy.internal.targetserver.DeploymentManager.handleDeploymentCommit(DeploymentManager.java:846)
      	at weblogic.deploy.internal.targetserver.DeploymentManager.activateDeploymentList(DeploymentManager.java:1275)
      	at weblogic.deploy.internal.targetserver.DeploymentManager.handleCommit(DeploymentManager.java:442)
      	at weblogic.deploy.internal.targetserver.DeploymentServiceDispatcher.commit(DeploymentServiceDispatcher.java:176)
      	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentReceiverCallbackDeliverer.java:195)
      	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer.java:13)
      	at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$2.run(DeploymentReceiverCallbackDeliverer.java:68)
      	at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:550)
      	at weblogic.work.ExecuteThread.execute(ExecuteThread.java:295)
      	at weblogic.work.ExecuteThread.run(ExecuteThread.java:254)
      

      I have started working on a fix that adopt the approach for clustering found in QuartzEndpoint to QuartzSchedulerPollConsumerScheduler.

      The patch is attached and I am asking for feedback.

      Attachments

        Issue Links

          Activity

            People

              davsclaus Claus Ibsen
              steffen.ryll Steffen Ryll
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: