Uploaded image for project: 'Falcon'
  1. Falcon
  2. FALCON-1739 Tracking API differences between Native Scheduler and Oozie
  3. FALCON-1741

Rerun API behaviour different in case of succeeded instances

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9
    • Fix Version/s: 0.9
    • Component/s: oozie, scheduler
    • Labels:
      None

      Description

      In case of succeeded instances , rerun without -force API has different behaviour based on the scheduler choice.

      For oozie :
      The state of instances changes to RUNNING.

      dataqa@lda01:~$ fiprerun processMerlinOozie -start 2016-01-08T12:13Z -end 2016-01-08T12:15Z
      Consolidated Status: SUCCEEDED
      
      Instances:
      Instance		Cluster		SourceCluster		Status		Start		End		Details					Log
      -----------------------------------------------------------------------------------------------
      2016-01-08T12:13Z	ProcessMultipleClustersTest-corp-9706f068	-	RUNNING	2016-01-08T13:03Z	2016-01-08T13:03Z	-	http://8RPCG32.corp.inmobi.com:11000/oozie?job=0001811-160104160825636-oozie-oozi-W
      2016-01-08T12:13Z	ProcessMultipleClustersTest-corp-0b270a1d	-	RUNNING	2016-01-08T13:03Z	2016-01-08T13:03Z	-	http://lda01:11000/oozie?job=0002247-160104115615658-oozie-oozi-W
      
      Additional Information:
      Response: ua1/RERUN
      ua2/RERUN
      Request Id: ua1/871377866@qtp-630572412-35 - 7190c4c8-bacb-4639-8d48-c9e639f544da
      ua2/1554129706@qtp-536122141-13 - bc18127b-1bf8-4ea1-99e6-b1f10ba3a441
      
      

      For native :
      The state of instances changes to READY. Also params are present as a part of rerun API>

      dataqa@lda01:~$ fiprerun ProcessMultipleClustersTest-agregator-coord16-8f55f59b -start 2016-01-08T12:13Z -end 2016-01-08T12:15Z
      Consolidated Status: SUCCEEDED
      
      Instances:
      Instance		Cluster		SourceCluster		Status		Start		End		Details					Log
      -----------------------------------------------------------------------------------------------
      2016-01-08T12:13Z	ProcessMultipleClustersTest-corp-9706f068	-	READY	2016-01-08T13:03Z	2016-01-08T13:03Z	-	http://8RPCG32.corp.inmobi.com:11000/oozie?job=0001812-160104160825636-oozie-oozi-W
      Workflow params
      shouldRecord=false
      timeStamp=2016-01-08-12-13
      mapreduce.job.user.name=dataqa
      falconInPaths=NONE
      feedNames=NONE
      oozie.wf.rerun.failnodes=true
      falcon.libpath=/tmp/falcon-regression-working/lib
      ENTITY_PATH=/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/3cd7594ed13b218460df53a9e827eb28_1452252282546
      entityType=PROCESS
      systemJMSNotificationEnabled=false
      nominalTime=2016-01-08-12-13
      feedInstancePaths=NONE
      field2=value2
      logDir=hdfs://10.14.114.202:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/logs
      userWorkflowEngine=oozie
      oozie.use.system.libpath=true
      userJMSNotificationEnabled=true
      queueName=default
      falconDataOperation=GENERATE
      falcon.system.rerun=true
      fileTime=${formatTime(dateOffset(instanceTime(), 1, 'DAY'), 'yyyy-MMM-dd')}
      field1=value1
      workflowEngineUrl=http://10.14.114.202:11000/oozie/
      datasource=NA
      brokerImplClass=org.apache.activemq.ActiveMQConnectionFactory
      userBrokerImplClass=org.apache.activemq.ActiveMQConnectionFactory
      brokerUrl=tcp://localhost:61616
      ENTITY_NAME=FALCON_PROCESS_DEFAULT_ProcessMultipleClustersTest-agregator-coord16-8f55f59b
      availabilityFlag=NA
      userWorkflowName=ProcessMultipleClustersTest-agregator-coord16-8f55f59b-workflow
      brokerTTL=4320
      entityName=ProcessMultipleClustersTest-agregator-coord16-8f55f59b
      srcClusterName=NA
      userBrokerUrl=tcp://10.14.114.202:61616?daemon=true
      falconInputFeeds=NONE
      userWorkflowVersion=1.0
      oozie.libpath=/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/3cd7594ed13b218460df53a9e827eb28_1452252282546/lib
      cluster=ProcessMultipleClustersTest-corp-9706f068
      jobPriority=NORMAL
      colo.name=ua1
      nameNode=hdfs://10.14.114.202:8020
      user=${user()}
      jobTracker=10.14.114.202:8021
      oozie.wf.application.path=hdfs://10.14.114.202:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/3cd7594ed13b218460df53a9e827eb28_1452252282546
      
      2016-01-08T12:13Z	ProcessMultipleClustersTest-corp-0b270a1d	-	READY	2016-01-08T13:03Z	2016-01-08T13:03Z	-	http://lda01:11000/oozie?job=0002248-160104115615658-oozie-oozi-W
      Workflow params
      fileTime=${formatTime(dateOffset(instanceTime(), 1, 'DAY'), 'yyyy-MMM-dd')}
      cluster=ProcessMultipleClustersTest-corp-0b270a1d
      availabilityFlag=NA
      falconDataOperation=GENERATE
      falconInputFeeds=NONE
      feedNames=NONE
      colo.name=ua2
      shouldRecord=false
      falcon.libpath=/tmp/falcon-regression-working/lib
      mapreduce.job.user.name=dataqa
      feedInstancePaths=NONE
      systemJMSNotificationEnabled=false
      brokerImplClass=org.apache.activemq.ActiveMQConnectionFactory
      userBrokerImplClass=org.apache.activemq.ActiveMQConnectionFactory
      entityName=ProcessMultipleClustersTest-agregator-coord16-8f55f59b
      falcon.system.rerun=true
      jobTracker=192.168.138.236:8021
      userWorkflowVersion=1.0
      workflowEngineUrl=http://192.168.138.236:11000/oozie/
      jobPriority=NORMAL
      ENTITY_NAME=FALCON_PROCESS_DEFAULT_ProcessMultipleClustersTest-agregator-coord16-8f55f59b
      nameNode=hdfs://192.168.138.236:8020
      nominalTime=2016-01-08-12-13
      oozie.libpath=/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/e6720cfbbe1d8a50488a7c47da94b506_1452252285506/lib
      oozie.use.system.libpath=true
      entityType=PROCESS
      srcClusterName=NA
      userWorkflowEngine=oozie
      oozie.wf.rerun.failnodes=true
      timeStamp=2016-01-08-12-13
      brokerUrl=tcp://localhost:61616
      field1=value1
      oozie.wf.application.path=hdfs://192.168.138.236:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/e6720cfbbe1d8a50488a7c47da94b506_1452252285506
      queueName=default
      brokerTTL=4320
      userWorkflowName=ProcessMultipleClustersTest-agregator-coord16-8f55f59b-workflow
      datasource=NA
      ENTITY_PATH=/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/e6720cfbbe1d8a50488a7c47da94b506_1452252285506
      userJMSNotificationEnabled=true
      userBrokerUrl=tcp://192.168.138.236:61616?daemon=true
      falconInPaths=NONE
      user=${user()}
      field2=value2
      logDir=hdfs://192.168.138.236:8020/tmp/falcon-regression-staging/falcon/workflows/process/ProcessMultipleClustersTest-agregator-coord16-8f55f59b/logs
      
      
      Additional Information:
      Response: ua1/RERUN
      ua2/RERUN
      Request Id: ua1/871377866@qtp-630572412-35 - 8d118d4d-c0ef-4335-a9af-10364498ec4f
      ua2/1554129706@qtp-536122141-13 - c2a3fc50-8b05-47ce-9c85-ca432b96d923
      
      

        Attachments

        1. FALCON-1741-v1.patch
          4 kB
          Pallavi Rao

          Issue Links

            Activity

              People

              • Assignee:
                pallavi.rao Pallavi Rao
                Reporter:
                pragya.mittal Pragya Mittal
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: