Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3581

Callback does not applied in Oozie server, workflows stuk in RUNNING states.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: 4.3.1
    • Fix Version/s: None
    • Component/s: action, workflow
    • Labels:
      None

      Description

      oozie version 4.3.1.3.1.0.0-78

      with HDP3.10 stack , release provides Oozie 4.3.1 and the additional Apache patches listed here: https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.0/release-notes/content/patch_oozie.html

      I use Hadoop kerberized cluster, run on OOZIE, YARN Mr jobs.

      1. OOzie run mr-job via YARN

      2. After YARN mr job completed,YYARN mr-job Successfully sent Callback request to OOzie,

      3. in logs OOzie server, i can see this request, but OOZIE does not apply this callback request, so in WF-action-id i still see RUNNING state (until action.check process check wf-ids and swith action-id to SUCCESS state)

      LOGS in YARN:

      2020-01-27 12:16:39,749 INFO [Thread-78] org.eclipse.jetty.util.log: Job end notification trying http://hdp3-oozie:11000/oozie/callback?id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
      2020-01-27 12:16:39,772 INFO [Thread-78] org.eclipse.jetty.util.log: Job end notification to http://hdp3-oozie:11000/oozie/callback?id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED succeeded
      2020-01-27 12:16:39,772 INFO [Thread-78] org.eclipse.jetty.util.log: Job end notification succeeded for job_1579778851579_31505

       

      Oozie logs about this event:

      2020-01-27 12:16:39,770 DEBUG CallbackServlet:526 - SERVER[hdp3-oo-2] USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0005607-200123121357414-oozie-oozi-W] ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java] Received a CallbackServlet.doGet() with query string id=0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java&status=SUCCEEDED
      2020-01-27 12:16:39,776 DEBUG CompletedActionXCommand:526 - SERVER[hdp3-oo-2] USER[-] GROUP[-] TOKEN[] APP[-] JOB[0005607-200123121357414-oozie-oozi-W] ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java] Execute command [callback] key [null]
      2020-01-27 12:16:39,776 DEBUG CompletedActionXCommand:526 - SERVER[hdp3-oo-2] USER[-] GROUP[-] TOKEN[] APP[-] JOB[0005607-200123121357414-oozie-oozi-W] ACTION[0005607-200123121357414-oozie-oozi-W@rdb-full-table-extract-java] Queuing [1] commands with delay [0]ms

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vkotsubinskiy Kotsubinsky Victor
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: