Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2110

cancel delegation token of launcher jobs that stay till child jobs finish

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: 4.2.0
    • Component/s: None
    • Labels:
      None

      Description

      set mapreduce.job.complete.cancel.delegation.tokens=true for launcher job that wait for child jobs to finish (most of actions, pig,hive,java, distcp, etc). this is currently set to false.

      keep mapreduce.job.complete.cancel.delegation.tokens=false for launcher job that exits right after a child job launched (mapred action), such that it does not delete delegation token while child job still running.

      this change reduce the load on Namenode in terms of the number of tokens it has to maintain.

        Attachments

        1. OOZIE-2110.patch
          6 kB
          Ryota Egashira
        2. OOZIE-2110-v2.patch
          7 kB
          Ryota Egashira

          Activity

            People

            • Assignee:
              egashira Ryota Egashira
              Reporter:
              egashira Ryota Egashira
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: