Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-3035

HDFS HA and log aggregation: getting HDFS delegation token from YARN renewer within JavaActionExecutor

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 4.3.0
    • Fix Version/s: 5.0.0b1
    • Component/s: None
    • Labels:
      None
    • Environment:

      Description

      In a secure environment, when both HDFS HA and log aggregation are turned on, JavaActionExecutor is not able to call YarnClient#submitApplication since HDFS_DELEGATION_TOKEN is missing.

      In those cases we need to get HDFS_DELEGATION_TOKEN from YARN:

      • get server principal / YARN renewer via HadoopAccessorService#getServerPrincipal
      • get HDFS_DELEGATION_TOKEN via DFSClient#getDelegationToken
      • add HDFS_DELEGATION_TOKEN to Credentials

        Attachments

        1. OOZIE-3035.005.patch
          17 kB
          Andras Piros
        2. OOZIE-3035.004.patch
          17 kB
          Andras Piros
        3. OOZIE-3035.003.patch
          9 kB
          Andras Piros
        4. OOZIE-3035.002.patch
          8 kB
          Andras Piros
        5. OOZIE-3035.001.patch
          4 kB
          Andras Piros

          Issue Links

            Activity

              People

              • Assignee:
                andras.piros Andras Piros
                Reporter:
                andras.piros Andras Piros
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: