Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.205.0, 0.23.0
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http.

      Description

      The JT may have issues renewing hftp tokens which disrupt long distcp jobs. The problem is the JT's delegation token renewal code is built on brittle assumptions. The token's service field contains only the "ip:port" pair. The renewal process assumes that the scheme must be hdfs. If that fails due to a VersionMismatchException, it tries https based on another assumption that it must be hftp if it's not hdfs. A number of other exceptions, most commonly IOExceptions, can be generated which fouls up the renewal since it won't fallback to https.

      1. delegation.patch
        24 kB
        Owen O'Malley
      2. MAPREDUCE-2764.patch
        65 kB
        Daryn Sharp
      3. MAPREDUCE-2764-2.patch
        54 kB
        Daryn Sharp
      4. MAPREDUCE-2764-3.patch
        64 kB
        Daryn Sharp
      5. MAPREDUCE-2764-4.patch
        72 kB
        Daryn Sharp
      6. MAPREDUCE-2764-trunk.patch
        83 kB
        Jitendra Nath Pandey
      7. MAPREDUCE-2764-trunk.patch
        81 kB
        Jitendra Nath Pandey
      8. MAPREDUCE-2764-trunk.patch
        67 kB
        Jitendra Nath Pandey
      9. token-renew.patch
        61 kB
        Owen O'Malley
      10. token-renew.patch
        59 kB
        Owen O'Malley
      11. token-renew-trunk.patch
        63 kB
        Owen O'Malley

        Issue Links

          Activity

          Daryn Sharp created issue -
          Kihwal Lee made changes -
          Field Original Value New Value
          Link This issue is blocked by HADOOP-7472 [ HADOOP-7472 ]
          Daryn Sharp made changes -
          Link This issue relates to HADOOP-7510 [ HADOOP-7510 ]
          Daryn Sharp made changes -
          Attachment MAPREDUCE-2764.patch [ 12489283 ]
          Kihwal Lee made changes -
          Link This issue is blocked by HADOOP-7472 [ HADOOP-7472 ]
          Daryn Sharp made changes -
          Link This issue incorporates HDFS-2257 [ HDFS-2257 ]
          Daryn Sharp made changes -
          Link This issue requires HADOOP-7541 [ HADOOP-7541 ]
          Owen O'Malley made changes -
          Attachment delegation.patch [ 12491752 ]
          Daryn Sharp made changes -
          Attachment MAPREDUCE-2764-2.patch [ 12492162 ]
          Daryn Sharp made changes -
          Attachment MAPREDUCE-2764-3.patch [ 12492691 ]
          Daryn Sharp made changes -
          Attachment MAPREDUCE-2764-4.patch [ 12492841 ]
          Owen O'Malley made changes -
          Attachment token-renew.patch [ 12493286 ]
          Daryn Sharp made changes -
          Assignee Daryn Sharp [ daryn ] Owen O'Malley [ owen.omalley ]
          Owen O'Malley made changes -
          Attachment token-renew.patch [ 12493530 ]
          Daryn Sharp made changes -
          Link This issue breaks HDFS-2326 [ HDFS-2326 ]
          Daryn Sharp made changes -
          Link This issue breaks HDFS-2328 [ HDFS-2328 ]
          Daryn Sharp made changes -
          Link This issue incorporates MAPREDUCE-2869 [ MAPREDUCE-2869 ]
          Owen O'Malley made changes -
          Link This issue incorporates HADOOP-7625 [ HADOOP-7625 ]
          Owen O'Malley made changes -
          Link This issue incorporates HADOOP-7644 [ HADOOP-7644 ]
          Owen O'Malley made changes -
          Attachment token-renew-trunk.patch [ 12497056 ]
          Owen O'Malley made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http.
          Fix Version/s 0.23.0 [ 12315570 ]
          Jitendra Nath Pandey made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jitendra Nath Pandey made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Tsz Wo Nicholas Sze made changes -
          Link This issue blocks HDFS-2385 [ HDFS-2385 ]
          Jitendra Nath Pandey made changes -
          Attachment MAPREDUCE-2764-trunk.patch [ 12498858 ]
          Jitendra Nath Pandey made changes -
          Attachment MAPREDUCE-2764-trunk.patch [ 12498952 ]
          Jitendra Nath Pandey made changes -
          Attachment MAPREDUCE-2764-trunk.patch [ 12498967 ]
          Jitendra Nath Pandey made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.24.0 [ 12317654 ]
          Resolution Fixed [ 1 ]
          Jitendra Nath Pandey made changes -
          Link This issue is related to MAPREDUCE-3192 [ MAPREDUCE-3192 ]
          Matt Foley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Daryn Sharp made changes -
          Link This issue breaks HDFS-2473 [ HDFS-2473 ]
          Gavin made changes -
          Link This issue blocks HDFS-2385 [ HDFS-2385 ]
          Gavin made changes -
          Link This issue is depended upon by HDFS-2385 [ HDFS-2385 ]
          Allen Wittenauer made changes -
          Fix Version/s 0.24.0 [ 12317654 ]

            People

            • Assignee:
              Owen O'Malley
              Reporter:
              Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development