Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2764

Fix renewal of dfs delegation tokens

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments


    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s:, 0.23.0
    • Component/s: None
    • Labels:
    • Release Note:
      Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http.


      The JT may have issues renewing hftp tokens which disrupt long distcp jobs. The problem is the JT's delegation token renewal code is built on brittle assumptions. The token's service field contains only the "ip:port" pair. The renewal process assumes that the scheme must be hdfs. If that fails due to a VersionMismatchException, it tries https based on another assumption that it must be hftp if it's not hdfs. A number of other exceptions, most commonly IOExceptions, can be generated which fouls up the renewal since it won't fallback to https.


        1. MAPREDUCE-2764-trunk.patch
          83 kB
          Jitendra Nath Pandey
        2. MAPREDUCE-2764-trunk.patch
          81 kB
          Jitendra Nath Pandey
        3. MAPREDUCE-2764-trunk.patch
          67 kB
          Jitendra Nath Pandey
        4. token-renew-trunk.patch
          63 kB
          Owen O'Malley
        5. token-renew.patch
          61 kB
          Owen O'Malley
        6. token-renew.patch
          59 kB
          Owen O'Malley
        7. MAPREDUCE-2764-4.patch
          72 kB
          Daryn Sharp
        8. MAPREDUCE-2764-3.patch
          64 kB
          Daryn Sharp
        9. MAPREDUCE-2764-2.patch
          54 kB
          Daryn Sharp
        10. delegation.patch
          24 kB
          Owen O'Malley
        11. MAPREDUCE-2764.patch
          65 kB
          Daryn Sharp

        Issue Links



            • Assignee:
              omalley Owen O'Malley
              daryn Daryn Sharp


              • Created:

                Issue deployment