Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2764

Fix renewal of dfs delegation tokens



    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    •, 0.23.0
    • None
    • None
    • Generalizes token renewal and canceling to a common interface and provides a plugin interface for adding renewers for new kinds of tokens. Hftp changed to store the tokens as HFTP and renew them over http.


      The JT may have issues renewing hftp tokens which disrupt long distcp jobs. The problem is the JT's delegation token renewal code is built on brittle assumptions. The token's service field contains only the "ip:port" pair. The renewal process assumes that the scheme must be hdfs. If that fails due to a VersionMismatchException, it tries https based on another assumption that it must be hftp if it's not hdfs. A number of other exceptions, most commonly IOExceptions, can be generated which fouls up the renewal since it won't fallback to https.


        1. MAPREDUCE-2764.patch
          65 kB
          Daryn Sharp
        2. delegation.patch
          24 kB
          Owen O'Malley
        3. MAPREDUCE-2764-2.patch
          54 kB
          Daryn Sharp
        4. MAPREDUCE-2764-3.patch
          64 kB
          Daryn Sharp
        5. MAPREDUCE-2764-4.patch
          72 kB
          Daryn Sharp
        6. token-renew.patch
          59 kB
          Owen O'Malley
        7. token-renew.patch
          61 kB
          Owen O'Malley
        8. token-renew-trunk.patch
          63 kB
          Owen O'Malley
        9. MAPREDUCE-2764-trunk.patch
          67 kB
          Jitendra Nath Pandey
        10. MAPREDUCE-2764-trunk.patch
          81 kB
          Jitendra Nath Pandey
        11. MAPREDUCE-2764-trunk.patch
          83 kB
          Jitendra Nath Pandey

        Issue Links



              omalley Owen O'Malley
              daryn Daryn Sharp
              0 Vote for this issue
              9 Start watching this issue