Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12052

Set SWEBHDFS delegation token kind when ssl is enabled in HttpFS

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.7.3, 3.0.0-alpha3
    • Fix Version/s: 3.0.0-beta1, 2.10.0
    • Component/s: httpfs, webhdfs
    • Labels:
      None

      Description

      When httpfs runs with httpfs.ssl.enabled it should return SWEBHDFS delegation tokens.

      Currently, httpfs returns WEBHDFS delegation "kind" for tokens regardless of whether ssl is enabled or not. If clients directly connect to renew tokens (for example, hdfs dfs) all works because httpfs doesn't check whether token kind is for swebhdfs or webhdfs. However, this breaks when yarn rm needs to renew the token for the job (for example, when running hadoop distcp). Since DT kind is WEBHDFS, rm tries to establish non-ssl connection to httpfs and fails.

      I've tested a simple patch which I'll upload to this jira, and it fixes this issue (hadoop distcp works).

        Attachments

        1. HDFS-12052.00.patch
          1 kB
          Zoran Dimitrijevic
        2. HDFS-12052.01.patch
          19 kB
          Zoran Dimitrijevic
        3. HDFS-12052.02.patch
          21 kB
          Zoran Dimitrijevic
        4. HDFS-12052.03.patch
          31 kB
          Zoran Dimitrijevic
        5. HDFS-12052.04.patch
          32 kB
          Zoran Dimitrijevic
        6. HDFS-12052.05.patch
          30 kB
          Zoran Dimitrijevic
        7. HDFS-12052.06.patch
          21 kB
          Zoran Dimitrijevic
        8. HDFS-12052.07.patch
          21 kB
          Zoran Dimitrijevic

          Activity

            People

            • Assignee:
              3opan Zoran Dimitrijevic
              Reporter:
              3opan Zoran Dimitrijevic
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: