Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3461

HFTP should use the same port & protocol for getting the delegation token

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      Currently, hftp uses http to the Namenode's https port, which doesn't work.

      1. h3461_20120924.patch
        17 kB
        Tsz Wo Nicholas Sze
      2. h3461_20120925.patch
        18 kB
        Tsz Wo Nicholas Sze
      3. hdfs-3461-branch-1.patch
        35 kB
        Owen O'Malley
      4. hdfs-3461-branch-1.patch
        34 kB
        Owen O'Malley
      5. hdfs-3461-doAs.patch
        1 kB
        Owen O'Malley

        Issue Links

          Activity

          Hide
          Daryn Sharp added a comment -

          We test hftp pretty heavily and haven't seen this bug. Which branch has this issue?

          Show
          Daryn Sharp added a comment - We test hftp pretty heavily and haven't seen this bug. Which branch has this issue?
          Hide
          Owen O'Malley added a comment -

          This is the 1.1 branch with HDFS-2617 applied.

          Show
          Owen O'Malley added a comment - This is the 1.1 branch with HDFS-2617 applied.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h3461_20120924.patch: Owen's patch. I only made it apply to branch-1.

          Show
          Tsz Wo Nicholas Sze added a comment - h3461_20120924.patch: Owen's patch. I only made it apply to branch-1.
          Hide
          Owen O'Malley added a comment -

          Unfortunately, the HDFS-2617 patch that went in made the KSSL configurable instead of removing it and thus this patch needs to follow suit. I'll refactor the current patch to use ssl when hadoop.security.use-weak-http-crypto is set to true and security is enabled.

          Show
          Owen O'Malley added a comment - Unfortunately, the HDFS-2617 patch that went in made the KSSL configurable instead of removing it and thus this patch needs to follow suit. I'll refactor the current patch to use ssl when hadoop.security.use-weak-http-crypto is set to true and security is enabled.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          h3461_20120925.patch: updates TestHftpFileSystem. Otherwise, it will fail.

          Show
          Tsz Wo Nicholas Sze added a comment - h3461_20120925.patch: updates TestHftpFileSystem. Otherwise, it will fail.
          Hide
          Daryn Sharp added a comment -

          To follow on with Owen's comments, we can't remove the secure port methods. They need to be conditionalized based on whether kssl or spnego is enabled.

          Show
          Daryn Sharp added a comment - To follow on with Owen's comments, we can't remove the secure port methods. They need to be conditionalized based on whether kssl or spnego is enabled.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Thanks Daryn. Will leave this with Owen.

          Show
          Tsz Wo Nicholas Sze added a comment - Thanks Daryn. Will leave this with Owen.
          Hide
          Eli Collins added a comment -

          Hftp using http to the Namenode's https port doesn't work on trunk and branch-2 either, I've got a fix for that up on HDFS-3983.

          Show
          Eli Collins added a comment - Hftp using http to the Namenode's https port doesn't work on trunk and branch-2 either, I've got a fix for that up on HDFS-3983 .
          Hide
          Eli Collins added a comment -

          Also, this patch needs to support KSSL anyway otherwise distcp'ing from a pre-SPNEGO clusters using Hftp will fail. Ie we'd have to force all secure users to upgrade to SPNEGO.

          Show
          Eli Collins added a comment - Also, this patch needs to support KSSL anyway otherwise distcp'ing from a pre-SPNEGO clusters using Hftp will fail. Ie we'd have to force all secure users to upgrade to SPNEGO.
          Hide
          Owen O'Malley added a comment -

          Ok, I've tested this patch with:

          • kssl
          • spnego
          • spnego + ssl

          For all three cases, I've tested

          • fetchdt get, renew, cancel
          • hftp
          • hsftp

          To work with SSL, I also needed to fix HDFS-3993.

          Show
          Owen O'Malley added a comment - Ok, I've tested this patch with: kssl spnego spnego + ssl For all three cases, I've tested fetchdt get, renew, cancel hftp hsftp To work with SSL, I also needed to fix HDFS-3993 .
          Hide
          Owen O'Malley added a comment -

          I had to fix a couple of the test cases to reflect the new service name in the tokens.

          Show
          Owen O'Malley added a comment - I had to fix a couple of the test cases to reflect the new service name in the tokens.
          Hide
          Jitendra Nath Pandey added a comment -

          The patch removes the ugi.doAs wrapper in HftpFileSystem.getDelegationToken, is that intended? The ugi in the filesystem could be different from the current user and that would change the existing behavior.

          Show
          Jitendra Nath Pandey added a comment - The patch removes the ugi.doAs wrapper in HftpFileSystem.getDelegationToken, is that intended? The ugi in the filesystem could be different from the current user and that would change the existing behavior.
          Hide
          Owen O'Malley added a comment -

          Good catch. In most cases, it doesn't matter since the getDelegationToken is called by the constructor, but it will in general. Here's the delta patch to put the doAs back.

          Show
          Owen O'Malley added a comment - Good catch. In most cases, it doesn't matter since the getDelegationToken is called by the constructor, but it will in general. Here's the delta patch to put the doAs back.
          Hide
          Jitendra Nath Pandey added a comment -

          The patch with the doAs patch looks ok to me.
          +1.

          Show
          Jitendra Nath Pandey added a comment - The patch with the doAs patch looks ok to me. +1.
          Hide
          Matt Foley added a comment -

          Fix was committed by Owen on 10/1/2012.

          Show
          Matt Foley added a comment - Fix was committed by Owen on 10/1/2012.
          Hide
          Matt Foley added a comment -

          Closed upon release of Hadoop-1.1.0.

          Show
          Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.

            People

            • Assignee:
              Owen O'Malley
              Reporter:
              Owen O'Malley
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development