Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1033

In secure clusters, NN and SNN should verify that the remote principal during image and edits transfer

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: namenode, security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently anyone can connect and download image/edits from Namenode. In a secure cluster we can verify the identity of the principal making the request; we should disallow requests from anyone except the NN and SNN principals (and their hosts due to the lousy KerbSSL limitation).

      1. HDFS-1033-Y20.patch
        9 kB
        Jakob Homan
      2. HDFS-1033.patch
        4 kB
        Jakob Homan
      3. HDFS-1033-2.patch
        7 kB
        Jakob Homan
      4. HDFS-1033-3.patch
        8 kB
        Jakob Homan

        Activity

        Jakob Homan created issue -
        Jakob Homan made changes -
        Field Original Value New Value
        Summary In securre clusters, NN and SNN should verify that the remote principal during image and edits transfer In secure clusters, NN and SNN should verify that the remote principal during image and edits transfer
        Hide
        Jakob Homan added a comment -

        Y! 20 patch, not for commit. Trunk patch soon...

        Show
        Jakob Homan added a comment - Y! 20 patch, not for commit. Trunk patch soon...
        Jakob Homan made changes -
        Attachment HDFS-1033-Y20.patch [ 12438477 ]
        Hide
        Devaraj Das added a comment -

        Looks good

        Show
        Devaraj Das added a comment - Looks good
        Hide
        Jakob Homan added a comment -

        Patch for trunk. Straight forward port of patch, but shouldn't be committed yet as a couple other patches went in first. This patch is smaller than the Y20 patch as a bunch of changes to the keys that had been in this one were leapfrogged by HDFS-1163, it appears.

        Show
        Jakob Homan added a comment - Patch for trunk. Straight forward port of patch, but shouldn't be committed yet as a couple other patches went in first. This patch is smaller than the Y20 patch as a bunch of changes to the keys that had been in this one were leapfrogged by HDFS-1163 , it appears.
        Jakob Homan made changes -
        Attachment HDFS-1033.patch [ 12448523 ]
        Hide
        Jakob Homan added a comment -

        Updated patch to include unit test. This patch actually doesn't depend on anything else that's not gone in yet, so it can go in once it gets a clean bill of health.

        Show
        Jakob Homan added a comment - Updated patch to include unit test. This patch actually doesn't depend on anything else that's not gone in yet, so it can go in once it gets a clean bill of health.
        Jakob Homan made changes -
        Attachment HDFS-1033-2.patch [ 12448531 ]
        Hide
        Jakob Homan added a comment -

        submitting patch.

        Show
        Jakob Homan added a comment - submitting patch.
        Jakob Homan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Fix Version/s 0.22.0 [ 12314241 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448531/HDFS-1033-2.patch
        against trunk revision 959792.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448531/HDFS-1033-2.patch against trunk revision 959792. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/418/console This message is automatically generated.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12448531/HDFS-1033-2.patch
        against trunk revision 959874.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448531/HDFS-1033-2.patch against trunk revision 959874. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/419/console This message is automatically generated.
        Hide
        Jakob Homan added a comment -

        I can't reproduce the failed tests. They're not related, although they are concerning. Patch is ready for review.

        Show
        Jakob Homan added a comment - I can't reproduce the failed tests. They're not related, although they are concerning. Patch is ready for review.
        Hide
        Jakob Homan added a comment -

        Jitendra noticed I missed the SNN part of the patch, which is actually blocked by several other patches. Canceling patch.

        Show
        Jakob Homan added a comment - Jitendra noticed I missed the SNN part of the patch, which is actually blocked by several other patches. Canceling patch.
        Jakob Homan made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Jakob Homan added a comment -

        Updated patch with SNN fixes. Some lines had already been done by other patches, so are not present here.

        Show
        Jakob Homan added a comment - Updated patch with SNN fixes. Some lines had already been done by other patches, so are not present here.
        Jakob Homan made changes -
        Attachment HDFS-1033-3.patch [ 12449128 ]
        Hide
        Jakob Homan added a comment -

        submitting patch.

        Show
        Jakob Homan added a comment - submitting patch.
        Jakob Homan made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Jitendra Nath Pandey added a comment -

        +1 for the patch.

        Show
        Jitendra Nath Pandey added a comment - +1 for the patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12449128/HDFS-1033-3.patch
        against trunk revision 962696.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12449128/HDFS-1033-3.patch against trunk revision 962696. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/213/console This message is automatically generated.
        Hide
        Jakob Homan added a comment -

        Test failures are the usual, unrelated suspects. I've committed this. Resolving as fixed.

        Show
        Jakob Homan added a comment - Test failures are the usual, unrelated suspects. I've committed this. Resolving as fixed.
        Jakob Homan made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Jakob Homan made changes -
        Affects Version/s 0.22.0 [ 12314241 ]
        Component/s name-node [ 12312926 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #339 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/339/)
        HDFS-1033. In secure clusters, NN and SNN should verify that the remote principal during image and edits transfer.

        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #339 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/339/ ) HDFS-1033 . In secure clusters, NN and SNN should verify that the remote principal during image and edits transfer.
        Konstantin Shvachko made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        6d 2h 3m 1 Jakob Homan 08/Jul/10 01:51
        Open Open Patch Available Patch Available
        114d 19h 59m 2 Jakob Homan 09/Jul/10 22:34
        Patch Available Patch Available Resolved Resolved
        21h 2m 1 Jakob Homan 10/Jul/10 19:37
        Resolved Resolved Closed Closed
        519d 10h 42m 1 Konstantin Shvachko 12/Dec/11 06:19

          People

          • Assignee:
            Jakob Homan
            Reporter:
            Jakob Homan
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development