Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11069

Tighten the authorization of datanode RPC

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      The current implementation of checkSuperuserPrivilege() allows the datanode user from any node to be recognized as a super user. If one datanode is compromised, the intruder can issue shutdownDatanode(), evictWriters(), triggerBlockReport(), etc. against all other datanodes. Although this does not expose stored data, it can cause service disruptions.

      This needs to be tightened to allow only the local datanode user.

        Issue Links

          Activity

          Hide
          kihwal Kihwal Lee added a comment -
          private void checkSuperuserPrivilege() {
          ...
              if (callerUgi.getShortUserName().equals(dnUserName)) {
                return;
              }
          ...
          

          Instead of checking only the short name, the full name should be checked. E.g. dn_user/datanode01.yourdomain.com@YOURDOMAIN.COM instead of simply dn_user.

          Show
          kihwal Kihwal Lee added a comment - private void checkSuperuserPrivilege() { ... if (callerUgi.getShortUserName().equals(dnUserName)) { return ; } ... Instead of checking only the short name, the full name should be checked. E.g. dn_user/datanode01.yourdomain.com@YOURDOMAIN.COM instead of simply dn_user .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 mvninstall 6m 46s trunk passed
          +1 compile 0m 44s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 41s trunk passed
          +1 javadoc 0m 38s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 42s the patch passed
          +1 javac 0m 42s the patch passed
          +1 checkstyle 0m 25s the patch passed
          +1 mvnsite 0m 48s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 48s the patch passed
          +1 javadoc 0m 35s the patch passed
          -1 unit 81m 4s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          99m 28s



          Reason Tests
          Failed junit tests hadoop.hdfs.TestFileCorruption
            hadoop.security.TestPermission



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:9560f25
          JIRA Issue HDFS-11069
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835615/HDFS-11069.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux eba215e54b61 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / ac35ee9
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17326/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17326/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17326/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 6m 46s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 38s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 42s the patch passed +1 javac 0m 42s the patch passed +1 checkstyle 0m 25s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 35s the patch passed -1 unit 81m 4s hadoop-hdfs in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 99m 28s Reason Tests Failed junit tests hadoop.hdfs.TestFileCorruption   hadoop.security.TestPermission Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-11069 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12835615/HDFS-11069.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux eba215e54b61 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / ac35ee9 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17326/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17326/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17326/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          TestPermission is broken by HDFS-10455.

          The other test passes.

          -------------------------------------------------------
           T E S T S
          -------------------------------------------------------
          Running org.apache.hadoop.hdfs.TestFileCorruption
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.826 sec - in org.apache.hadoop.hdfs.TestFileCorruption
          
          Results :
          
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
          
          Show
          kihwal Kihwal Lee added a comment - TestPermission is broken by HDFS-10455 . The other test passes. ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.hadoop.hdfs.TestFileCorruption Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.826 sec - in org.apache.hadoop.hdfs.TestFileCorruption Results : Tests run: 5, Failures: 0, Errors: 0, Skipped: 0
          Hide
          daryn Daryn Sharp added a comment -

          +1 Good change to reduce privilege escalation.

          Show
          daryn Daryn Sharp added a comment - +1 Good change to reduce privilege escalation.
          Hide
          kihwal Kihwal Lee added a comment -

          Thanks for the review, Daryn. I've verified it working as expected (allow local, deny remote) on a secure cluster.

          Show
          kihwal Kihwal Lee added a comment - Thanks for the review, Daryn. I've verified it working as expected (allow local, deny remote) on a secure cluster.
          Hide
          kihwal Kihwal Lee added a comment -

          Committed to trunk, branch-2, branch-2.8 and branch-2.7.

          Show
          kihwal Kihwal Lee added a comment - Committed to trunk, branch-2, branch-2.8 and branch-2.7.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10708 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10708/)
          HDFS-11069. Tighten the authorization of datanode RPC. Contributed by (kihwal: rev ae48c496dce8d0eae4571fc64e6850d602bae688)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10708 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10708/ ) HDFS-11069 . Tighten the authorization of datanode RPC. Contributed by (kihwal: rev ae48c496dce8d0eae4571fc64e6850d602bae688) (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          xkrogen Erik Krogen added a comment -

          Hey Kihwal Lee, it looks like you missed some fix versions (2.8.? and I think probably 2.9) when you committed, can you update them?

          Show
          xkrogen Erik Krogen added a comment - Hey Kihwal Lee , it looks like you missed some fix versions (2.8.? and I think probably 2.9) when you committed, can you update them?
          Hide
          jojochuang Wei-Chiu Chuang added a comment -

          Hi Kihwal Lee, I'm just curious, for security concerns, should NameNode also tighten its RPC authorization as well? Any reason why not? One reason might be the NameNode HA, but I wonder if there are other rationales too. Thanks.

          Show
          jojochuang Wei-Chiu Chuang added a comment - Hi Kihwal Lee , I'm just curious, for security concerns, should NameNode also tighten its RPC authorization as well? Any reason why not? One reason might be the NameNode HA, but I wonder if there are other rationales too. Thanks.
          Hide
          kihwal Kihwal Lee added a comment -

          Erik Krogen. Fixed. Once it was a convention to not include never-been-released lines in the fix version field at the time of closing jira. This no longer is the case.

          Wei-Chiu Chuang In terms of user authorization, a hdfs superuser for one namenode should also be a superuser for the other namenode and datanodes. A datanode user shouldn't be a privileged user and allowing one DN user to have the admin permission on other DNs was giving it more privilege than needed.

          Show
          kihwal Kihwal Lee added a comment - Erik Krogen . Fixed. Once it was a convention to not include never-been-released lines in the fix version field at the time of closing jira. This no longer is the case. Wei-Chiu Chuang In terms of user authorization, a hdfs superuser for one namenode should also be a superuser for the other namenode and datanodes. A datanode user shouldn't be a privileged user and allowing one DN user to have the admin permission on other DNs was giving it more privilege than needed.
          Hide
          xkrogen Erik Krogen added a comment -

          Ah, thank you for the context, Kihwal. I am too new for that

          Show
          xkrogen Erik Krogen added a comment - Ah, thank you for the context, Kihwal. I am too new for that

            People

            • Assignee:
              kihwal Kihwal Lee
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development