HBase
  1. HBase
  2. HBASE-11311

Secure Bulk Load does not execute chmod 777 on the files

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.99.0, 0.98.2
    • Fix Version/s: 0.99.0, 0.98.4
    • Component/s: security
    • Labels:
      None

      Description

      With HBASE-10902 we lost the setPermission() on the hfiles that we are moving, resulting in a Permission denied when we try to remove the file

      Permission denied: user=hbase, access=WRITE, inode="/hbase/.../3de9fdb2a99e4ef9937b4622317d10e0_SeqId_2_":testuser:hadoop:-rwxr-xr-x
      
       for(Pair<byte[], String> el: familyPaths) {
      -   Path p = new Path(el.getSecond());
      -   LOG.trace("Setting permission for: " + p);
      -   fs.setPermission(p, PERM_ALL_ACCESS);
      
      1. HBASE-11311-v0.patch
        1.0 kB
        Matteo Bertozzi

        Issue Links

          Activity

          Hide
          Enis Soztutar added a comment -

          Closing this issue after 0.99.0 release.

          Show
          Enis Soztutar added a comment - Closing this issue after 0.99.0 release.
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #311 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/311/)
          HBASE-11311 Secure Bulk Load does not execute chmod 777 on the files (matteo.bertozzi: rev 04db3c37620a66fdcce35ebf4ce36d5785a9f4a9)

          • hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #311 (See https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/311/ ) HBASE-11311 Secure Bulk Load does not execute chmod 777 on the files (matteo.bertozzi: rev 04db3c37620a66fdcce35ebf4ce36d5785a9f4a9) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in HBase-0.98 #329 (See https://builds.apache.org/job/HBase-0.98/329/)
          HBASE-11311 Secure Bulk Load does not execute chmod 777 on the files (matteo.bertozzi: rev 04db3c37620a66fdcce35ebf4ce36d5785a9f4a9)

          • hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
          Show
          Hudson added a comment - SUCCESS: Integrated in HBase-0.98 #329 (See https://builds.apache.org/job/HBase-0.98/329/ ) HBASE-11311 Secure Bulk Load does not execute chmod 777 on the files (matteo.bertozzi: rev 04db3c37620a66fdcce35ebf4ce36d5785a9f4a9) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #5188 (See https://builds.apache.org/job/HBase-TRUNK/5188/)
          HBASE-11311 Secure Bulk Load does not execute chmod 777 on the files (matteo.bertozzi: rev 20874896623694070443ad06e69c302d169c1446)

          • hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5188 (See https://builds.apache.org/job/HBase-TRUNK/5188/ ) HBASE-11311 Secure Bulk Load does not execute chmod 777 on the files (matteo.bertozzi: rev 20874896623694070443ad06e69c302d169c1446) hbase-server/src/main/java/org/apache/hadoop/hbase/security/access/SecureBulkLoadEndpoint.java
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12649439/HBASE-11311-v0.patch
          against trunk revision .
          ATTACHMENT ID: 12649439

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12649439/HBASE-11311-v0.patch against trunk revision . ATTACHMENT ID: 12649439 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/9728//console This message is automatically generated.
          Hide
          Ted Yu added a comment -

          +1

          Show
          Ted Yu added a comment - +1
          Hide
          Andrew Purtell added a comment -

          +1

          Show
          Andrew Purtell added a comment - +1
          Hide
          Matteo Bertozzi added a comment -

          ok, looks like that adding back the 3 lines is enough,
          tested on a single cluster secure/unsecure (the dir to bulkload is on the same hdfs as /hbase)
          and different clusters secure only (the dir to bulkload is on cluster1 and /hbase is on cluster2)

          Show
          Matteo Bertozzi added a comment - ok, looks like that adding back the 3 lines is enough, tested on a single cluster secure/unsecure (the dir to bulkload is on the same hdfs as /hbase) and different clusters secure only (the dir to bulkload is on cluster1 and /hbase is on cluster2)
          Hide
          Andrew Purtell added a comment -

          Lesson learned: don't touch any existing behavior unless 200% you know what you are doing

          We also didn't catch that line deletion but should have Jerry. Our/my bad.

          Show
          Andrew Purtell added a comment - Lesson learned: don't touch any existing behavior unless 200% you know what you are doing We also didn't catch that line deletion but should have Jerry. Our/my bad.
          Hide
          Jerry He added a comment -

          The reason I didn't see this is that my 'hbase' is in the dfs superusergroup
          Lesson learned: don't touch any existing behavior unless 200% you know what you are doing

          Show
          Jerry He added a comment - The reason I didn't see this is that my 'hbase' is in the dfs superusergroup Lesson learned: don't touch any existing behavior unless 200% you know what you are doing
          Hide
          Jerry He added a comment -

          Thanks, Matteo
          That makes sense.

          Show
          Jerry He added a comment - Thanks, Matteo That makes sense.
          Hide
          Matteo Bertozzi added a comment -

          the error is on the hbase side when the file will be archived (compaction, table drop and so on).
          but the problem is that the file is not owned or accessible to the hbase user as before.

          Show
          Matteo Bertozzi added a comment - the error is on the hbase side when the file will be archived (compaction, table drop and so on). but the problem is that the file is not owned or accessible to the hbase user as before.
          Hide
          Jerry He added a comment -

          Hi, Matteo

          The error happened during bulk loading, or happened in HBase log?

          Show
          Jerry He added a comment - Hi, Matteo The error happened during bulk loading, or happened in HBase log?

            People

            • Assignee:
              Matteo Bertozzi
              Reporter:
              Matteo Bertozzi
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development