Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-12052

BulkLoad Failed due to no write permission on input files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.99.0, 0.98.6
    • Fix Version/s: 0.98.7, 0.99.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      SecureBulkLoadEndPoint can be used in un-secure env to bulk load data without hitting "Permission denied" for hbase user.

      Description

      The issue is that HBase bulkload is done by Region Server which normally runs under hbase user while the input hfile folder & the user starts the bulkload could be any user.

      Below is the error message when user "hrt_qa" bulkload files which "hrt_qa" has the write permission while the bulkload operation still fail with "Permission denied" error.

      We had similar handling for this issue in secure env so the proposed fix is to reuse SecureBulkLoadEndPoint in un-secure env as well. In the future, we can rename the class to BulkLoadEndPoint.

      java.io.IOException: Exception in rename
              at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.rename(HRegionFileSystem.java:947)
              at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.commitStoreFile(HRegionFileSystem.java:347)
              at org.apache.hadoop.hbase.regionserver.HRegionFileSystem.bulkLoadStoreFile(HRegionFileSystem.java:421)
              at org.apache.hadoop.hbase.regionserver.HStore.bulkLoadHFile(HStore.java:723)
              at org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:3603)
              at org.apache.hadoop.hbase.regionserver.HRegion.bulkLoadHFiles(HRegion.java:3525)
              at org.apache.hadoop.hbase.regionserver.HRegionServer.bulkLoadHFile(HRegionServer.java:3276)
              at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:28863)
              at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2008)
              at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:92)
              at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.consumerLoop(SimpleRpcScheduler.java:160)
              at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler.access$000(SimpleRpcScheduler.java:38)
              at org.apache.hadoop.hbase.ipc.SimpleRpcScheduler$1.run(SimpleRpcScheduler.java:110)
              at java.lang.Thread.run(Thread.java:662)
      Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/tmp/a0f3ee35-4c8f-4077-93d0-94d8e5bae914/0":hrt_qa:hdfs:drwxr-xr-x
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:265)
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:251)
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:232)
              at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:179)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:5515)
      

        Attachments

        1. HBASE-12052.patch
          12 kB
          Jeffrey Zhong

          Activity

            People

            • Assignee:
              jeffreyz Jeffrey Zhong
              Reporter:
              jeffreyz Jeffrey Zhong
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: