Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20869

Endpoint-based Export use incorrect user to write to destination

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 3.0.0, 2.2.0, 2.1.1, 2.0.2
    • Component/s: Coprocessors
    • Labels:
      None
    • Environment:

      Hadoop 3.0.0 + HBase 2.0.0, Kerberos.

    • Hadoop Flags:
      Reviewed

      Description

      HBASE-15806 implemented an endpoint based export. It gets caller's HDFS delegation token, and RegionServer is supposed to write out exported files as the caller.

      Everything works fine if you use run export as hbase user. However, once you use a different user to export, it fails.

      To reproduce,

      Add to configuration key hbase.coprocessor.region.classes the coprocessor class org.apache.hadoop.hbase.coprocessor.Export.

      create a table t1, assign permission to a user foo:

       

      hbase(main):004:0> user_permission 't1'
      User Namespace,Table,Family,Qualifier:Permission
      hbase default,t1,,: [Permission: actions=READ,WRITE,EXEC,CREATE,ADMIN]
      foo default,t1,,: [Permission: actions=READ,WRITE,EXEC,CREATE,ADMIN]

       

      As user foo, execute the following command:

       

      $ hdfs dfs -mkdir /tmp/export_hbase2
      $ hbase org.apache.hadoop.hbase.coprocessor.Export t1 /tmp/export_hbase2/t2/
      ....
      18/07/10 14:03:59 INFO client.RpcRetryingCallerImpl: Call exception, tries=6, retries=6, started=4457 ms ago, cancelled=false, msg=org.apache.hadoop.security.AccessControlException: Permission denied: user=hbase, access=WRITE, inode="/tmp/export_hbase2/t2":foo:supergroup:drwxr-xr-x
      at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:400)
      at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:256)
      at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:194)
      at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1846)
      at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1830)
      at org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkAncestorAccess(FSDirectory.java:1789)
      at org.apache.hadoop.hdfs.server.namenode.FSDirWriteFileOp.resolvePathForStartFile(FSDirWriteFileOp.java:316)
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFileInt(FSNamesystem.java:2411)
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startFile(FSNamesystem.java:2343)
      at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.create(NameNodeRpcServer.java:764)
      at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.create(ClientNamenodeProtocolServerSideTranslatorPB.java:451)
      at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
      at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
      at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:422)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
      
      at sun.reflect.GeneratedConstructorAccessor25.newInstance(Unknown Source)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121)
      at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88)
      at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:278)
      at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1195)
      at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1174)
      at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1112)
      at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:462)
      at org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:459)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:473)
      at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:400)
      at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1103)
      at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:1168)
      at org.apache.hadoop.io.SequenceFile.createWriter(SequenceFile.java:285)
      at org.apache.hadoop.hbase.coprocessor.Export$SecureWriter.<init>(Export.java:445)
      at org.apache.hadoop.hbase.coprocessor.Export.processData(Export.java:214)
      at org.apache.hadoop.hbase.coprocessor.Export.export(Export.java:338)
      at org.apache.hadoop.hbase.protobuf.generated.ExportProtos$ExportService.callMethod(ExportProtos.java:2030)
      at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:8007)
      at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:2409)
      at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:2391)
      at org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42010)
      at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:409)
      at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:130)
      at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:324)
      at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:304)

      This is not a problem in unit test, because HBase runs as the same user as HDFS superuser, so it always has the permission to write.

      I have a fix and a test code. Chia-Ping Tsai could you help review?

      The doc could use some help too. Not very easy to follow for first time user. I'll file a doc Jira for that.

       

        Attachments

        1. HBASE-20869.master.002.patch
          3 kB
          Wei-Chiu Chuang
        2. HBASE-20869.master.001.patch
          3 kB
          Wei-Chiu Chuang

          Issue Links

            Activity

              People

              • Assignee:
                jojochuang Wei-Chiu Chuang
                Reporter:
                jojochuang Wei-Chiu Chuang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: