Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13553 RBF: Support global quota
  3. HDFS-13710

RBF: setQuota and getQuotaUsage should check the dfs.federation.router.quota.enable

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0
    • 2.10.0, 3.2.0, 3.1.1
    • federation, hdfs
    • None
    • Reviewed

    Description

      when I use the command below, some exceptions happened.

       

      hdfs dfsrouteradmin -setQuota /tmp -ssQuota 1G 
      

       the logs follow.

      Successfully set quota for mount point /tmp
      

      It looks like the quota is set successfully, but some exceptions happen in the rbf server log.

      java.io.IOException: No remote locations available
      at org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:1002)
      at org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:967)
      at org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:940)
      at org.apache.hadoop.hdfs.server.federation.router.Quota.setQuota(Quota.java:84)
      at org.apache.hadoop.hdfs.server.federation.router.RouterAdminServer.synchronizeQuota(RouterAdminServer.java:255)
      at org.apache.hadoop.hdfs.server.federation.router.RouterAdminServer.updateMountTableEntry(RouterAdminServer.java:238)
      at org.apache.hadoop.hdfs.protocolPB.RouterAdminProtocolServerSideTranslatorPB.updateMountTableEntry(RouterAdminProtocolServerSideTranslatorPB.java:179)
      at org.apache.hadoop.hdfs.protocol.proto.RouterProtocolProtos$RouterAdminProtocolService$2.callBlockingMethod(RouterProtocolProtos.java:259)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2115)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2111)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:422)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1867)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2111)
      

      I find the dfs.federation.router.quota.enable is false by default. And it causes the problem. I think we should check the parameter when we call setQuota and getQuotaUsage. 

       

       

      Attachments

        1. HDFS-13710.002.patch
          5 kB
          yanghuafeng
        2. HDFS-13710.003.patch
          6 kB
          yanghuafeng
        3. HDFS-13710.004.patch
          6 kB
          yanghuafeng
        4. HDFS-13710.005.patch
          6 kB
          yanghuafeng
        5. HDFS-13710.006.patch
          6 kB
          yanghuafeng
        6. HDFS-13710.007.patch
          6 kB
          yanghuafeng
        7. HDFS-13710.008.patch
          6 kB
          Yiqun Lin
        8. HDFS-13710.patch
          3 kB
          yanghuafeng

        Activity

          People

            hfyang20071 yanghuafeng
            hfyang20071 yanghuafeng
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: