Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24800

Enhance ACL region initialization

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: acl, MTTR
    • Labels:
      None

      Description

      RegionServer persist ACL table entries into Zookeeper during ACL region open,

            private void initialize(RegionCoprocessorEnvironment e) throws IOException {
          final Region region = e.getRegion();
          Configuration conf = e.getConfiguration();
          Map<byte[], ListMultimap<String, UserPermission>> tables = PermissionStorage.loadAll(region);
          // For each table, write out the table's permissions to the respective
          // znode for that table.
          for (Map.Entry<byte[], ListMultimap<String, UserPermission>> t:
            tables.entrySet()) {
            byte[] entry = t.getKey();
            ListMultimap<String, UserPermission> perms = t.getValue();
            byte[] serialized = PermissionStorage.writePermissionsAsBytes(perms, conf);
            zkPermissionWatcher.writeToZookeeper(entry, serialized);
          }
          initialized = true;
        }
      

      Currently RegionServer send 2 RPC (one to create the table path and another to set the data) for each table sequentially.

       try {
            ZKUtil.createWithParents(watcher, zkNode);
            ZKUtil.updateExistingNodeData(watcher, zkNode, permsData, -1);
          } catch (KeeperException e) {
            LOG.error("Failed updating permissions for entry '" +
                entryName + "'", e);
            watcher.abort("Failed writing node "+zkNode+" to zookeeper", e);
          }
      

      If a cluster have huge number of tables then ACL region open will take time. Example, it took ~9 min to write 60k tables ACL into ZK. 

      We should send ZK Ops in a single multi() to enhance this.

        Attachments

          Activity

            People

            • Assignee:
              pankajkumar Pankaj Kumar
              Reporter:
              pankajkumar Pankaj Kumar
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: