Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17861

Regionserver down when checking the permission of staging dir if hbase.rootdir is on S3

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.4.0, 1.3.3, 1.2.7
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Some object store does not support unix style permission, this jira fix the permission check issue when specify staging dir in different file system. Currently it covers s3, wasb, swift.

      Description

      Found some issue, when set up HBASE-17437: Support specifying a WAL directory outside of the root directory.

      The region server are showdown when I add following config into hbase-site.xml
      hbase.rootdir = s3a://xx//xx
      hbase.wal.dir = hdfs://xx/xx
      hbase.coprocessor.region.classes = org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint
      Error is below

      org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint threw java.lang.IllegalStateException: Directory already exists but permissions aren't set to '-rwx--x--x'
      

      The reason is that, when hbase enable securebulkload, hbase will create a folder in s3, it can not set above permission, because in s3, all files are listed as having full read/write permissions and all directories appear to have full rwx permissions. See Object stores have differerent authorization models in https://hadoop.apache.org/docs/current3/hadoop-aws/tools/hadoop-aws/index.html

        Attachments

        1. HBASE-17861.branch-1.V1.patch
          2 kB
          Yi Liang
        2. HBASE-17861.branch-1.V2.patch
          2 kB
          Yi Liang
        3. HBASE-17861.branch-1.V3.patch
          3 kB
          Yi Liang
        4. HBASE-17861.branch-1.V4.patch
          3 kB
          Yi Liang
        5. HBASE-17861-V1.patch
          2 kB
          Yi Liang

          Issue Links

            Activity

              People

              • Assignee:
                easyliangjob Yi Liang
                Reporter:
                easyliangjob Yi Liang
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: