Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5498

Secure Bulk Load

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.5, 0.95.0
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      This feature adds a new optional configuration parameter:

      hbase.bulkload.staging.dir

      This defines the path on DFS that HBase will use to create random/secret directories under.
      The default location is /tmp/hbase-staging
      Show
      This feature adds a new optional configuration parameter: hbase.bulkload.staging.dir This defines the path on DFS that HBase will use to create random/secret directories under. The default location is /tmp/hbase-staging

      Description

      Design doc: https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load

      Short summary:

      Security as it stands does not cover the bulkLoadHFiles() feature. Users calling this method will bypass ACLs. Also loading is made more cumbersome in a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data from user's directory to the hbase directory, which would require certain write access privileges set.

      Our solution is to create a coprocessor which makes use of AuthManager to verify if a user has write access to the table. If so, launches a MR job as the hbase user to do the importing (ie rewrite from text to hfiles). One tricky part this job will have to do is impersonate the calling user when reading the input files. We can do this by expecting the user to pass an hdfs delegation token as part of the secureBulkLoad() coprocessor call and extend an inputformat to make use of that token. The output is written to a temporary directory accessible only by hbase and then bulkloadHFiles() is called.

        Attachments

        1. HBASE-5498_draft.patch
          22 kB
          Francis Christopher Liu
        2. HBASE-5498_draft_94.patch
          18 kB
          Francis Christopher Liu
        3. HBASE-5498_94.patch
          37 kB
          Francis Christopher Liu
        4. HBASE-5498_94.patch
          47 kB
          Francis Christopher Liu
        5. HBASE-5498_trunk.patch
          46 kB
          Francis Christopher Liu
        6. HBASE-5498_trunk_2.patch
          50 kB
          Francis Christopher Liu
        7. HBASE-5498_94_2.patch
          49 kB
          Francis Christopher Liu
        8. HBASE-5498_94_3.patch
          50 kB
          Francis Christopher Liu
        9. HBASE-5498_trunk_3.patch
          50 kB
          Francis Christopher Liu
        10. HBASE-5498_trunk_4.patch
          222 kB
          Francis Christopher Liu
        11. HBASE-5498_trunk_2.patch
          223 kB
          Ted Yu
        12. HBASE-5498_trunk_5.patch
          223 kB
          Francis Christopher Liu
        13. HBASE-5498_94_3.patch
          52 kB
          Francis Christopher Liu

          Issue Links

            Activity

              People

              • Assignee:
                toffer Francis Christopher Liu
                Reporter:
                toffer Francis Christopher Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                27 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: