[HBASE-5498] Secure Bulk Load

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.94.5, 0.95.0
• Component/s: security
• Hadoop Flags: Reviewed

Release Note

This feature adds a new optional configuration parameter:

hbase.bulkload.staging.dir

This defines the path on DFS that HBase will use to create random/secret directories under.
The default location is /tmp/hbase-staging

• Assignee: Francis Christopher Liu
• Reporter: Francis Christopher Liu
• Created: 01/Mar/12 00:20
• Updated: 07/Feb/14 23:11
• Resolved: 15/Jan/13 23:54

Description

Design doc: https://cwiki.apache.org/confluence/display/HCATALOG/HBase+Secure+Bulk+Load

Short summary:

Security as it stands does not cover the bulkLoadHFiles() feature. Users calling this method will bypass ACLs. Also loading is made more cumbersome in a secure setting because of hdfs privileges. bulkLoadHFiles() moves the data from user's directory to the hbase directory, which would require certain write access privileges set.

Our solution is to create a coprocessor which makes use of AuthManager to verify if a user has write access to the table. If so, launches a MR job as the hbase user to do the importing (ie rewrite from text to hfiles). One tricky part this job will have to do is impersonate the calling user when reading the input files. We can do this by expecting the user to pass an hdfs delegation token as part of the secureBulkLoad() coprocessor call and extend an inputformat to make use of that token. The output is written to a temporary directory accessible only by hbase and then bulkloadHFiles() is called.

Issue Links

• is blocked by: HBASE-6432 HRegionServer doesn't properly set clusterId in conf
• is duplicated by: HBASE-6422 Add switch in LoadIncrementalHFiles API to allow for programatically changing perms on output directory
• is part of: HBASE-6101 Ensure Observers cover all relevant RPC and lifecycle code paths
• is part of: HBASE-6096 AccessController v2