[HBASE-14821] CopyTable should allow overriding more config properties for peer cluster

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0, 1.3.0, 2.0.0
• Component/s: mapreduce

Release Note

Configuration properties for org.apache.hadoop.hbase.mapreduce.TableOutputFormat can now be overridden by prefixing the property keys with "hbase.mapred.output.". When the configuration is applied to TableOutputFormat, these entries will be rewritten with the prefix removed -- ie. "hbase.mapred.output.hbase.security.authentication" becomes "hbase.security.authentication". This can be useful when directing output to a peer cluster with different security configuration, for example.

• Assignee: Gary Helmling
• Reporter: Gary Helmling
• Created: 16/Nov/15 21:57
• Updated: 25/Nov/15 02:27
• Resolved: 24/Nov/15 20:29

Description

When using CopyTable across two separate clusters, you can specify the ZK quorum for the destination cluster, but not much else in configuration overrides. This can be a problem when the cluster configurations differ, such as when using security with different configurations for server principals.

We should provide a general way to override configuration properties for the peer / destination cluster. One option would be to allow use of a prefix for command line properties ("peer.property."). Properties matching this prefix will be stripped and merged to the peer configuration.