[HBASE-17130] Add support to specify an arbitrary number of reducers when writing HFiles for bulk load

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: mapreduce
• Assignee: Esteban Gutierrez
• Reporter: Esteban Gutierrez
• Created: 18/Nov/16 23:32
• Updated: 13/Nov/17 05:28

Description

From the discussion from HBASE-16894 there is a set of use cases where writing to multiple regions in a single reducer can be helpful to reduce the overhead of MR jobs when a large number of regions exist in an HBase cluster and some regions can present a data skew, e.g. 100s or 1000s of regions with a very small number of rows vs. regions with 10s or millions or rows as part of the same job. And merging regions is not an option for the use case.

Issue Links

• is duplicated by: HBASE-19226 Limit the reduce tasks number of incremental load
• relates to: HBASE-16894 Create more than 1 split per region, generalize HBASE-12590