[HBASE-15773] CellCounter improvements

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0, 1.3.0
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: mapreduce

Release Note

The CellCounter map reduce job now supports additional configuration options on the Scan instance it creates, using the org.apache.hadoop.hbase.mapreduce.TableInputFormat defined property names. For a full list of the options, run ./hbase org.apache.hadoop.hbase.mapreduce.CellCounter with no arguments.

CellCounter also no longer creates job counters for per-rowkey and per-rowkey/qualifier cell counts. For most tables, these counters would cause the job to fail due to mapreduce job counter limits.

• Assignee: Gary Helmling
• Reporter: Gary Helmling
• Created: 05/May/16 17:51
• Updated: 24/Jun/22 18:51
• Resolved: 06/May/16 18:27

Description

Looking at the CellCounter map reduce, it seems like it can be improved in a few areas:

• it does not currently support setting scan batching. This is important when we're fetching all versions for columns. Actually, it would be nice to support all of the scan configuration currently provided in TableInputFormat.
• generating job counters containing row keys and column qualifiers is guaranteed to blow up on anything but the smallest table. This is not usable and doesn't make any sense when the same counts are in the job output. The row and qualifier specific counters should be dropped.

Issue Links

• is duplicated by: HBASE-13707 CellCounter uses to many counters