[DRILL-1346] Use HBase table size information to improve scan parallelization

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.5.0
• Component/s: Storage - HBase
• Assignee: Aditya Kishore
• Reporter: Aditya Kishore
• Created: 27/Aug/14 07:07
• Updated: 10/Oct/14 07:09
• Resolved: 31/Aug/14 17:40

Description

Currently we use a pseudo-estimated value to calculate the scan size which does not take the actual size of data into account.

HBase, through o.a.h.h.client.HBaseAdmin.getClusterStatus(), provides a way to retrieve the actual data size of each region. We can use this to approximate the size of scan and use it to improve the scan parallelization.

Issue Links

• is duplicated by: DRILL-535 HbaseGroupScan should return more accurate size information.