[HIVE-13584] HBaseStorageHandler should support table pre-split

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0
• Component/s: HBase Handler
• Labels: HBase PreSplit
• Target Version/s: 3.0.0

Release Note

- This patch gives opportunity for creation of pre-splitted HBase table via Hive. Via TBLPROPERTIES could be configured following for HBase table:
- Splits number
- Split algorithm (if is not configured by default would be used 'org.apache.hadoop.hbase.util.RegionSplitter.UniformSplit' algorithm)
- Split algorithm first row key
- Split algorithm last row key

For more details see provided example in description field of this issue.

• Assignee: Svetozar Ivanov
• Reporter: Svetozar Ivanov
• Created: 21/Apr/16 19:30
• Updated: 27/Feb/24 22:23

Description

Currently is not possible creation of HBase table via HBaseStorageHandler with specified pre-split strategy.

I use it in my code with applied patch in that way:

public String buildCreateTblStatement(TableDescriptor tableDescriptor) {
        StringBuilder createTableStatement = ...;

        ...
        ...
        String tblProperties = buildTblProperties(tableDescriptor);
        if (StringUtils.isNotBlank(tblProperties)) {
             createTableStatement.append(" TBLPROPERTIES (" + tblProperties + ")");
        }
         ...
        return createTableStatement.toString();
}


private static String buildTblProperties(TableDescriptor tableDescriptor) {

        ...
        ...

        if (supportsPreSplit(tableDescriptor)) {
            sb.append(", ");
            sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLITS_NUMBER));
            sb.append(" = ");
            sb.append(quoteInSingleQuotes(String.valueOf(tableDescriptor.getNumRegions())));

            GeneratedPKDescriptor columnDescriptor = (GeneratedPKDescriptor) tableDescriptor.getPkDescriptors().get(0);
            sb.append(", ");
            sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_FIRSTROW_KEY));
            sb.append(" = ");
            sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().firstRowKey()));
            sb.append(", ");
            sb.append(quoteInSingleQuotes(HBaseStorageHandler.SPLIT_ALGORITHM_LASTROW_KEY));
            sb.append(" = ");
            sb.append(quoteInSingleQuotes(columnDescriptor.getPkGenerator().lastRowKey()));
        }

        logger.trace("TBLPROPERTIES are [{}]", sb);
        return sb.toString();
}