[HIVE-11233] Include Apache Phoenix support in HBaseStorageHandler

• Type: New Feature
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.2.1, 2.0.0
• Component/s: HBase Handler
• Labels: Binary Hbase Numeric Phoenix Sortable
• Target Version/s: 3.0.0

Release Note

Thanks of this patch are possible read or write operations for HBase table in Apache Phoenix persistence mode. Following classes could be considered as 3 keypoints:
- org.apache.hadoop.hive.hbase.phoenix.PhoenixValueFactory
- org.apache.hadoop.hive.hbase.phoenix.PhoenixKeyFactory
- org.apache.hadoop.hive.hbase.phoenix.PhoenixCompositeKeyFactory

Note: See the code sample provided in description field of that issue for more details.

• Flags: Patch
• Assignee: Svetozar Ivanov
• Reporter: Svetozar Ivanov
• Created: 12/Jul/15 13:04
• Updated: 27/Feb/24 22:23

Description

Currently HBaseStorageHandler doesn't provide mechanism for storage of binary sortable key and values. It is necessary when given HBase table is used for persistence by Apache Hive and Apache Phoenix. In that way all byte arrays read or written by Hive will be compatible with binary sortable format used in Phoenix.

It turns out the major difference is in all numeric data types accordingly officially provided documentation - https://phoenix.apache.org/language/datatypes.html.

That's how I use it in my code with applied patch:

    private static String buildWithSerDeProperties(TableDescriptor tableDescriptor) {
        Map<String, String> serdePropertiesMap = new HashMap<>();

        serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_NAME, tableDescriptor.getTableName());
        serdePropertiesMap.put(HBaseSerDe.HBASE_TABLE_DEFAULT_STORAGE_TYPE, BINARY_STORAGE_TYPE);
        serdePropertiesMap.put(HBaseSerDe.HBASE_COLUMNS_MAPPING, buildHBaseColumnsDefinition(tableDescriptor));


        serdePropertiesMap.put(HBaseSerDe.HBASE_VALUE_FACTORY_CLASS, PhoenixValueFactory.class.getName());
        /* Use different key factory for simple and composite primary key */
        if (tableDescriptor.getPkDescriptors().size() == 1) {
            serdePropertiesMap.put(HBaseSerDe.HBASE_KEY_FACTORY_CLASS, PhoenixKeyFactory.class.getName());
        } else {
            serdePropertiesMap.put(HBaseSerDe.HBASE_COMPOSITE_KEY_FACTORY, PhoenixCompositeKeyFactory.class.getName());
        }

        String serDeProperties = serdePropertiesMap.entrySet().stream()
                .map(e -> quoteInSingleQuotes(e.getKey()) + " = " + quoteInSingleQuotes(e.getValue()))
                .collect(Collectors.joining(COLUMNS_SEPARATOR));

        logger.debug("SERDEPROPERTIES are [{}]", serDeProperties);
        return serDeProperties;
    }