[HIVE-19996] Beeline performance poor with drivers having slow DatabaseMetaData.getPrimaryKeys impl

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.2.1
• Component/s: Beeline

Environment

Issue detected using Beeline with HBase Phoenix thin driver and a result set with many columns.

• Assignee: Kevin Minder
• Reporter: Kevin Minder
• Created: 26/Jun/18 14:18
• Updated: 26/Jun/18 14:42

Description

Beeline performance is rather poor for table output format when two conditions occur for the same result set.

1. The result set has a large number of columns.
2. The driver being used has a slow implementation of DatabaseMetaData.getPrimaryKeys.

For example testing has shown that for a query with ~100 columns using the HBase Phoenix thin driver the execution time can be cut from ~30 seconds to ~2 seconds by using CSV output format vs table output format. For example: select * from system.catalog;

This is due to how primary keys are detected. Currently the Rows implementation will make a metadata call for every column to determine it is a primary key for display purposes. I propose optimizing this such that a metadata call is only made for each unique table in the result set's columns.