[IMPALA-3776] 'describe formatted' broken for avro tables with mismatching column information in hive metastore

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.5.0
• Fix Version/s: Impala 2.7.0, Impala 2.6.2
• Component/s: Frontend
• Target Version: Impala 2.7.0, Impala 2.6.2
• Assignee: Lars Volker
• Reporter: Lars Volker
• Created: 23/Jun/16 05:38
• Updated: 08/Sep/16 16:23
• Resolved: 26/Aug/16 17:26

Description

For Avro tables the column information in the underlying database of the Hive metastore can be different from what is specified in the 'avro.schema.url' file. HIVE-6308 aimed to improve upon this, but for older tables the two don't necessarily align.

There are two possible cases:

1. Hive's underlying database contains a column, which is not present in the Avro schema file. In this case we encounter a NullPointerException in DescribeResultFactory.java#L189 when trying to lookup the column in our internal table.
2. The avro schema contains a column, which is not present in the underlying database. In this case the column will not be displayed in describe formatted.

Issue Links

• is related to: IMPALA-3782 'describe formatted' output does not match Hive's output for Avro tables