[HIVE-17261] Hive use deprecated ParquetInputSplit constructor which blocked parquet dictionary filter

• Type: Improvement
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Fix Version/s: 3.2.0
• Component/s: Database/Schema
• Target Version/s: 2.3.0
• Assignee: Junjie Chen Assign to me
• Reporter: Junjie Chen
• Created: 07/Aug/17 08:31
• Updated: 27/Jun/18 18:30

Description

Hive use deprecated ParquetInputSplit in https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java#L128

Please see interface definition in https://github.com/apache/parquet-mr/blob/master/parquet-hadoop/src/main/java/org/apache/parquet/hadoop/ParquetInputSplit.java#L80

Old interface set rowgroupoffset values which will lead to skip dictionary filter in parquet.

Issue Links

• Blocked: HIVE-17593 DataWritableWriter strip spaces for CHAR type before writing, but predicate generator doesn't do same thing.
• links to: RB