[HIVE-11117] Hive external table - skip header and trailer property issue

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Labels: check mapreduce

Environment

Production

• Assignee: Unassigned Assign to me
• Reporter: Janarthanan
• Created: 25/Jun/15 22:37
• Updated: 06/Apr/24 09:46

Description

I am using an external hive table pointing to a HDFS location. The external table is partitioned on year/mm/dd folders. When there are more than one partition folder (ex: /2015/01/02/file.txt & /2015/01/03/file2.txt), the select on external table, skips the DATA RECORD instead of skipping the header/trailer record from one of the file).

tblproperties ("skip.header.line.count"="1");

Resolution: On enabling hive.input format instead of text input format and execution using TEZ engine instead of MapReduce resovled the issue.

How to resolve the problem without setting these parameters ? I don't want to run the hive query using TEZ.