[HIVE-8497] StatsNoJobTask doesn't close RecordReader, FSDataInputStream of which keeps open to prevent stale data clean

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.14.0
• Fix Version/s: 0.14.0

Environment

Windows

• Assignee: Xiaobing Zhou
• Reporter: Xiaobing Zhou
• Created: 17/Oct/14 18:30
• Updated: 18/Nov/14 22:05
• Resolved: 20/Oct/14 21:44

Description

run the test

mvn -Phadoop-2  test -Dtest=TestCliDriver -Dqfile=alter_merge_stats_orc.q

to reproduce it. Simply, this query does three data loads which generates three base orc files.
ANALYZE TABLE...COMPUTE STATISTICS NOSCAN will execute StatsNoJobTask to get stats, where file handle is held so as not able to clean base file. As a result, after running ALTER TABLE..CONCATENATE, follow-up queries go to stale base file and merged file.

Issue Links

• links to: REVIEW BOARD