[HIVE-10209] FetchTask with VC may fail because ExecMapper.done is true

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.2.0
• Component/s: Query Processor
• Assignee: Chao Sun
• Reporter: Chao Sun
• Created: 03/Apr/15 05:59
• Updated: 25/Sep/15 06:27
• Resolved: 09/Apr/15 22:30

Description

ExecMapper.done is a static variable, and may cause issues in the following example:

set hive.fetch.task.conversion=minimal;
select * from src where key < 10 limit 1;
set hive.fetch.task.conversion=more;
select *, BLOCK__OFFSET_INSIDE__FILE from src where key < 10;

The second select won't return any result, if running in local mode.

The issue is, the first select query will be converted to a MapRedTask with only a mapper. And, when the task is done, because of the limit operator, ExecMapper.done will be set to true.

Then, when the second select query begin to execute, it will call FetchOperator::getRecordReader(), and since here we have virtual column, an instance of HiveRecordReader will be returned. The problem is, HiveRecordReader::doNext() will check ExecMapper.done. In this case, since the value is true, it will quit immediately.

In short, I think making ExecMapper.done static is a bad idea. The first query should in no way affect the second one.

Issue Links

• is part of: HIVE-10134 Fix test failures after HIVE-10130 [Spark Branch]