[PIG-3799] TestCustomPartitioner is broken in tez branch

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: tez-branch
• Fix Version/s: tez-branch
• Component/s: tez
• Assignee: Cheolsoo Park
• Reporter: Cheolsoo Park
• Created: 06/Mar/14 19:24
• Updated: 21/Nov/14 05:58
• Resolved: 09/Mar/14 01:15

Description

This is a regression of PIG-3795. In TezCompiler, visitDistinct() doesn't set the requested parallelism of TezOperator, resulting that only one reducer runs for the following query-

A = LOAD 'table_testCustomPartitionerDistinct' as (a0:int, a1:int);
B = distinct A PARTITION BY org.apache.pig.test.utils.SimpleCustomPartitioner3 parallel 2;

The test fails because it sees a single output file while it expects two.