[PIG-5203] Partitioner E2E test fails on spark

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: Ádám Szita
• Reporter: Ádám Szita
• Created: 29/Mar/17 13:50
• Updated: 21/Jun/17 09:18
• Resolved: 07/Apr/17 01:41

Description

Cause of failure is the difference between expected and actual result:

Actual:
18 160
19 164
20 166
...
Expected:
18 80
18 80
19 82
19 82
....

Seems like although we specify PARTITION BY org.apache.pig.test.utils.SimpleCustomPartitioner2 parallel 2 in the query it has no effect in Spark mode.