-
Type:
Sub-task
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: None
-
Fix Version/s: spark-branch
-
Component/s: spark
-
Labels:None
Cause of failure is the difference between expected and actual result:
Actual:
18 160
19 164
20 166
...
Expected:
18 80
18 80
19 82
19 82
....
Seems like although we specify PARTITION BY org.apache.pig.test.utils.SimpleCustomPartitioner2 parallel 2 in the query it has no effect in Spark mode.