[PIG-4438] Limit after sort does not work in spark mode

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: spark-branch
• Component/s: spark
• Assignee: liyunzhang
• Reporter: liyunzhang
• Created: 02/Mar/15 03:13
• Updated: 21/Jun/17 09:18
• Resolved: 07/Apr/15 09:26

Description

when pig script executes "order" before "limit" in spark mode, the results will be wrong.
cat testlimit.txt
1 orange
3 coconut
5 grape
6 pear
2 apple
4 mango

testlimit.pig:
a = load './testlimit.txt' as (x:int, y:chararray);
b = order a by x;
c = limit b 1;
store c into './testlimit.out';

the result:
1 orange
2 apple
3 coconut
4 mango
5 grape
6 pear

the correct result should be:
1 orange

Issue Links

• links to: review board