[PIG-3051] java.lang.IndexOutOfBoundsException failure with LimitOptimizer + ColumnPruning

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0, 0.11
• Fix Version/s: 0.11, 0.12.0
• Component/s: parser
• Assignee: Koji Noguchi
• Reporter: Koji Noguchi
• Created: 15/Nov/12 17:21
• Updated: 22/Feb/13 04:53
• Resolved: 27/Dec/12 22:12

Description

Had a user hitting
"Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1" error when he had multiple stores and limit in his code.

I couldn't reproduce this with short pig code (due to ColumnPruning somehow not happening when shortened), but here's a snippet.

...
G3 = FOREACH G2 GENERATE sortCol, FLATTEN(group) as label, (long)COUNT(G1) as cnt;
G4 = ORDER G3 BY cnt DESC PARALLEL 25;
ONEROW = LIMIT G4 1;
U1 = FOREACH ONEROW GENERATE 3 as sortcol, 'somelabel' as label, cnt;
store U1 into 'u1' using PigStorage();
store G4 into 'g4' using PigStorage();

With '-t ColumnMapKeyPrune', job didn't hit the error.