[PIG-558] Distinct followed by a Join results in Invalid size 0 for a tuple error

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.2.0
• Fix Version/s: 0.2.0
• Component/s: impl
• Assignee: Pradeep Kamath
• Reporter: Viraj Bhat
• Created: 06/Dec/08 02:28
• Updated: 24/Mar/10 22:04
• Resolved: 06/Jan/09 01:24

Description

The following Pig script does a right outer join after the DISTINCT.

nonuniqtable1 = LOAD 'table1' AS (f1:chararray);
table1 = DISTINCT nonuniqtable1;
table2 = LOAD 'table2' AS (f1:chararray, f2:int);
temp = COGROUP table1 BY f1 INNER, table2 BY f1;
DESCRIBE temp;
explain temp;
dump temp;

========================================================================================================
It results in the following error. This is true for other join types as well.
========================================================================================================
java.io.IOException: Invalid size 0 for a tuple
at org.apache.pig.data.DataReaderWriter.readDatum(DataReaderWriter.java:57)
at org.apache.pig.data.DataReaderWriter.readDatum(DataReaderWriter.java:62)
at org.apache.pig.builtin.BinStorage.getNext(BinStorage.java:90)
at org.apache.pig.backend.executionengine.PigSlice.next(PigSlice.java:103)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:157)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.SliceWrapper$1.next(SliceWrapper.java:133)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:165)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:45)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
========================================================================================================