[PIG-1281] Detect org.apache.pig.data.DataByteArray cannot be cast to org.apache.pig.data.Tuple type of errors at Compile Type during creation of logical plan

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.9.0
• Assignee: Thejas Nair
• Reporter: Viraj Bhat
• Created: 06/Mar/10 01:03
• Updated: 04/Aug/11 00:34
• Resolved: 20/Apr/11 22:44

Description

This is more of an enhancement request, where we can detect simple errors during compile time during creation of Logical plan rather than at the backend.

I created a script which contains an error which gets detected in the backend as a cast error when in fact we can detect it in the front end(group is a single element so group.$0 projection operation will not work).

inputdata = LOAD '/user/viraj/mymapdata' AS (co1, col2, col3, col4);

projdata = FILTER inputdata BY (col1 is not null);

groupprojdata = GROUP projdata BY col1;

cleandata = FOREACH groupprojdata {
                     bagproj = projdata.col1;
                     dist_bags = DISTINCT bagproj;
                     GENERATE group.$0 as newcol1, COUNT(dist_bags) as newcol2;
                      };

cleandata1 = GROUP cleandata by newcol2;

cleandata2 = FOREACH cleandata1 { GENERATE group.$0 as finalcol1, COUNT(cleandata.newcol1) as finalcol2; };

ordereddata = ORDER cleandata2 by finalcol2;

store into 'finalresult' using PigStorage();

Issue Links

• is duplicated by: PIG-1992 projecting a column from within non tuple/bag datatype should result in error in frontend
• is related to: PIG-1997 define semantics of referring to a column within bytearray column