[PIG-1277] Pig should give error message when cogroup on tuple keys of different inner type

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.9.0
• Component/s: impl
• Hadoop Flags: Reviewed
• Assignee: Alan Gates
• Reporter: Daniel Dai
• Created: 05/Mar/10 19:02
• Updated: 04/Aug/11 00:35
• Resolved: 24/Dec/10 01:32

Description

When we cogroup on a tuple, if the inner type of tuple does not match, we treat them as different keys. This is confusing. It is desirable to give error/warnings when it happens.

Here is one example:
UDF:

public class MapGenerate extends EvalFunc<Map> {
    @Override
    public Map exec(Tuple input) throws IOException {
        // TODO Auto-generated method stub
        Map m = new HashMap();
        m.put("key", new Integer(input.size()));
        return m;
    }
    
    @Override
    public Schema outputSchema(Schema input) {
        return new Schema(new Schema.FieldSchema(null, DataType.MAP));
    }
}

Pig script:

a = load '1.txt' as (a0);
b = foreach a generate a0, MapGenerate(*) as m:map[];
c = foreach b generate a0, m#'key' as key;
d = load '2.txt' as (c0, c1);
e = cogroup c by (a0, key), d by (c0, c1);
dump e;

1.txt

1

2.txt

1 1

User expected result (which is not right):

((1,1),{(1,1)},{(1,1)})

Real result:

((1,1),{(1,1)},{})
((1,1),{},{(1,1)})

We shall give user the message that we can not merge the key due to the type mismatch.

Issue Links

• is duplicated by: PIG-999 sorting on map-value fails if map-value is not of bytearray type
• is duplicated by: PIG-1065 In-determinate behaviour of Union when there are 2 non-matching schema's