[PIG-1068] COGROUP fails with 'Type mismatch in key from map: expected org.apache.pig.impl.io.NullableText, recieved org.apache.pig.impl.io.NullableTuple'

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.6.0
• Assignee: Richard Ding
• Reporter: Vikram Oberoi
• Created: 30/Oct/09 20:47
• Updated: 24/Mar/10 22:15
• Resolved: 03/Dec/09 22:57

Description

The COGROUP in the following script fails in its map:

logs = LOAD '$LOGS' USING PigStorage() AS (ts:int, id:chararray, command:chararray, comments:chararray);                                                                                                       
                                                                                                                                                                                                               
SPLIT logs INTO logins IF command == 'login', all_quits IF command == 'quit';                                                                                                                                  
                                                                                                                                                                                                               
-- Project login clients and count them by ID.                                                                                                                                                                 
login_info = FOREACH logins {                                                                                                                                                                                  
    GENERATE id as id,                                                                                                                                                                                         
    comments AS client;                                                                                                                                                                                        
};                                                                                                                                                                                                             
                                                                                                                                                                                                               
logins_grouped = GROUP login_info BY (id, client);                                                                                                                                                             
                                                                                                                                                                                                               
count_logins_by_client = FOREACH logins_grouped {                                                                                                                                                              
    generate group.id AS id, group.client AS client, COUNT($1) AS count;                                                                                                                                       
}                                                                                                                                                                                                              
                                                                                                                                                                                                               
-- Get the first quit.                                                                                                                                                                                         
all_quits_grouped = GROUP all_quits BY id;                                                                                                                                                                     
                                                                                                                                                                                                               
quits = FOREACH all_quits_grouped {                                                                                                                                                                            
    ordered = ORDER all_quits BY ts ASC;                                                                                                                                                                       
    last_quit = LIMIT ordered 1;                                                                                                                                                                               
    GENERATE FLATTEN(last_quit);                                                                                                                                                                               
}                                                                                                                                                                                                              
                                                                                                                                                                                                               
-- Now, group all the info together.                                                                                                                                                                           
joined_session_info = COGROUP quits BY id, count_logins_by_client BY id;                                                                                                                                       
                                                                                                                                                                                                               
DUMP joined_session_info;

Here's the stack trace:

java.io.IOException: Type mismatch in key from map: expected org.apache.pig.impl.io.NullableText, recieved org.apache.pig.impl.io.NullableTuple
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:415)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.collect(PigMapReduce.java:108)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:229)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce$Map.map(PigMapReduce.java:93)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:157)