[PIG-5230] Fix the RuntimeException throws in SecondaryKeySortUtil

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: liyunzhang
• Reporter: liyunzhang
• Created: 27/Apr/17 06:31
• Updated: 28/Apr/17 08:55
• Resolved: 28/Apr/17 08:55

Description

there is possibility that curKey is null after PIG-5164. we should remove the code to avoid RuntimeException.

following script can trigger the exception.

a = load './studenttab10k.mk1' as (name, age:int, gpa:float);
a1 = filter a by gpa is null or gpa >= 3.9;
a2 = filter a by gpa < 2;
b = union a1, a2;
c = load './voternulltab10k' as (name, age, registration, contributions);
d = join b by name left outer, c by name using 'replicated';
e = stream d through `cat` as (name, age, gpa, name1, age1, registration, contributions);
f = foreach e generate name, age, gpa, registration, contributions;
g = group f by name;
g1 = group f by name; -- Two separate groupbys to ensure secondary key partitioner
h = foreach g { 
    inner1 = order f by age, gpa, registration, contributions;
    inner2 = limit inner1 1;
    generate inner2, SUM(f.age); };
i = foreach g1 {
    inner1 = order f by age asc, gpa desc, registration asc, contributions desc;
    inner2 = limit inner1 1;
    generate inner2, SUM(f.age); };
store h into './MultiQuery_Union_3.1.out';
store i into './MultiQuery_Union_3.2.out';

cat studenttab10k.mk1

ulysses thompson	64	1.90
katie carson	25	3.65
	65	0.73
holly davidson	57	2.43
fred miller	55	3.77